* Re: [BUG] md: oops sync_page_io+0x38/0x180
From: Ming Lei @ 2017-06-23 11:16 UTC (permalink / raw)
To: linux-raid; +Cc: Shaohua Li
In-Reply-To: <20170623104744.GA5830@ming.t460p>
On Fri, Jun 23, 2017 at 06:47:44PM +0800, Ming Lei wrote:
> Hi,
>
> When I boot a VM, the following kernel oops is triggerd:
>
> [ 4.850206] BUG: unable to handle kernel NULL pointer dereference at 00000000000006a0
> [ 4.851131] IP: sync_page_io+0x38/0x180
> [ 4.851445] PGD 2759bc067
> [ 4.851446] P4D 2759bc067
> [ 4.851621] PUD 277838067
> [ 4.851835] PMD 0
>
> [ 4.852152] Oops: 0000 [#1] PREEMPT SMP
> [ 4.852494] Dumping ftrace buffer:
> [ 4.852758] (ftrace buffer empty)
> [ 4.853062] Modules linked in: nd_pmem psmouse serio_raw ahci libahci floppy nvme nvme_core ib_iser rdma_cm iw_cm ib_cm ib_core configfs iscsi_tcp libiscsi_tcp libiscsi scsi_transport_iscsi null_blk configs autofs4
> [ 4.855190] CPU: 3 PID: 661 Comm: mdadm Not tainted 4.12.0-rc6.quiesce-v3-next-20170623-09867-ga73468728fd8 #248
> [ 4.856310] Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.10.2-0-g5f4c7b1-prebuilt.qemu-project.org 04/01/2014
> [ 4.857458] task: ffffa33bf749b340 task.stack: ffffa667812e8000
> [ 4.857992] RIP: 0010:sync_page_io+0x38/0x180
> [ 4.858349] RSP: 0018:ffffa667812ebb18 EFLAGS: 00010297
> [ 4.858804] RAX: 0000000000000000 RBX: 0000000000000000 RCX: ffffd43fc9da2100
> [ 4.859465] RDX: 0000000000000000 RSI: 0000000000000000 RDI: ffffa33bf740d800
> [ 4.860137] RBP: ffffa667812ebb50 R08: 0000000000000000 R09: 0000000000000000
> [ 4.860774] R10: 0000000023386c39 R11: 0000000000000000 R12: ffffa33bf740d800
> [ 4.861817] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000001
> [ 4.863150] FS: 00007f0aa0304700(0000) GS:ffffa33bfac00000(0000) knlGS:0000000000000000
> [ 4.864412] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
> [ 4.865001] CR2: 00000000000006a0 CR3: 0000000276883000 CR4: 00000000003406e0
> [ 4.865663] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> [ 4.866332] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
> [ 4.867025] Call Trace:
> [ 4.867178] read_disk_sb.constprop.66+0x72/0xc0
> [ 4.867572] super_1_load+0xb7/0x7a0
> [ 4.867897] ? blkdev_get_by_dev+0x58/0x70
> [ 4.868222] ? lock_rdev+0x6c/0xb0
> [ 4.868476] md_import_device+0x187/0x230
> [ 4.868851] ? check_preemption_disabled+0x35/0x120
> [ 4.869266] add_new_disk+0xc2/0x760
> [ 4.869541] md_ioctl+0x1fc7/0x23a0
> [ 4.869824] ? __might_fault+0x67/0xd0
> [ 4.870109] ? autostart_arrays+0x710/0x710
> [ 4.870451] blkdev_ioctl+0x5b8/0xbd0
> [ 4.870750] block_ioctl+0x61/0x80
> [ 4.871003] ? blkdev_fallocate+0x240/0x240
> [ 4.871371] do_vfs_ioctl+0xb0/0x890
> [ 4.871640] ? check_preemption_disabled+0x35/0x120
> [ 4.872061] ? entry_SYSCALL_64_fastpath+0x5/0xc2
> [ 4.872463] ? __this_cpu_preempt_check+0x1c/0x20
> [ 4.872867] ? __fget_light+0x56/0xb0
> [ 4.873148] ? security_file_ioctl+0x62/0x80
> [ 4.873498] SyS_ioctl+0x94/0xc0
> [ 4.873735] entry_SYSCALL_64_fastpath+0x23/0xc2
> [ 4.874121] RIP: 0033:0x7f0a9fe26687
> [ 4.874391] RSP: 002b:00007ffca16ec628 EFLAGS: 00000246 ORIG_RAX: 0000000000000010
> [ 4.875105] RAX: ffffffffffffffda RBX: 00007ffca16efa6b RCX: 00007f0a9fe26687
> [ 4.875779] RDX: 00007ffca16eca38 RSI: 0000000040140921 RDI: 0000000000000004
> [ 4.876544] RBP: ffffffffa35cbbcc R08: 0000000000200000 R09: 0000000000000014
> [ 4.877253] R10: 0000000000000361 R11: 0000000000000246 R12: ffffa667812ebf88
> [ 4.877968] R13: 0000000000000004 R14: 00007ffca16ec8cd R15: 0000000001db35b0
> [ 4.878657] ? __this_cpu_preempt_check+0x1c/0x20
> [ 4.879341] Code: 41 55 41 54 49 89 fc 53 49 89 f5 45 89 ce 48 83 ec 10 89 55 d0 48 89 4d c8 44 89 45 d4 44 8b 7d 10 e8 7d 63 91 ff 49 8b 44 24 18 <48> 8b 98 a0 06 00 00 48 85 db 0f 84 12 01 00 00 e8 63 63 91 ff
> [ 4.885691] RIP: sync_page_io+0x38/0x180 RSP: ffffa667812ebb18
> [ 4.886208] CR2: 00000000000006a0
> [ 4.886441] ---[ end trace 5e7fbbf3076a4aab ]---
Looks this oops is caused by the following patch:
5a85071c2cbc md: use a separate bio_set for synchronous IO.
Once it is reverted, my VM boots successfully without this oops.
Thanks,
Ming
^ permalink raw reply
* [BUG] md: oops sync_page_io+0x38/0x180
From: Ming Lei @ 2017-06-23 10:47 UTC (permalink / raw)
Cc: linux-raid, Shaohua Li
Hi,
When I boot a VM, the following kernel oops is triggerd:
[ 4.850206] BUG: unable to handle kernel NULL pointer dereference at 00000000000006a0
[ 4.851131] IP: sync_page_io+0x38/0x180
[ 4.851445] PGD 2759bc067
[ 4.851446] P4D 2759bc067
[ 4.851621] PUD 277838067
[ 4.851835] PMD 0
[ 4.852152] Oops: 0000 [#1] PREEMPT SMP
[ 4.852494] Dumping ftrace buffer:
[ 4.852758] (ftrace buffer empty)
[ 4.853062] Modules linked in: nd_pmem psmouse serio_raw ahci libahci floppy nvme nvme_core ib_iser rdma_cm iw_cm ib_cm ib_core configfs iscsi_tcp libiscsi_tcp libiscsi scsi_transport_iscsi null_blk configs autofs4
[ 4.855190] CPU: 3 PID: 661 Comm: mdadm Not tainted 4.12.0-rc6.quiesce-v3-next-20170623-09867-ga73468728fd8 #248
[ 4.856310] Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.10.2-0-g5f4c7b1-prebuilt.qemu-project.org 04/01/2014
[ 4.857458] task: ffffa33bf749b340 task.stack: ffffa667812e8000
[ 4.857992] RIP: 0010:sync_page_io+0x38/0x180
[ 4.858349] RSP: 0018:ffffa667812ebb18 EFLAGS: 00010297
[ 4.858804] RAX: 0000000000000000 RBX: 0000000000000000 RCX: ffffd43fc9da2100
[ 4.859465] RDX: 0000000000000000 RSI: 0000000000000000 RDI: ffffa33bf740d800
[ 4.860137] RBP: ffffa667812ebb50 R08: 0000000000000000 R09: 0000000000000000
[ 4.860774] R10: 0000000023386c39 R11: 0000000000000000 R12: ffffa33bf740d800
[ 4.861817] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000001
[ 4.863150] FS: 00007f0aa0304700(0000) GS:ffffa33bfac00000(0000) knlGS:0000000000000000
[ 4.864412] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 4.865001] CR2: 00000000000006a0 CR3: 0000000276883000 CR4: 00000000003406e0
[ 4.865663] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 4.866332] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
[ 4.867025] Call Trace:
[ 4.867178] read_disk_sb.constprop.66+0x72/0xc0
[ 4.867572] super_1_load+0xb7/0x7a0
[ 4.867897] ? blkdev_get_by_dev+0x58/0x70
[ 4.868222] ? lock_rdev+0x6c/0xb0
[ 4.868476] md_import_device+0x187/0x230
[ 4.868851] ? check_preemption_disabled+0x35/0x120
[ 4.869266] add_new_disk+0xc2/0x760
[ 4.869541] md_ioctl+0x1fc7/0x23a0
[ 4.869824] ? __might_fault+0x67/0xd0
[ 4.870109] ? autostart_arrays+0x710/0x710
[ 4.870451] blkdev_ioctl+0x5b8/0xbd0
[ 4.870750] block_ioctl+0x61/0x80
[ 4.871003] ? blkdev_fallocate+0x240/0x240
[ 4.871371] do_vfs_ioctl+0xb0/0x890
[ 4.871640] ? check_preemption_disabled+0x35/0x120
[ 4.872061] ? entry_SYSCALL_64_fastpath+0x5/0xc2
[ 4.872463] ? __this_cpu_preempt_check+0x1c/0x20
[ 4.872867] ? __fget_light+0x56/0xb0
[ 4.873148] ? security_file_ioctl+0x62/0x80
[ 4.873498] SyS_ioctl+0x94/0xc0
[ 4.873735] entry_SYSCALL_64_fastpath+0x23/0xc2
[ 4.874121] RIP: 0033:0x7f0a9fe26687
[ 4.874391] RSP: 002b:00007ffca16ec628 EFLAGS: 00000246 ORIG_RAX: 0000000000000010
[ 4.875105] RAX: ffffffffffffffda RBX: 00007ffca16efa6b RCX: 00007f0a9fe26687
[ 4.875779] RDX: 00007ffca16eca38 RSI: 0000000040140921 RDI: 0000000000000004
[ 4.876544] RBP: ffffffffa35cbbcc R08: 0000000000200000 R09: 0000000000000014
[ 4.877253] R10: 0000000000000361 R11: 0000000000000246 R12: ffffa667812ebf88
[ 4.877968] R13: 0000000000000004 R14: 00007ffca16ec8cd R15: 0000000001db35b0
[ 4.878657] ? __this_cpu_preempt_check+0x1c/0x20
[ 4.879341] Code: 41 55 41 54 49 89 fc 53 49 89 f5 45 89 ce 48 83 ec 10 89 55 d0 48 89 4d c8 44 89 45 d4 44 8b 7d 10 e8 7d 63 91 ff 49 8b 44 24 18 <48> 8b 98 a0 06 00 00 48 85 db 0f 84 12 01 00 00 e8 63 63 91 ff
[ 4.885691] RIP: sync_page_io+0x38/0x180 RSP: ffffa667812ebb18
[ 4.886208] CR2: 00000000000006a0
[ 4.886441] ---[ end trace 5e7fbbf3076a4aab ]---
Thanks,
Ming
^ permalink raw reply
* Re: [PATCH] fs: buffer: Modify alloc_page_buffers.
From: Sean Fu @ 2017-06-23 8:26 UTC (permalink / raw)
To: Al Viro
Cc: shli, anton, jack, axboe, ebiggers, rpeterso, bmarzins,
linux-raid, linux-kernel, linux-fsdevel, linux-ntfs-dev
In-Reply-To: <20170619160315.GD10672@ZenIV.linux.org.uk>
On Mon, Jun 19, 2017 at 05:03:16PM +0100, Al Viro wrote:
> On Mon, Jun 19, 2017 at 09:01:36PM +0800, Sean Fu wrote:
> > Make alloc_page_buffers support circular buffer list and initialise
> > b_state field.
> > Optimize the performance by removing the buffer list traversal to create
> > circular buffer list.
>
> > - bh = head = alloc_page_buffers(page, blocksize, 1);
> > + bh = head = alloc_page_buffers(page, blocksize, 1, 0, 0);
>
> Frankly, I don't like that change of calling conventions; it's very easy to
> mess the order of arguments when using interfaces like that and it's hell
> to find when trying to debug the resulting mess.
>
> Do you really get an observable change in performance? What loads are
> triggering it?
Sorry for my mistake.
Infact, The time of writting a large file depends on saveral other
factors, eg system workload.
Yesterday, I tried to test the time of single calling create_empty_buffers by kretprobe
and found that the performance difference is too small to measure it.
before:
[ 944.632027] create_empty_buffers returned 878736736 and took 2160 ns
to execute
[ 944.632286] create_empty_buffers returned 878962832 and took 451 ns
to execute
[ 944.632302] create_empty_buffers returned 878962016 and took 226 ns
to execute
[ 944.632728] create_empty_buffers returned 878962832 and took 235 ns
to execute
[ 944.633105] create_empty_buffers returned 878962832 and took 167 ns
to execute
[ 944.633421] create_empty_buffers returned 878962832 and took 160 ns
to execute
after:
[39209.076519] create_empty_buffers returned 383804768 and took 1666 ns
to execute
[39209.077032] create_empty_buffers returned 383804768 and took 366 ns
to execute
[39209.077120] create_empty_buffers returned 558412336 and took 179 ns
to execute
[39209.077127] create_empty_buffers returned 558413152 and took 148 ns
to execute
[39209.077525] create_empty_buffers returned 558412336 and took 201 ns
to execute
[39209.078255] create_empty_buffers returned 814328768 and took 880 ns
to execute
[39209.078498] create_empty_buffers returned 558412336 and took 564 ns
to execute
[39209.078737] create_empty_buffers returned 558413152 and took 196 ns
to execute
This patch also complicates code.
Please ignore it.
Thanks all of you.
^ permalink raw reply
* Re: [PATCH v6 0/2] IV Generation algorithms for dm-crypt
From: Herbert Xu @ 2017-06-23 8:13 UTC (permalink / raw)
Cc: broonie, arnd, linux-crypto, linux-kernel, dm-devel, linux-raid,
rnayak, binoy.jayan, David Gstir, Eric Biggers
In-Reply-To: <1498106510-19793-1-git-send-email-binoy.jayan@linaro.org>
Binoy Jayan <binoy.jayan@linaro.org> wrote:
> ===============================================================================
> dm-crypt optimization for larger block sizes
> ===============================================================================
>
> Currently, the iv generation algorithms are implemented in dm-crypt.c. The goal
> is to move these algorithms from the dm layer to the kernel crypto layer by
> implementing them as template ciphers so they can be used in relation with
> algorithms like aes, and with multiple modes like cbc, ecb etc. As part of this
> patchset, the iv-generation code is moved from the dm layer to the crypto layer
> and adapt the dm-layer to send a whole 'bio' (as defined in the block layer)
> at a time. Each bio contains the in memory representation of physically
> contiguous disk blocks. Since the bio itself may not be contiguous in main
> memory, the dm layer sets up a chained scatterlist of these blocks split into
> physically contiguous segments in memory so that DMA can be performed.
There is currently a patch-set for fscrypt to add essiv support. It
would be interesting to know whether your implementation of essiv
can also be used in that patchset. That would confirm that we're on
the right track.
Cheers,
--
Email: Herbert Xu <herbert@gondor.apana.org.au>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt
^ permalink raw reply
* (unknown),
From: mdavis @ 2017-06-23 2:49 UTC (permalink / raw)
To: linux-raid
[-- Attachment #1: 5600669634007.zip --]
[-- Type: application/zip, Size: 5665 bytes --]
^ permalink raw reply
* Re: [PATCH 5/7] x86, boot, LLVM: Use regparm=0 for memcpy and memset
From: H. Peter Anvin @ 2017-06-22 22:38 UTC (permalink / raw)
To: Peter Zijlstra, Michael Davidson
Cc: Michal Marek, Thomas Gleixner, Ingo Molnar, Herbert Xu,
David S. Miller, Shaohua Li, Alexander Potapenko, Dmitry Vyukov,
Matthias Kaehlcke, x86, linux-kbuild, linux-kernel, linux-crypto,
linux-raid
In-Reply-To: <20170317120855.zwqame63yvy5afhg@hirez.programming.kicks-ass.net>
On 03/17/17 05:08, Peter Zijlstra wrote:
> On Thu, Mar 16, 2017 at 05:15:18PM -0700, Michael Davidson wrote:
>> Use the standard regparm=0 calling convention for memcpy and
>> memset when building with clang.
>>
>> This is a work around for a long standing clang bug
>> (see https://llvm.org/bugs/show_bug.cgi?id=3997) where
>> clang always uses the standard regparm=0 calling convention
>> for any implcit calls to memcpy and memset that it generates
>> (eg for structure assignments and initialization) even if an
>> alternate calling convention such as regparm=3 has been specified.
>
> Seriously, fix LLVM already.
>
Yes, this is a real stinker, in no small part because once clang is
fixed to DTRT then this is actually broken...
-hpa
^ permalink raw reply
* Re: [PATCH] md: make suspend range wait timed out
From: NeilBrown @ 2017-06-22 21:54 UTC (permalink / raw)
To: Shaohua Li, Mikulas Patocka; +Cc: linux-raid, Shaohua Li
In-Reply-To: <20170621160704.xyiy6fea4rokag72@kernel.org>
[-- Attachment #1: Type: text/plain, Size: 2535 bytes --]
On Wed, Jun 21 2017, Shaohua Li wrote:
> On Wed, Jun 21, 2017 at 10:09:08AM -0400, Mikulas Patocka wrote:
>>
>>
>> On Mon, 19 Jun 2017, Shaohua Li wrote:
>>
>> > > Write errors only get back to the application if it calls fsync(), and
>> > > many don't do that. Write errors can easily cause a filesystem to go
>> > > read-only, and require an fsck. I think we should be very cautious
>> > > about triggering write errors.
>> > >
>> > > NFS will hang indefinitely rather then return an error if the server is
>> > > not available. That can certainly be annoying, but the alternative has
>> > > been tried, and it leads to random data corruption.
>> > > The two cases are only comparable at a very high level, but I think
>> > > this result should encourage substantial caution.
>> >
>> > It's hard to say if an IO error or an infinite wait is better, but since there
>> > is better option in this case, I don't want to argue. I'll repost a patch to
>> > reset suspend range after a timeout, assume this is your suggestion.
>> >
>> > Thanks,
>> > Shaohua
>>
>> Automatically resetting the suspend range could result in data corruption,
>> so it is even worse than a deadlock.
>
> depending on how you look at this. a deadlock means you will eventually hard
> reset the system, and that will result in data corruption.
But in that circumstance (purely hypothetical at this stage) the
sysadmin knows that something has gone wrong. In the other, they might
not.
Invisible data corruption is much worse that visible.
The suspend functionality is used by user-space programs. If you think
the current interface is not ideal, maybe it would be better to design
an interface that a user-space program can use which is safe to use, but
also fails safe. Maybe it could give the kernel a PID with the meaning
"if you have to invalidate my suspend request, please kill the pid
first". That would have risks of its own of course.
The suspend interface is currently used:
- to enable backup of a region which it is being reshaped in-place.
As time goes on, this will be used less and less as the
change-data-offset approach to reshape doesn't need this.
- to stablise a region while raid6check performs parity calculations
and possibly "corrects" one block.
You at least need to analyse the failure modes of these before you can
justify making any change to the interface they use.
But again, I really don't think there is a problem that needs fixing.
NeilBrown
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 832 bytes --]
^ permalink raw reply
* Re: mdadm goes crazy after changing chunk size
From: d tbsky @ 2017-06-22 8:08 UTC (permalink / raw)
To: linux-raid
In-Reply-To: <CAC6SzHLu35=V88phP_X_OO472NbPXhjzGzraFUMEabSXgz1N0w@mail.gmail.com>
Hi:
I know why mdadm refuse to grow. when I change chunk size, the
disk data-offset increase. how can I recover the space?
I tried "mdadm --grow --data-offset=128M /dev/md2", but get response
"mdadm: --data-offset too small on /dev/sde1"
data-offset result like below:
mdadm -E /dev/sde1
/dev/sde1:
Magic : a92b4efc
Version : 1.2
Feature Map : 0x0
Array UUID : 5b2c25fc:b4ccc860:ba8685fe:5e0433f7
Name : love-1:3 (local to host love-1)
Creation Time : Tue Jun 13 15:21:32 2017
Raid Level : raid5
Raid Devices : 3
Avail Dev Size : 5263812239 (2509.98 GiB 2695.07 GB)
Array Size : 5263812224 (5019.96 GiB 5390.14 GB)
Used Dev Size : 5263812224 (2509.98 GiB 2695.07 GB)
Data Offset : 2550222848 sectors
Super Offset : 8 sectors
Unused Space : before=2550222760 sectors, after=15 sectors
State : clean
Device UUID : 26d406ad:9dd2dcc1:d37a3d36:edb8431e
Update Time : Thu Jun 22 15:54:33 2017
Bad Block Log : 512 entries available at offset 72 sectors
Checksum : 3dceb21f - correct
Events : 5183
Layout : left-symmetric
Chunk Size : 64K
Device Role : Active device 0
Array State : AAA ('A' == active, '.' == missing, 'R' == replacing)
2017-06-21 11:31 GMT+08:00 d tbsky <tbskyd@gmail.com>:
> Hi:
> I want to test the performance of different chunk size, so I
> create a small raid (about 20G~200G) and use command like "mdadm
> --grow -c 64 /dev/md2) to change the chunk size.
>
> after changing chunk size, I found almost every time the raid can
> not re-assemble after reboot.
> the error message shows like "xxx does not have a valid v1.2
> superblock, not importing!".
>
> I found I can use "mdadm --assemble --update=devicesize ....." to
> correct it. so I just continue my testing.
>
> now testing is done, so I want to grow the small raid to full size,
> but I am surprised that the " Used Dev Size" stocked at some stage and
> can not grow to full size. maybe I miss some command parameter?
>
> I think I need to re-create the raid. but maybe someone is
> interested to see what happened. my environment is rhel 7.3 (but
> redhat backport kernel 4.x softwareraid stack to their 3.10 kernel).
>
> I have several testing raidset. below is a 4disk raid6:
> ==========================================================================================
> command "mdadm --detail /dev/md1":
>
> /dev/md1:
> Version : 1.2
> Creation Time : Mon Jun 5 17:58:52 2017
> Raid Level : raid6
> Array Size : 4116416128 (3925.72 GiB 4215.21 GB)
> Used Dev Size : 2058208064 (1962.86 GiB 2107.61 GB)
> Raid Devices : 4
> Total Devices : 4
> Persistence : Superblock is persistent
>
> Update Time : Wed Jun 21 11:06:32 2017
> State : clean
> Active Devices : 4
> Working Devices : 4
> Failed Devices : 0
> Spare Devices : 0
>
> Layout : left-symmetric
> Chunk Size : 64K
>
> Name : localhost.localdomain:pv00
> UUID : cc6a6b68:3d066e91:8bac3ba0:96448f78
> Events : 8180
>
> Number Major Minor RaidDevice State
> 0 8 3 0 active sync /dev/sda3
> 1 8 19 1 active sync /dev/sdb3
> 5 8 51 2 active sync /dev/sdd3
> 4 8 35 3 active sync /dev/sdc3
> ==========================================================================================
> ==========================================================================================
> command "fdisk -lu /dev/sda /dev/sdb /dev/sdc /dev/sdd":
>
> Disk /dev/sda: 4000.8 GB, 4000787030016 bytes, 7814037168 sectors
> Units = sectors of 1 * 512 = 512 bytes
> Sector size (logical/physical): 512 bytes / 4096 bytes
> I/O size (minimum/optimal): 4096 bytes / 4096 bytes
> Disk label type: gpt
>
>
> # Start End Size Type Name
> 1 2048 4095 1M BIOS boot parti
> 2 4096 620543 301M Linux RAID
> 3 620544 7814037134 3.7T Linux RAID
>
> Disk /dev/sdb: 4000.8 GB, 4000787030016 bytes, 7814037168 sectors
> Units = sectors of 1 * 512 = 512 bytes
> Sector size (logical/physical): 512 bytes / 4096 bytes
> I/O size (minimum/optimal): 4096 bytes / 4096 bytes
> Disk label type: gpt
>
>
> # Start End Size Type Name
> 1 2048 4095 1M BIOS boot parti
> 2 4096 620543 301M Linux RAID
> 3 620544 7814037134 3.7T Linux RAID
>
> Disk /dev/sdc: 4000.8 GB, 4000787030016 bytes, 7814037168 sectors
> Units = sectors of 1 * 512 = 512 bytes
> Sector size (logical/physical): 512 bytes / 4096 bytes
> I/O size (minimum/optimal): 4096 bytes / 4096 bytes
> Disk label type: gpt
>
>
> # Start End Size Type Name
> 1 2048 4095 1M BIOS boot parti
> 2 4096 620543 301M Linux RAID
> 3 620544 7814037134 3.7T Linux RAID
>
> Disk /dev/sdd: 4000.8 GB, 4000787030016 bytes, 7814037168 sectors
> Units = sectors of 1 * 512 = 512 bytes
> Sector size (logical/physical): 512 bytes / 4096 bytes
> I/O size (minimum/optimal): 4096 bytes / 4096 bytes
> Disk label type: gpt
>
>
> # Start End Size Type Name
> 1 2048 4095 1M BIOS boot parti
> 2 4096 620543 301M Linux RAID
> 3 620544 7814037134 3.7T Linux RAID
> ==========================================================================================
> ==========================================================================================
> command "mdadm --grow --size=max /dev/md1":
>
> mdadm: component size of /dev/md1 unchanged at 2058208064K
> ==========================================================================================
>
>
> another raid5 is more strange. it assemble correctly after change
> chunk size (this is unusual in my testing environment without
> "--update=devicesize").
> the strange part is the md2 huge bitmap chunk:
> ==========================================================================================
> command "cat /proc/mdstat"
>
> Personalities : [raid1] [raid6] [raid5] [raid4]
> md2 : active raid5 sde1[0] sdf1[1] sdg1[3]
> 5263812224 blocks super 1.2 level 5, 64k chunk, algorithm 2 [3/3] [UUU]
> bitmap: 0/1 pages [0KB], 18014398507384832KB chunk
>
> md0 : active raid1 sdc2[4] sdb2[1] sda2[0] sdd2[5]
> 308160 blocks super 1.0 [4/4] [UUUU]
> bitmap: 0/1 pages [0KB], 65536KB chunk
>
> md1 : active raid6 sdd3[5] sdb3[1] sdc3[4] sda3[0]
> 4116416128 blocks super 1.2 level 6, 64k chunk, algorithm 2 [4/4] [UUUU]
>
> unused devices: <none>
> ==========================================================================================
>
>
> the raid5 device also can not grow to full size:
> ==========================================================================================
> command "mdadm --detail /dev/md2"
>
> /dev/md2:
> Version : 1.2
> Creation Time : Tue Jun 13 15:21:32 2017
> Raid Level : raid5
> Array Size : 5263812224 (5019.96 GiB 5390.14 GB)
> Used Dev Size : 2631906112 (2509.98 GiB 2695.07 GB)
> Raid Devices : 3
> Total Devices : 3
> Persistence : Superblock is persistent
>
> Intent Bitmap : Internal
>
> Update Time : Wed Jun 21 10:45:39 2017
> State : clean
> Active Devices : 3
> Working Devices : 3
> Failed Devices : 0
> Spare Devices : 0
>
> Layout : left-symmetric
> Chunk Size : 64K
>
> Name : love-1:3 (local to host love-1)
> UUID : 5b2c25fc:b4ccc860:ba8685fe:5e0433f7
> Events : 5176
>
> Number Major Minor RaidDevice State
> 0 8 65 0 active sync /dev/sde1
> 1 8 81 1 active sync /dev/sdf1
> 3 8 97 2 active sync /dev/sdg1
> ==========================================================================================
> ==========================================================================================
> command "fdisk -lu /dev/sde /dev/sdf /dev/sdg":
>
> Disk /dev/sde: 4000.8 GB, 4000787030016 bytes, 7814037168 sectors
> Units = sectors of 1 * 512 = 512 bytes
> Sector size (logical/physical): 512 bytes / 4096 bytes
> I/O size (minimum/optimal): 4096 bytes / 4096 bytes
> Disk label type: gpt
>
>
> # Start End Size Type Name
> 1 2048 7814037134 3.7T Linux RAID
>
> Disk /dev/sdf: 4000.8 GB, 4000787030016 bytes, 7814037168 sectors
> Units = sectors of 1 * 512 = 512 bytes
> Sector size (logical/physical): 512 bytes / 4096 bytes
> I/O size (minimum/optimal): 4096 bytes / 4096 bytes
> Disk label type: gpt
>
>
> # Start End Size Type Name
> 1 2048 7814037134 3.7T Linux RAID
>
> Disk /dev/sdg: 4000.8 GB, 4000787030016 bytes, 7814037168 sectors
> Units = sectors of 1 * 512 = 512 bytes
> Sector size (logical/physical): 512 bytes / 4096 bytes
> I/O size (minimum/optimal): 4096 bytes / 4096 bytes
> Disk label type: gpt
>
>
> # Start End Size Type Name
> 1 2048 7814037134 3.7T Linux RAID
> ==========================================================================================
> ==========================================================================================
> command "mdadm --grow --size=max /dev/md2"
> mdadm: component size of /dev/md2 unchanged at 2631906112K
> ==========================================================================================
^ permalink raw reply
* Re: [PATCH] md/bitmap: don't read page from device with Bitmap_sync
From: Guoqing Jiang @ 2017-06-22 6:45 UTC (permalink / raw)
To: Shaohua Li, NeilBrown; +Cc: linux-raid
In-Reply-To: <20170621174719.dwih6qdk2a5nd7yl@kernel.org>
On 06/22/2017 01:47 AM, Shaohua Li wrote:
> On Tue, Jun 20, 2017 at 01:59:45PM +1000, Neil Brown wrote:
>> On Mon, Jun 19 2017, Shaohua Li wrote:
>>
>>> On Mon, Jun 19, 2017 at 09:11:23AM +1000, Neil Brown wrote:
>>>> On Fri, Jun 16 2017, Shaohua Li wrote:
>>>>
>>>>> On Fri, Jun 16, 2017 at 05:19:27PM +0800, Guoqing Jiang wrote:
>>>>>> The device owns Bitmap_sync flag needs recovery
>>>>>> to become in sync, and read page from this type
>>>>>> device could get stale status.
>>>>>>
>>>>>> Signed-off-by: Guoqing Jiang <gqjiang@suse.com>
>>>>>> ---
>>>>>> When develop for clustered raid10 feature, if add a
>>>>>> disk under grow mode in master node, I could get
>>>>>> the "bitmap superblock UUID mismatch" warning due
>>>>>> to the page is read from Bitmap_sync device.
>>>>>>
>>>>>> drivers/md/bitmap.c | 3 ++-
>>>>>> 1 file changed, 2 insertions(+), 1 deletion(-)
>>>>>>
>>>>>> diff --git a/drivers/md/bitmap.c b/drivers/md/bitmap.c
>>>>>> index bf7419a..bf34cd8 100644
>>>>>> --- a/drivers/md/bitmap.c
>>>>>> +++ b/drivers/md/bitmap.c
>>>>>> @@ -156,7 +156,8 @@ static int read_sb_page(struct mddev *mddev, loff_t offset,
>>>>>>
>>>>>> rdev_for_each(rdev, mddev) {
>>>>>> if (! test_bit(In_sync, &rdev->flags)
>>>>> Why In_sync is set for the rdev? I think it shoudn't.
>>>> There are several states a device can be in.
>>>> If ->raid_disk is < 0, then the device is a spare, and doesn't contain
>>>> and valid data.
>>>> Otherwise ->raid_disk >=0 and:
>>>> In_sync is clear, which means that blocks before ->recovery_offset
>>>> contain valid data, and blocks after there don't
>>>> In_sync is set, which means ->recovery_offset is irrelevant and
>>>> should be treated as MaxSector
>>>> Bitmap_sync is set, which could apply in either of the above cases,
>>>> and means blocks corresponding to bits that are set in the bitmap
>>>> may not be up to date.
>>>>
>>>> Bitmap_sync is only relevant before a device has been handed to the
>>>> personality. After it has been added, ->recovery_cp ensures that the
>>>> blocks that might be wrong are not read until until the bitmap-based
>>>> recovery has fixed them up.
>>>>
>>>> Bitmap_sync is only use to stop the device from being given to the
>>>> personality if a recovery won't be started because the array is
>>>> read-only.
>>>>
>>>> So it is perfectly valid for both In_sync and Bitmap_sync to be set.
>>>> If they are, it makes sense to avoid reading bitmap information from the
>>>> Bitmap_sync device, as that will be out-of-date.
>>>>
>>>> I'm not quite sure why Guoqing is getting a UUID mismatch ... it
>>>> suggests that the new device wasn't initialized properly. So there
>>>> might be another bug. But I think this is definitely a bug.
>>> Can you describe a scenario a disk has both bits set? I had hard time to
>>> imagine it.
>> An array with a bitmap has a device fail (cable break?).
>> So the bitmap stops clearing bits and ->events_cleared stays
>> where it was, with the same value as events in the superblock
>> of the failed device.
>>
>> The device then starts working (cable is replaced) and
>> it device is added back to the array.
>> super_1_validate notices that the array has a bitmap and that the event
>> count in the new device is >= mddev->bitmap->events_cleared, but <
>> mddev->events.
>> i.e. it is in the range for which a bitmap-based recovery is both
>> possible and needed.
>> So it sets Bitmap_sync.
>> Then it notices that the 'role' in a valid role for the array,
>> so that role is stored in rdev->saved_raid_disk and, as there is no
>> MD_FEATURE_RECOVERY_OFFSET, In_sync is set.
>> A similar sequence can happen in super_90_validate.
> This part is quite confusing actually. The failed device is kicked out, so the
> array is degraded. When we readd the device, we shouldn't set In_sync.
>
> Actually we set In_sync in super_1_validate, we then clear it in add_bound_rdev
> and reset it till recovery finishes. Set it till recovery finishes makes sense
> to me, the super_1_validate is confusing.
>
>>> md_update_sb will update super and bitmap super at almost the same time. It's
>>> possible we update super but not bitmap super and there is power loss. Is this
>>> the case we have the both bits set? There seems no mechanism to prevent the
>>> first disk has older bitmap super than the second disk in this scenario.
>> This isn't really about the superblock and bitmap on the one device
>> being in sync or not.
>> It is the superblock on one (failed) device having a different event
>> count than the bitmap (on the other devices) that is actively being used
>> by the array.
> My point is we could use a disk's superblock, but its bitmap superblock is
> stale (even is old)
>
> Anyway, I still didn't understand when the problem could happen. Maybe Guoqing
> can clarify.
My case is not same as what Neil described. But if add a disk to cluster
raid10 under grow
mode, I can see:
1. super_1_validate sets Bitmap_sync flag first, this is because the new
added disk's event
is less than array's event. But for native raid, both of the events are
0, so Bitmap_sync is
not set, this difference is caused by do_md_run only calls
md_allow_write for clustered raid,
and IIRC we have to do it to resolve deadlock.
2. In_sync is set by raid10_start_reshape.
This is called by either clustered raid or native raid, I don't think we
can change it here ...
Based on the above, it is possible that both of the two flags can exist
together.
Thanks,
Guoqing
^ permalink raw reply
* [PATCH v6 2/2] crypto: Multikey template for essiv
From: Binoy Jayan @ 2017-06-22 4:41 UTC (permalink / raw)
To: Mark Brown
Cc: Arnd Bergmann, linux-crypto, linux-kernel, dm-devel, linux-raid,
Rajendra, Binoy Jayan
In-Reply-To: <1498106510-19793-1-git-send-email-binoy.jayan@linaro.org>
Just for reference and to get the performance numbers.
Not for merging.
Depends on the following patches by Gilad:
MAINTAINERS: add Gilad BY as maintainer for ccree
staging: ccree: add devicetree bindings
staging: ccree: add TODO list
staging: add ccree crypto driver
A multi key template implementation which calls the underlying
iv generator 'essiv-aes-du512-dx' cum crypto algorithm. This
template sits on top of the underlying IV generator and accepts
a key length that is a multiple of the underlying key length.
This has not been tested on Juno with the CryptoCell accelerator
for which it was written for.
The underlying IV generator 'essiv-aes-du512-dx' generates IV for
every 512 byte blocks.
Signed-off-by: Binoy Jayan <binoy.jayan@linaro.org>
---
drivers/md/dm-crypt.c | 5 +-
drivers/staging/ccree/Makefile | 2 +-
drivers/staging/ccree/essiv.c | 777 ++++++++++++++++++++++++++++
drivers/staging/ccree/essiv_sw.c | 1040 ++++++++++++++++++++++++++++++++++++++
4 files changed, 1821 insertions(+), 3 deletions(-)
create mode 100644 drivers/staging/ccree/essiv.c
create mode 100644 drivers/staging/ccree/essiv_sw.c
diff --git a/drivers/md/dm-crypt.c b/drivers/md/dm-crypt.c
index bef54f5..32f75dd 100644
--- a/drivers/md/dm-crypt.c
+++ b/drivers/md/dm-crypt.c
@@ -1555,7 +1555,8 @@ static int __init geniv_register_algs(void)
if (err)
goto out_undo_plain;
- err = crypto_register_template(&crypto_essiv_tmpl);
+ err = 0;
+ // err = crypto_register_template(&crypto_essiv_tmpl);
if (err)
goto out_undo_plain64;
@@ -1594,7 +1595,7 @@ static void __exit geniv_deregister_algs(void)
{
crypto_unregister_template(&crypto_plain_tmpl);
crypto_unregister_template(&crypto_plain64_tmpl);
- crypto_unregister_template(&crypto_essiv_tmpl);
+ // crypto_unregister_template(&crypto_essiv_tmpl);
crypto_unregister_template(&crypto_benbi_tmpl);
crypto_unregister_template(&crypto_null_tmpl);
crypto_unregister_template(&crypto_lmk_tmpl);
diff --git a/drivers/staging/ccree/Makefile b/drivers/staging/ccree/Makefile
index 44f3e3e..524e930 100644
--- a/drivers/staging/ccree/Makefile
+++ b/drivers/staging/ccree/Makefile
@@ -1,3 +1,3 @@
obj-$(CONFIG_CRYPTO_DEV_CCREE) := ccree.o
-ccree-y := ssi_driver.o ssi_sysfs.o ssi_buffer_mgr.o ssi_request_mgr.o ssi_cipher.o ssi_hash.o ssi_aead.o ssi_ivgen.o ssi_sram_mgr.o ssi_pm.o ssi_pm_ext.o
+ccree-y := ssi_driver.o ssi_sysfs.o ssi_buffer_mgr.o ssi_request_mgr.o ssi_cipher.o ssi_hash.o ssi_aead.o ssi_ivgen.o ssi_sram_mgr.o ssi_pm.o ssi_pm_ext.o essiv.o
ccree-$(CCREE_FIPS_SUPPORT) += ssi_fips.o ssi_fips_ll.o ssi_fips_ext.o ssi_fips_local.o
diff --git a/drivers/staging/ccree/essiv.c b/drivers/staging/ccree/essiv.c
new file mode 100644
index 0000000..719b8bf
--- /dev/null
+++ b/drivers/staging/ccree/essiv.c
@@ -0,0 +1,777 @@
+/*
+ * Copyright (C) 2003 Jana Saout <jana@saout.de>
+ * Copyright (C) 2004 Clemens Fruhwirth <clemens@endorphin.org>
+ * Copyright (C) 2006-2015 Red Hat, Inc. All rights reserved.
+ * Copyright (C) 2013 Milan Broz <gmazyland@gmail.com>
+ *
+ * This file is released under the GPL.
+ */
+
+#include <linux/completion.h>
+#include <linux/err.h>
+#include <linux/module.h>
+#include <linux/init.h>
+#include <linux/kernel.h>
+#include <linux/key.h>
+#include <linux/bio.h>
+#include <linux/blkdev.h>
+#include <linux/mempool.h>
+#include <linux/slab.h>
+#include <linux/crypto.h>
+#include <linux/workqueue.h>
+#include <linux/kthread.h>
+#include <linux/backing-dev.h>
+#include <linux/atomic.h>
+#include <linux/scatterlist.h>
+#include <linux/rbtree.h>
+#include <linux/ctype.h>
+#include <asm/page.h>
+#include <asm/unaligned.h>
+#include <crypto/hash.h>
+#include <crypto/md5.h>
+#include <crypto/algapi.h>
+#include <crypto/skcipher.h>
+#include <keys/user-type.h>
+#include <linux/device-mapper.h>
+#include <crypto/internal/skcipher.h>
+#include <linux/backing-dev.h>
+#include <linux/log2.h>
+#include <crypto/geniv.h>
+
+#define DM_MSG_PREFIX "crypt"
+#define MAX_SG_LIST (BIO_MAX_PAGES * 8)
+#define MIN_IOS 64
+#define LMK_SEED_SIZE 64 /* hash + 0 */
+#define TCW_WHITENING_SIZE 16
+
+struct geniv_ctx;
+struct geniv_req_ctx;
+
+/* Sub request for each of the skcipher_request's for a segment */
+struct geniv_subreq {
+ struct scatterlist src;
+ struct scatterlist dst;
+ struct geniv_req_ctx *rctx;
+ struct skcipher_request req CRYPTO_MINALIGN_ATTR;
+};
+
+struct geniv_req_ctx {
+ struct geniv_subreq *subreq;
+ int is_write;
+ sector_t iv_sector;
+ unsigned int nents;
+ struct completion restart;
+ atomic_t req_pending;
+ struct skcipher_request *req;
+};
+
+struct crypt_iv_operations {
+ int (*ctr)(struct geniv_ctx *ctx);
+ void (*dtr)(struct geniv_ctx *ctx);
+ int (*init)(struct geniv_ctx *ctx);
+ int (*wipe)(struct geniv_ctx *ctx);
+ int (*generator)(struct geniv_ctx *ctx,
+ struct geniv_req_ctx *rctx,
+ struct geniv_subreq *subreq, u8 *iv);
+ int (*post)(struct geniv_ctx *ctx,
+ struct geniv_req_ctx *rctx,
+ struct geniv_subreq *subreq, u8 *iv);
+};
+
+struct geniv_ctx {
+ unsigned int tfms_count;
+ struct crypto_skcipher *child;
+ struct crypto_skcipher **tfms;
+ char *ivmode;
+ unsigned int iv_size;
+ unsigned int iv_start;
+ char *algname;
+ char *ivopts;
+ char *cipher;
+ char *ciphermode;
+ const struct crypt_iv_operations *iv_gen_ops;
+ void *iv_private;
+ struct crypto_skcipher *tfm;
+ mempool_t *subreq_pool;
+ unsigned int key_size;
+ unsigned int key_extra_size;
+ unsigned int key_parts; /* independent parts in key buffer */
+ enum setkey_op keyop;
+ char *msg;
+ u8 *key;
+};
+
+static inline
+struct geniv_req_ctx *geniv_req_ctx(struct skcipher_request *req)
+{
+ struct crypto_skcipher *tfm = crypto_skcipher_reqtfm(req);
+ unsigned long align = crypto_skcipher_alignmask(tfm);
+
+ return (void *) PTR_ALIGN((u8 *) skcipher_request_ctx(req), align + 1);
+}
+
+static u8 *iv_of_subreq(struct geniv_ctx *ctx, struct geniv_subreq *subreq)
+{
+ struct crypto_skcipher *tfm = crypto_skcipher_reqtfm(subreq->rctx->req);
+ return (u8*) ALIGN((unsigned long) ((char *) subreq + ctx->iv_start),
+ crypto_skcipher_alignmask(tfm) + 1);
+}
+
+/*
+ * Different IV generation algorithms:
+ *
+ * essiv: "encrypted sector|salt initial vector", the sector number is
+ * encrypted with the bulk cipher using a salt as key. The salt
+ * should be derived from the bulk cipher's key via hashing.
+ *
+ */
+
+static int crypt_iv_essiv_gen(struct geniv_ctx *ctx,
+ struct geniv_req_ctx *rctx,
+ struct geniv_subreq *subreq, u8 *iv)
+{
+ memset(iv, 0, ctx->iv_size);
+ *(__le64 *)iv = cpu_to_le64(rctx->iv_sector);
+
+ return 0;
+}
+
+static const struct crypt_iv_operations crypt_iv_essiv_ops = {
+ .generator = crypt_iv_essiv_gen
+};
+
+static int geniv_setkey_set(struct geniv_ctx *ctx)
+{
+ int ret = 0;
+
+ if (ctx->iv_gen_ops && ctx->iv_gen_ops->init)
+ ret = ctx->iv_gen_ops->init(ctx);
+ return ret;
+}
+
+static int geniv_setkey_wipe(struct geniv_ctx *ctx)
+{
+ int ret = 0;
+
+ if (ctx->iv_gen_ops && ctx->iv_gen_ops->wipe) {
+ ret = ctx->iv_gen_ops->wipe(ctx);
+ if (ret)
+ return ret;
+ }
+ return ret;
+}
+
+static int geniv_init_iv(struct geniv_ctx *ctx)
+{
+ int ret = -EINVAL;
+
+ DMDEBUG("IV Generation algorithm : %s\n", ctx->ivmode);
+
+ if (ctx->ivmode == NULL)
+ ctx->iv_gen_ops = NULL;
+ else if (strcmp(ctx->ivmode, "essiv") == 0)
+ ctx->iv_gen_ops = &crypt_iv_essiv_ops;
+ else {
+ ret = -EINVAL;
+ DMERR("Invalid IV mode %s\n", ctx->ivmode);
+ goto end;
+ }
+
+ /* Allocate IV */
+ if (ctx->iv_gen_ops && ctx->iv_gen_ops->ctr) {
+ ret = ctx->iv_gen_ops->ctr(ctx);
+ if (ret < 0) {
+ DMERR("Error creating IV for %s\n", ctx->ivmode);
+ goto end;
+ }
+ }
+
+ /* Initialize IV (set keys for ESSIV etc) */
+ if (ctx->iv_gen_ops && ctx->iv_gen_ops->init) {
+ ret = ctx->iv_gen_ops->init(ctx);
+ if (ret < 0)
+ DMERR("Error creating IV for %s\n", ctx->ivmode);
+ }
+ ret = 0;
+end:
+ return ret;
+}
+
+static void geniv_free_tfms(struct geniv_ctx *ctx)
+{
+ unsigned int i;
+
+ if (!ctx->tfms)
+ return;
+
+ for (i = 0; i < ctx->tfms_count; i++)
+ if (ctx->tfms[i] && !IS_ERR(ctx->tfms[i])) {
+ crypto_free_skcipher(ctx->tfms[i]);
+ ctx->tfms[i] = NULL;
+ }
+
+ kfree(ctx->tfms);
+ ctx->tfms = NULL;
+}
+
+/* Allocate memory for the underlying cipher algorithm. Ex: cbc(aes)
+ */
+
+static int geniv_alloc_tfms(struct crypto_skcipher *parent,
+ struct geniv_ctx *ctx)
+{
+ unsigned int i, reqsize, align;
+ int err = 0;
+
+ ctx->tfms = kcalloc(ctx->tfms_count, sizeof(struct crypto_skcipher *),
+ GFP_KERNEL);
+ if (!ctx->tfms) {
+ err = -ENOMEM;
+ goto end;
+ }
+
+ /* First instance is already allocated in geniv_init_tfm */
+ ctx->tfms[0] = ctx->child;
+ for (i = 1; i < ctx->tfms_count; i++) {
+ ctx->tfms[i] = crypto_alloc_skcipher(ctx->ciphermode, 0, 0);
+ if (IS_ERR(ctx->tfms[i])) {
+ err = PTR_ERR(ctx->tfms[i]);
+ geniv_free_tfms(ctx);
+ goto end;
+ }
+
+ /* Setup the current cipher's request structure */
+ align = crypto_skcipher_alignmask(parent);
+ align &= ~(crypto_tfm_ctx_alignment() - 1);
+ reqsize = align + sizeof(struct geniv_req_ctx) +
+ crypto_skcipher_reqsize(ctx->tfms[i]);
+ crypto_skcipher_set_reqsize(parent, reqsize);
+ }
+
+end:
+ return err;
+}
+
+/* Initialize the cipher's context with the key, ivmode and other parameters.
+ * Also allocate IV generation template ciphers and initialize them.
+ */
+
+static int geniv_setkey_init(struct crypto_skcipher *parent,
+ struct geniv_key_info *info)
+{
+ struct geniv_ctx *ctx = crypto_skcipher_ctx(parent);
+ int ret = -ENOMEM;
+
+ if (info->tfms_count > 1) {
+ DMERR("Cannot use keycount > 1 for this cipher\n");
+ ret = -EINVAL;
+ goto end;
+ }
+ ctx->tfms_count = info->tfms_count;
+ ctx->key = info->key;
+ ctx->key_size = info->key_size;
+ ctx->key_parts = info->key_parts;
+ ctx->ivopts = info->ivopts;
+
+ ret = geniv_alloc_tfms(parent, ctx);
+ if (ret)
+ goto end;
+
+ ret = geniv_init_iv(ctx);
+
+end:
+ return ret;
+}
+
+static int geniv_setkey_tfms(struct crypto_skcipher *parent,
+ struct geniv_ctx *ctx,
+ struct geniv_key_info *info)
+{
+ unsigned int subkey_size;
+ int ret = 0, i;
+
+ /* Ignore extra keys (which are used for IV etc) */
+ subkey_size = (ctx->key_size - ctx->key_extra_size)
+ >> ilog2(ctx->tfms_count);
+
+ for (i = 0; i < ctx->tfms_count; i++) {
+ struct crypto_skcipher *child = ctx->tfms[i];
+ char *subkey = ctx->key + (subkey_size) * i;
+
+ crypto_skcipher_clear_flags(child, CRYPTO_TFM_REQ_MASK);
+ crypto_skcipher_set_flags(child,
+ crypto_skcipher_get_flags(parent) &
+ CRYPTO_TFM_REQ_MASK);
+ ret = crypto_skcipher_setkey(child, subkey, subkey_size);
+ if (ret) {
+ DMERR("Error setting key for tfms[%d]\n", i);
+ break;
+ }
+ crypto_skcipher_set_flags(parent,
+ crypto_skcipher_get_flags(child) &
+ CRYPTO_TFM_RES_MASK);
+ }
+
+ return ret;
+}
+
+static int geniv_setkey(struct crypto_skcipher *parent,
+ const u8 *key, unsigned int keylen)
+{
+ int err = 0;
+ struct geniv_ctx *ctx = crypto_skcipher_ctx(parent);
+ struct geniv_key_info *info = (struct geniv_key_info *) key;
+
+ DMDEBUG("SETKEY Operation : %d\n", info->keyop);
+
+ switch (info->keyop) {
+ case SETKEY_OP_INIT:
+ err = geniv_setkey_init(parent, info);
+ break;
+ case SETKEY_OP_SET:
+ err = geniv_setkey_set(ctx);
+ break;
+ case SETKEY_OP_WIPE:
+ err = geniv_setkey_wipe(ctx);
+ break;
+ }
+
+ if (err)
+ goto end;
+
+ err = geniv_setkey_tfms(parent, ctx, info);
+
+end:
+ return err;
+}
+
+static void geniv_async_done(struct crypto_async_request *async_req, int error);
+
+static int geniv_alloc_subreq(struct skcipher_request *req,
+ struct geniv_ctx *ctx,
+ struct geniv_req_ctx *rctx)
+{
+ int key_index, r = 0;
+ struct skcipher_request *sreq;
+
+ if (!rctx->subreq) {
+ rctx->subreq = mempool_alloc(ctx->subreq_pool, GFP_NOIO);
+ if (!rctx->subreq)
+ r = -ENOMEM;
+ }
+
+ sreq = &rctx->subreq->req;
+ rctx->subreq->rctx = rctx;
+
+ key_index = rctx->iv_sector & (ctx->tfms_count - 1);
+
+ skcipher_request_set_tfm(sreq, ctx->tfms[key_index]);
+ skcipher_request_set_callback(sreq, req->base.flags,
+ geniv_async_done, rctx->subreq);
+ return r;
+}
+
+/* Asynchronous IO completion callback for each sector in a segment. When all
+ * pending i/o are completed the parent cipher's async function is called.
+ */
+
+static void geniv_async_done(struct crypto_async_request *async_req, int error)
+{
+ struct geniv_subreq *subreq =
+ (struct geniv_subreq *) async_req->data;
+ struct geniv_req_ctx *rctx = subreq->rctx;
+ struct skcipher_request *req = rctx->req;
+ struct crypto_skcipher *tfm = crypto_skcipher_reqtfm(req);
+ struct geniv_ctx *ctx = crypto_skcipher_ctx(tfm);
+ u8 *iv;
+
+ /*
+ * A request from crypto driver backlog is going to be processed now,
+ * finish the completion and continue in crypt_convert().
+ * (Callback will be called for the second time for this request.)
+ */
+
+ if (error == -EINPROGRESS) {
+ complete(&rctx->restart);
+ return;
+ }
+
+ iv = iv_of_subreq(ctx, subreq);
+ if (!error && ctx->iv_gen_ops && ctx->iv_gen_ops->post)
+ error = ctx->iv_gen_ops->post(ctx, rctx, subreq, iv);
+
+ mempool_free(subreq, ctx->subreq_pool);
+
+ /* req_pending needs to be checked before req->base.complete is called
+ * as we need 'req_pending' to be equal to 1 to ensure all subrequests
+ * are processed.
+ */
+ if (!atomic_dec_and_test(&rctx->req_pending)) {
+ /* Call the parent cipher's completion function */
+ skcipher_request_complete(req, error);
+ }
+}
+
+
+/* Common encryt/decrypt function for geniv template cipher. Before the crypto
+ * operation, it splits the memory segments (in the scatterlist) into 512 byte
+ * sectors. The initialization vector(IV) used is based on a unique sector
+ * number which is generated here.
+ */
+static int geniv_crypt(struct skcipher_request *req, int encrypt)
+{
+ struct crypto_skcipher *tfm = crypto_skcipher_reqtfm(req);
+ struct geniv_ctx *ctx = crypto_skcipher_ctx(tfm);
+ struct geniv_req_ctx *rctx = geniv_req_ctx(req);
+ struct geniv_req_info *rinfo = (struct geniv_req_info *) req->iv;
+ int ret = 0;
+ char *str __maybe_unused = encrypt ? "encrypt" : "decrypt";
+ u8 *iv;
+
+ /* Instance of 'struct geniv_req_info' is stored in IV ptr */
+ rctx->is_write = encrypt;
+ rctx->iv_sector = rinfo->iv_sector;
+ rctx->nents = rinfo->nents;
+ rctx->req = req;
+ rctx->subreq = NULL;
+
+ DMDEBUG("geniv:%s: starting sector=%d, #segments=%u\n", str,
+ (unsigned int) rctx->iv_sector, rctx->nents);
+
+ init_completion(&rctx->restart);
+ atomic_set(&rctx->req_pending, 1);
+
+ struct geniv_subreq *subreq;
+
+ ret = geniv_alloc_subreq(req, ctx, rctx);
+ if (ret)
+ goto end;
+
+ subreq = rctx->subreq;
+ subreq->rctx = rctx;
+ iv = iv_of_subreq(ctx, subreq);
+
+ atomic_inc(&rctx->req_pending);
+
+ if (ctx->iv_gen_ops)
+ ret = ctx->iv_gen_ops->generator(ctx, rctx, subreq, iv);
+
+ if (ret < 0) {
+ DMERR("Error in generating IV ret: %d\n", ret);
+ goto end;
+ }
+
+ skcipher_request_set_crypt(&subreq->req, req->src,
+ req->dst, req->cryptlen, iv);
+
+ if (encrypt)
+ ret = crypto_skcipher_encrypt(&subreq->req);
+
+ else
+ ret = crypto_skcipher_decrypt(&subreq->req);
+
+ switch (ret) {
+ /*
+ * The request was queued by a crypto driver
+ * but the driver request queue is full, let's wait.
+ */
+ case -EBUSY:
+ wait_for_completion(&rctx->restart);
+ reinit_completion(&rctx->restart);
+ /* fall through */
+ /*
+ * The request is queued and processed asynchronously,
+ * completion function geniv_async_done() is called.
+ */
+ case -EINPROGRESS:
+ /* Marking this NULL lets the creation of a new sub-
+ * request when 'geniv_alloc_subreq' is called.
+ */
+ rctx->subreq = NULL;
+ rctx->iv_sector++;
+ cond_resched();
+ break;
+ /*
+ * The request was already processed (synchronously).
+ */
+ case 0:
+ atomic_dec(&rctx->req_pending);
+ rctx->iv_sector++;
+ cond_resched();
+ break;
+
+ /* There was an error while processing the request. */
+ default:
+ atomic_dec(&rctx->req_pending);
+ return ret;
+ }
+
+ if (rctx->subreq && atomic_read(&rctx->req_pending) == 1) {
+ DMDEBUG("geniv:%s: Freeing sub request\n", str);
+ mempool_free(rctx->subreq, ctx->subreq_pool);
+ }
+
+end:
+ return ret;
+}
+
+static int geniv_encrypt(struct skcipher_request *req)
+{
+ return geniv_crypt(req, 1);
+}
+
+static int geniv_decrypt(struct skcipher_request *req)
+{
+ return geniv_crypt(req, 0);
+}
+
+static int geniv_init_tfm(struct crypto_skcipher *tfm)
+{
+ struct geniv_ctx *ctx = crypto_skcipher_ctx(tfm);
+ unsigned int reqsize, align;
+ size_t iv_size_padding;
+ char *algname, *chainmode;
+ int psize, ret = 0;
+
+ algname = (char *) crypto_tfm_alg_name(crypto_skcipher_tfm(tfm));
+ ctx->ciphermode = kmalloc(CRYPTO_MAX_ALG_NAME, GFP_KERNEL);
+ if (!ctx->ciphermode) {
+ ret = -ENOMEM;
+ goto out;
+ }
+
+ ctx->algname = kmalloc(CRYPTO_MAX_ALG_NAME, GFP_KERNEL);
+ if (!ctx->algname) {
+ ret = -ENOMEM;
+ goto free_ciphermode;
+ }
+
+ strlcpy(ctx->algname, algname, CRYPTO_MAX_ALG_NAME);
+ algname = ctx->algname;
+
+ /* Parse the algorithm name 'ivmode(chainmode(cipher))' */
+ ctx->ivmode = strsep(&algname, "(");
+ chainmode = strsep(&algname, "(");
+ ctx->cipher = strsep(&algname, ")");
+
+ if (strcmp(ctx->ivmode, "essiv") == 0)
+ // strlcpy(ctx->ciphermode, "essiv-aes-du512-dx", CRYPTO_MAX_ALG_NAME);
+ snprintf(ctx->ciphermode, CRYPTO_MAX_ALG_NAME, "%s(%s)",
+ chainmode, ctx->cipher);
+ else {
+ ret = -EINVAL;
+ DMERR("Invalid IV mode %s\n", ctx->ivmode);
+ goto free_algname;
+ }
+
+ DMDEBUG("ciphermode=%s, ivmode=%s\n", ctx->ciphermode, ctx->ivmode);
+
+ /*
+ * Usually the underlying cipher instances are spawned here, but since
+ * the value of tfms_count (which is equal to the key_count) is not
+ * known yet, create only one instance and delay the creation of the
+ * rest of the instances of the underlying cipher 'cbc(aes)' until
+ * the setkey operation is invoked.
+ * The first instance created i.e. ctx->child will later be assigned as
+ * the 1st element in the array ctx->tfms. Creation of atleast one
+ * instance of the cipher is necessary to be created here to uncover
+ * any errors earlier than during the setkey operation later where the
+ * remaining instances are created.
+ */
+ ctx->child = crypto_alloc_skcipher(ctx->ciphermode, 0, 0);
+ if (IS_ERR(ctx->child)) {
+ ret = PTR_ERR(ctx->child);
+ DMERR("Failed to create skcipher %s. err %d\n",
+ ctx->ciphermode, ret);
+ goto free_algname;
+ }
+
+ /* Setup the current cipher's request structure */
+ align = crypto_skcipher_alignmask(tfm);
+ align &= ~(crypto_tfm_ctx_alignment() - 1);
+ reqsize = align + sizeof(struct geniv_req_ctx)
+ + crypto_skcipher_reqsize(ctx->child);
+ crypto_skcipher_set_reqsize(tfm, reqsize);
+
+ ctx->iv_start = sizeof(struct geniv_subreq);
+ ctx->iv_start += crypto_skcipher_reqsize(tfm);
+
+ ctx->iv_size = crypto_skcipher_ivsize(tfm);
+ /* at least a 64 bit sector number should fit in our buffer */
+ if (ctx->iv_size)
+ ctx->iv_size = max(ctx->iv_size,
+ (unsigned int)(sizeof(u64) / sizeof(u8)));
+
+ if (crypto_skcipher_alignmask(tfm) < CRYPTO_MINALIGN) {
+ /* Allocate the padding exactly */
+ iv_size_padding = -ctx->iv_start
+ & crypto_skcipher_alignmask(tfm);
+ } else {
+ /*
+ * If the cipher requires greater alignment than kmalloc
+ * alignment, we don't know the exact position of the
+ * initialization vector. We must assume worst case.
+ */
+ iv_size_padding = crypto_skcipher_alignmask(tfm);
+ }
+
+ /* create memory pool for sub-request structure */
+ psize = ctx->iv_start + iv_size_padding + ctx->iv_size;
+
+ ctx->subreq_pool = mempool_create_kmalloc_pool(MIN_IOS, psize);
+ if (!ctx->subreq_pool) {
+ ret = -ENOMEM;
+ DMERR("Could not allocate crypt sub-request mempool\n");
+ goto free_skcipher;
+ }
+out:
+ return ret;
+
+free_skcipher:
+ crypto_free_skcipher(ctx->child);
+free_algname:
+ kfree(ctx->algname);
+free_ciphermode:
+ kfree(ctx->ciphermode);
+ goto out;
+}
+
+static void geniv_exit_tfm(struct crypto_skcipher *tfm)
+{
+ struct geniv_ctx *ctx = crypto_skcipher_ctx(tfm);
+
+ if (ctx->iv_gen_ops && ctx->iv_gen_ops->dtr)
+ ctx->iv_gen_ops->dtr(ctx);
+
+ mempool_destroy(ctx->subreq_pool);
+ geniv_free_tfms(ctx);
+ kfree(ctx->ciphermode);
+ kfree(ctx->algname);
+}
+
+static void geniv_free(struct skcipher_instance *inst)
+{
+ struct crypto_skcipher_spawn *spawn = skcipher_instance_ctx(inst);
+
+ crypto_drop_skcipher(spawn);
+ kfree(inst);
+}
+
+static int geniv_create(struct crypto_template *tmpl,
+ struct rtattr **tb, char *algname)
+{
+ struct crypto_attr_type *algt;
+ struct skcipher_instance *inst;
+ struct skcipher_alg *alg;
+ struct crypto_skcipher_spawn *spawn;
+ const char *cipher_name;
+ int err;
+
+ algt = crypto_get_attr_type(tb);
+
+ if (IS_ERR(algt))
+ return PTR_ERR(algt);
+
+ if ((algt->type ^ CRYPTO_ALG_TYPE_SKCIPHER) & algt->mask)
+ return -EINVAL;
+
+ cipher_name = crypto_attr_alg_name(tb[1]);
+
+ if (IS_ERR(cipher_name))
+ return PTR_ERR(cipher_name);
+
+ inst = kzalloc(sizeof(*inst) + sizeof(*spawn), GFP_KERNEL);
+ if (!inst)
+ return -ENOMEM;
+
+ spawn = skcipher_instance_ctx(inst);
+
+ crypto_set_skcipher_spawn(spawn, skcipher_crypto_instance(inst));
+ err = crypto_grab_skcipher(spawn, cipher_name, 0,
+ crypto_requires_sync(algt->type,
+ algt->mask));
+
+ if (err)
+ goto err_free_inst;
+
+ alg = crypto_spawn_skcipher_alg(spawn);
+
+ err = -EINVAL;
+
+ /* Only support blocks of size which is of a power of 2 */
+ if (!is_power_of_2(alg->base.cra_blocksize))
+ goto err_drop_spawn;
+
+ /* algname: essiv, base.cra_name: cbc(aes) */
+ err = -ENAMETOOLONG;
+ if (snprintf(inst->alg.base.cra_name, CRYPTO_MAX_ALG_NAME, "%s(%s)",
+ algname, alg->base.cra_name) >= CRYPTO_MAX_ALG_NAME)
+ goto err_drop_spawn;
+ if (snprintf(inst->alg.base.cra_driver_name, CRYPTO_MAX_ALG_NAME,
+ "%s(%s)", algname, alg->base.cra_driver_name) >=
+ CRYPTO_MAX_ALG_NAME)
+ goto err_drop_spawn;
+
+ inst->alg.base.cra_flags = CRYPTO_ALG_TYPE_BLKCIPHER;
+ inst->alg.base.cra_priority = alg->base.cra_priority;
+ inst->alg.base.cra_blocksize = alg->base.cra_blocksize;
+ inst->alg.base.cra_alignmask = alg->base.cra_alignmask;
+ inst->alg.base.cra_flags = alg->base.cra_flags & CRYPTO_ALG_ASYNC;
+ inst->alg.ivsize = alg->base.cra_blocksize;
+ inst->alg.chunksize = crypto_skcipher_alg_chunksize(alg);
+ inst->alg.min_keysize = crypto_skcipher_alg_min_keysize(alg);
+ inst->alg.max_keysize = crypto_skcipher_alg_max_keysize(alg);
+
+ inst->alg.setkey = geniv_setkey;
+ inst->alg.encrypt = geniv_encrypt;
+ inst->alg.decrypt = geniv_decrypt;
+
+ inst->alg.base.cra_ctxsize = sizeof(struct geniv_ctx);
+
+ inst->alg.init = geniv_init_tfm;
+ inst->alg.exit = geniv_exit_tfm;
+
+ inst->free = geniv_free;
+
+ err = skcipher_register_instance(tmpl, inst);
+ if (err)
+ goto err_drop_spawn;
+
+out:
+ return err;
+
+err_drop_spawn:
+ crypto_drop_skcipher(spawn);
+err_free_inst:
+ kfree(inst);
+ goto out;
+}
+
+static int crypto_essiv_create(struct crypto_template *tmpl,
+ struct rtattr **tb)
+{
+ return geniv_create(tmpl, tb, "essiv");
+}
+
+static struct crypto_template crypto_essiv_tmpl = {
+ .name = "essiv",
+ .create = crypto_essiv_create,
+ .module = THIS_MODULE,
+};
+
+static int __init essiv_init(void)
+{
+ return crypto_register_template(&crypto_essiv_tmpl);
+}
+
+static void __exit essiv_exit(void)
+{
+ crypto_unregister_template(&crypto_essiv_tmpl);
+}
+
+module_init(essiv_init);
+module_exit(essiv_exit);
+
+MODULE_AUTHOR("Binoy Jayan <binoy.jayan@linaro.org>");
+MODULE_DESCRIPTION(DM_NAME " essiv implementation for cryptocell 712");
+MODULE_LICENSE("GPL");
diff --git a/drivers/staging/ccree/essiv_sw.c b/drivers/staging/ccree/essiv_sw.c
new file mode 100644
index 0000000..f07956f
--- /dev/null
+++ b/drivers/staging/ccree/essiv_sw.c
@@ -0,0 +1,1040 @@
+/*
+ * Copyright (C) 2003 Jana Saout <jana@saout.de>
+ * Copyright (C) 2004 Clemens Fruhwirth <clemens@endorphin.org>
+ * Copyright (C) 2006-2015 Red Hat, Inc. All rights reserved.
+ * Copyright (C) 2013 Milan Broz <gmazyland@gmail.com>
+ *
+ * This file is released under the GPL.
+ */
+
+#include <linux/completion.h>
+#include <linux/err.h>
+#include <linux/module.h>
+#include <linux/init.h>
+#include <linux/kernel.h>
+#include <linux/key.h>
+#include <linux/bio.h>
+#include <linux/blkdev.h>
+#include <linux/mempool.h>
+#include <linux/slab.h>
+#include <linux/crypto.h>
+#include <linux/workqueue.h>
+#include <linux/kthread.h>
+#include <linux/backing-dev.h>
+#include <linux/atomic.h>
+#include <linux/scatterlist.h>
+#include <linux/rbtree.h>
+#include <linux/ctype.h>
+#include <asm/page.h>
+#include <asm/unaligned.h>
+#include <crypto/hash.h>
+#include <crypto/md5.h>
+#include <crypto/algapi.h>
+#include <crypto/skcipher.h>
+#include <keys/user-type.h>
+#include <linux/device-mapper.h>
+#include <crypto/internal/skcipher.h>
+#include <linux/backing-dev.h>
+#include <linux/log2.h>
+#include <crypto/geniv.h>
+
+#define DM_MSG_PREFIX "crypt"
+#define MAX_SG_LIST (BIO_MAX_PAGES * 8)
+#define MIN_IOS 64
+#define LMK_SEED_SIZE 64 /* hash + 0 */
+#define TCW_WHITENING_SIZE 16
+
+struct geniv_ctx;
+struct geniv_req_ctx;
+
+/* Sub request for each of the skcipher_request's for a segment */
+struct geniv_subreq {
+ struct scatterlist src;
+ struct scatterlist dst;
+ struct geniv_req_ctx *rctx;
+ struct skcipher_request req CRYPTO_MINALIGN_ATTR;
+};
+
+struct geniv_req_ctx {
+ struct geniv_subreq *subreq;
+ int is_write;
+ sector_t iv_sector;
+ unsigned int nents;
+ struct completion restart;
+ atomic_t req_pending;
+ struct skcipher_request *req;
+};
+
+struct crypt_iv_operations {
+ int (*ctr)(struct geniv_ctx *ctx);
+ void (*dtr)(struct geniv_ctx *ctx);
+ int (*init)(struct geniv_ctx *ctx);
+ int (*wipe)(struct geniv_ctx *ctx);
+ int (*generator)(struct geniv_ctx *ctx,
+ struct geniv_req_ctx *rctx,
+ struct geniv_subreq *subreq, u8 *iv);
+ int (*post)(struct geniv_ctx *ctx,
+ struct geniv_req_ctx *rctx,
+ struct geniv_subreq *subreq, u8 *iv);
+};
+
+struct geniv_essiv_private {
+ struct crypto_ahash *hash_tfm;
+ u8 *salt;
+};
+
+struct geniv_benbi_private {
+ int shift;
+};
+
+struct geniv_lmk_private {
+ struct crypto_shash *hash_tfm;
+ u8 *seed;
+};
+
+struct geniv_tcw_private {
+ struct crypto_shash *crc32_tfm;
+ u8 *iv_seed;
+ u8 *whitening;
+};
+
+struct geniv_ctx {
+ unsigned int tfms_count;
+ struct crypto_skcipher *child;
+ struct crypto_skcipher **tfms;
+ char *ivmode;
+ unsigned int iv_size;
+ unsigned int iv_start;
+ char *algname;
+ char *ivopts;
+ char *cipher;
+ char *ciphermode;
+ const struct crypt_iv_operations *iv_gen_ops;
+ union {
+ struct geniv_essiv_private essiv;
+ struct geniv_benbi_private benbi;
+ struct geniv_lmk_private lmk;
+ struct geniv_tcw_private tcw;
+ } iv_gen_private;
+ void *iv_private;
+ struct crypto_skcipher *tfm;
+ mempool_t *subreq_pool;
+ unsigned int key_size;
+ unsigned int key_extra_size;
+ unsigned int key_parts; /* independent parts in key buffer */
+ enum setkey_op keyop;
+ char *msg;
+ u8 *key;
+};
+
+static struct crypto_skcipher *any_tfm(struct geniv_ctx *ctx)
+{
+ return ctx->tfms[0];
+}
+
+static inline
+struct geniv_req_ctx *geniv_req_ctx(struct skcipher_request *req)
+{
+ struct crypto_skcipher *tfm = crypto_skcipher_reqtfm(req);
+ unsigned long align = crypto_skcipher_alignmask(tfm);
+
+ return (void *) PTR_ALIGN((u8 *) skcipher_request_ctx(req), align + 1);
+}
+
+static u8 *iv_of_subreq(struct geniv_ctx *ctx, struct geniv_subreq *subreq)
+{
+ struct crypto_skcipher *tfm = crypto_skcipher_reqtfm(subreq->rctx->req);
+ return (u8*) ALIGN((unsigned long) ((char *) subreq + ctx->iv_start),
+ crypto_skcipher_alignmask(tfm) + 1);
+}
+
+/*
+ * Different IV generation algorithms:
+ *
+ * essiv: "encrypted sector|salt initial vector", the sector number is
+ * encrypted with the bulk cipher using a salt as key. The salt
+ * should be derived from the bulk cipher's key via hashing.
+ *
+ * plumb: unimplemented, see:
+ * http://article.gmane.org/gmane.linux.kernel.device-mapper.dm-crypt/454
+ */
+
+/* Initialise ESSIV - compute salt but no local memory allocations */
+static int crypt_iv_essiv_init(struct geniv_ctx *ctx)
+{
+ struct geniv_essiv_private *essiv = &ctx->iv_gen_private.essiv;
+ struct scatterlist sg;
+ struct crypto_cipher *essiv_tfm;
+ int err;
+ AHASH_REQUEST_ON_STACK(req, essiv->hash_tfm);
+
+ sg_init_one(&sg, ctx->key, ctx->key_size);
+ ahash_request_set_tfm(req, essiv->hash_tfm);
+ ahash_request_set_callback(req, CRYPTO_TFM_REQ_MAY_SLEEP, NULL, NULL);
+ ahash_request_set_crypt(req, &sg, essiv->salt, ctx->key_size);
+
+ err = crypto_ahash_digest(req);
+ ahash_request_zero(req);
+ if (err)
+ return err;
+
+ essiv_tfm = ctx->iv_private;
+
+ err = crypto_cipher_setkey(essiv_tfm, essiv->salt,
+ crypto_ahash_digestsize(essiv->hash_tfm));
+ if (err)
+ return err;
+
+ return 0;
+}
+
+/* Wipe salt and reset key derived from volume key */
+static int crypt_iv_essiv_wipe(struct geniv_ctx *ctx)
+{
+ struct geniv_essiv_private *essiv = &ctx->iv_gen_private.essiv;
+ unsigned int salt_size = crypto_ahash_digestsize(essiv->hash_tfm);
+ struct crypto_cipher *essiv_tfm;
+ int r, err = 0;
+
+ memset(essiv->salt, 0, salt_size);
+
+ essiv_tfm = ctx->iv_private;
+ r = crypto_cipher_setkey(essiv_tfm, essiv->salt, salt_size);
+ if (r)
+ err = r;
+
+ return err;
+}
+
+/* Set up per cpu cipher state */
+static struct crypto_cipher *setup_essiv_cpu(struct geniv_ctx *ctx,
+ u8 *salt, unsigned int saltsize)
+{
+ struct crypto_cipher *essiv_tfm;
+ int err;
+
+ /* Setup the essiv_tfm with the given salt */
+ essiv_tfm = crypto_alloc_cipher(ctx->cipher, 0, CRYPTO_ALG_ASYNC);
+
+ if (IS_ERR(essiv_tfm)) {
+ DMERR("Error allocating crypto tfm for ESSIV\n");
+ return essiv_tfm;
+ }
+
+ if (crypto_cipher_blocksize(essiv_tfm) !=
+ crypto_skcipher_ivsize(any_tfm(ctx))) {
+ DMERR("Block size of ESSIV cipher does not match IV size of block cipher\n");
+ crypto_free_cipher(essiv_tfm);
+ return ERR_PTR(-EINVAL);
+ }
+
+ err = crypto_cipher_setkey(essiv_tfm, salt, saltsize);
+ if (err) {
+ DMERR("Failed to set key for ESSIV cipher\n");
+ crypto_free_cipher(essiv_tfm);
+ return ERR_PTR(err);
+ }
+ return essiv_tfm;
+}
+
+static void crypt_iv_essiv_dtr(struct geniv_ctx *ctx)
+{
+ struct crypto_cipher *essiv_tfm;
+ struct geniv_essiv_private *essiv = &ctx->iv_gen_private.essiv;
+
+ crypto_free_ahash(essiv->hash_tfm);
+ essiv->hash_tfm = NULL;
+
+ kzfree(essiv->salt);
+ essiv->salt = NULL;
+
+ essiv_tfm = ctx->iv_private;
+
+ if (essiv_tfm)
+ crypto_free_cipher(essiv_tfm);
+
+ ctx->iv_private = NULL;
+}
+
+static int crypt_iv_essiv_ctr(struct geniv_ctx *ctx)
+{
+ struct crypto_cipher *essiv_tfm = NULL;
+ struct crypto_ahash *hash_tfm = NULL;
+ u8 *salt = NULL;
+ int err;
+
+ if (!ctx->ivopts) {
+ DMERR("Digest algorithm missing for ESSIV mode\n");
+ return -EINVAL;
+ }
+
+ /* Allocate hash algorithm */
+ hash_tfm = crypto_alloc_ahash(ctx->ivopts, 0, CRYPTO_ALG_ASYNC);
+ if (IS_ERR(hash_tfm)) {
+ err = PTR_ERR(hash_tfm);
+ DMERR("Error initializing ESSIV hash. err=%d\n", err);
+ goto bad;
+ }
+
+ salt = kzalloc(crypto_ahash_digestsize(hash_tfm), GFP_KERNEL);
+ if (!salt) {
+ err = -ENOMEM;
+ goto bad;
+ }
+
+ ctx->iv_gen_private.essiv.salt = salt;
+ ctx->iv_gen_private.essiv.hash_tfm = hash_tfm;
+
+ essiv_tfm = setup_essiv_cpu(ctx, salt,
+ crypto_ahash_digestsize(hash_tfm));
+ if (IS_ERR(essiv_tfm)) {
+ crypt_iv_essiv_dtr(ctx);
+ return PTR_ERR(essiv_tfm);
+ }
+ ctx->iv_private = essiv_tfm;
+
+ return 0;
+
+bad:
+ if (hash_tfm && !IS_ERR(hash_tfm))
+ crypto_free_ahash(hash_tfm);
+ kfree(salt);
+ return err;
+}
+
+static int crypt_iv_essiv_gen(struct geniv_ctx *ctx,
+ struct geniv_req_ctx *rctx,
+ struct geniv_subreq *subreq, u8 *iv)
+{
+ struct crypto_cipher *essiv_tfm = ctx->iv_private;
+
+ memset(iv, 0, ctx->iv_size);
+ *(__le64 *)iv = cpu_to_le64(rctx->iv_sector);
+ crypto_cipher_encrypt_one(essiv_tfm, iv, iv);
+
+ return 0;
+}
+
+static const struct crypt_iv_operations crypt_iv_essiv_ops = {
+ .ctr = crypt_iv_essiv_ctr,
+ .dtr = crypt_iv_essiv_dtr,
+ .init = crypt_iv_essiv_init,
+ .wipe = crypt_iv_essiv_wipe,
+ .generator = crypt_iv_essiv_gen
+};
+
+static int geniv_setkey_set(struct geniv_ctx *ctx)
+{
+ int ret = 0;
+
+ if (ctx->iv_gen_ops && ctx->iv_gen_ops->init)
+ ret = ctx->iv_gen_ops->init(ctx);
+ return ret;
+}
+
+static int geniv_setkey_wipe(struct geniv_ctx *ctx)
+{
+ int ret = 0;
+
+ if (ctx->iv_gen_ops && ctx->iv_gen_ops->wipe) {
+ ret = ctx->iv_gen_ops->wipe(ctx);
+ if (ret)
+ return ret;
+ }
+ return ret;
+}
+
+static int geniv_init_iv(struct geniv_ctx *ctx)
+{
+ int ret = -EINVAL;
+
+ DMDEBUG("IV Generation algorithm : %s\n", ctx->ivmode);
+
+ if (ctx->ivmode == NULL)
+ ctx->iv_gen_ops = NULL;
+ else if (strcmp(ctx->ivmode, "essiv") == 0)
+ ctx->iv_gen_ops = &crypt_iv_essiv_ops;
+ else {
+ ret = -EINVAL;
+ DMERR("Invalid IV mode %s\n", ctx->ivmode);
+ goto end;
+ }
+
+ /* Allocate IV */
+ if (ctx->iv_gen_ops && ctx->iv_gen_ops->ctr) {
+ ret = ctx->iv_gen_ops->ctr(ctx);
+ if (ret < 0) {
+ DMERR("Error creating IV for %s\n", ctx->ivmode);
+ goto end;
+ }
+ }
+
+ /* Initialize IV (set keys for ESSIV etc) */
+ if (ctx->iv_gen_ops && ctx->iv_gen_ops->init) {
+ ret = ctx->iv_gen_ops->init(ctx);
+ if (ret < 0)
+ DMERR("Error creating IV for %s\n", ctx->ivmode);
+ }
+ ret = 0;
+end:
+ return ret;
+}
+
+static void geniv_free_tfms(struct geniv_ctx *ctx)
+{
+ unsigned int i;
+
+ if (!ctx->tfms)
+ return;
+
+ for (i = 0; i < ctx->tfms_count; i++)
+ if (ctx->tfms[i] && !IS_ERR(ctx->tfms[i])) {
+ crypto_free_skcipher(ctx->tfms[i]);
+ ctx->tfms[i] = NULL;
+ }
+
+ kfree(ctx->tfms);
+ ctx->tfms = NULL;
+}
+
+/* Allocate memory for the underlying cipher algorithm. Ex: cbc(aes)
+ */
+
+static int geniv_alloc_tfms(struct crypto_skcipher *parent,
+ struct geniv_ctx *ctx)
+{
+ unsigned int i, reqsize, align;
+ int err = 0;
+
+ ctx->tfms = kcalloc(ctx->tfms_count, sizeof(struct crypto_skcipher *),
+ GFP_KERNEL);
+ if (!ctx->tfms) {
+ err = -ENOMEM;
+ goto end;
+ }
+
+ /* First instance is already allocated in geniv_init_tfm */
+ ctx->tfms[0] = ctx->child;
+ for (i = 1; i < ctx->tfms_count; i++) {
+ ctx->tfms[i] = crypto_alloc_skcipher(ctx->ciphermode, 0, 0);
+ if (IS_ERR(ctx->tfms[i])) {
+ err = PTR_ERR(ctx->tfms[i]);
+ geniv_free_tfms(ctx);
+ goto end;
+ }
+
+ /* Setup the current cipher's request structure */
+ align = crypto_skcipher_alignmask(parent);
+ align &= ~(crypto_tfm_ctx_alignment() - 1);
+ reqsize = align + sizeof(struct geniv_req_ctx) +
+ crypto_skcipher_reqsize(ctx->tfms[i]);
+ crypto_skcipher_set_reqsize(parent, reqsize);
+ }
+
+end:
+ return err;
+}
+
+/* Initialize the cipher's context with the key, ivmode and other parameters.
+ * Also allocate IV generation template ciphers and initialize them.
+ */
+
+static int geniv_setkey_init(struct crypto_skcipher *parent,
+ struct geniv_key_info *info)
+{
+ struct geniv_ctx *ctx = crypto_skcipher_ctx(parent);
+ int ret = -ENOMEM;
+
+ ctx->tfms_count = info->tfms_count;
+ ctx->key = info->key;
+ ctx->key_size = info->key_size;
+ ctx->key_parts = info->key_parts;
+ ctx->ivopts = info->ivopts;
+
+ ret = geniv_alloc_tfms(parent, ctx);
+ if (ret)
+ goto end;
+
+ ret = geniv_init_iv(ctx);
+
+end:
+ return ret;
+}
+
+static int geniv_setkey_tfms(struct crypto_skcipher *parent,
+ struct geniv_ctx *ctx,
+ struct geniv_key_info *info)
+{
+ unsigned int subkey_size;
+ int ret = 0, i;
+
+ /* Ignore extra keys (which are used for IV etc) */
+ subkey_size = (ctx->key_size - ctx->key_extra_size)
+ >> ilog2(ctx->tfms_count);
+
+ for (i = 0; i < ctx->tfms_count; i++) {
+ struct crypto_skcipher *child = ctx->tfms[i];
+ char *subkey = ctx->key + (subkey_size) * i;
+
+ crypto_skcipher_clear_flags(child, CRYPTO_TFM_REQ_MASK);
+ crypto_skcipher_set_flags(child,
+ crypto_skcipher_get_flags(parent) &
+ CRYPTO_TFM_REQ_MASK);
+ ret = crypto_skcipher_setkey(child, subkey, subkey_size);
+ if (ret) {
+ DMERR("Error setting key for tfms[%d]\n", i);
+ break;
+ }
+ crypto_skcipher_set_flags(parent,
+ crypto_skcipher_get_flags(child) &
+ CRYPTO_TFM_RES_MASK);
+ }
+
+ return ret;
+}
+
+static int geniv_setkey(struct crypto_skcipher *parent,
+ const u8 *key, unsigned int keylen)
+{
+ int err = 0;
+ struct geniv_ctx *ctx = crypto_skcipher_ctx(parent);
+ struct geniv_key_info *info = (struct geniv_key_info *) key;
+
+ DMDEBUG("SETKEY Operation : %d\n", info->keyop);
+
+ switch (info->keyop) {
+ case SETKEY_OP_INIT:
+ err = geniv_setkey_init(parent, info);
+ break;
+ case SETKEY_OP_SET:
+ err = geniv_setkey_set(ctx);
+ break;
+ case SETKEY_OP_WIPE:
+ err = geniv_setkey_wipe(ctx);
+ break;
+ }
+
+ if (err)
+ goto end;
+
+ err = geniv_setkey_tfms(parent, ctx, info);
+
+end:
+ return err;
+}
+
+static void geniv_async_done(struct crypto_async_request *async_req, int error);
+
+static int geniv_alloc_subreq(struct skcipher_request *req,
+ struct geniv_ctx *ctx,
+ struct geniv_req_ctx *rctx)
+{
+ int key_index, r = 0;
+ struct skcipher_request *sreq;
+
+ if (!rctx->subreq) {
+ rctx->subreq = mempool_alloc(ctx->subreq_pool, GFP_NOIO);
+ if (!rctx->subreq)
+ r = -ENOMEM;
+ }
+
+ sreq = &rctx->subreq->req;
+ rctx->subreq->rctx = rctx;
+
+ key_index = rctx->iv_sector & (ctx->tfms_count - 1);
+
+ skcipher_request_set_tfm(sreq, ctx->tfms[key_index]);
+ skcipher_request_set_callback(sreq, req->base.flags,
+ geniv_async_done, rctx->subreq);
+ return r;
+}
+
+/* Asynchronous IO completion callback for each sector in a segment. When all
+ * pending i/o are completed the parent cipher's async function is called.
+ */
+
+static void geniv_async_done(struct crypto_async_request *async_req, int error)
+{
+ struct geniv_subreq *subreq =
+ (struct geniv_subreq *) async_req->data;
+ struct geniv_req_ctx *rctx = subreq->rctx;
+ struct skcipher_request *req = rctx->req;
+ struct crypto_skcipher *tfm = crypto_skcipher_reqtfm(req);
+ struct geniv_ctx *ctx = crypto_skcipher_ctx(tfm);
+ u8 *iv;
+
+ /*
+ * A request from crypto driver backlog is going to be processed now,
+ * finish the completion and continue in crypt_convert().
+ * (Callback will be called for the second time for this request.)
+ */
+
+ if (error == -EINPROGRESS) {
+ complete(&rctx->restart);
+ return;
+ }
+
+ iv = iv_of_subreq(ctx, subreq);
+ if (!error && ctx->iv_gen_ops && ctx->iv_gen_ops->post)
+ error = ctx->iv_gen_ops->post(ctx, rctx, subreq, iv);
+
+ mempool_free(subreq, ctx->subreq_pool);
+
+ /* req_pending needs to be checked before req->base.complete is called
+ * as we need 'req_pending' to be equal to 1 to ensure all subrequests
+ * are processed.
+ */
+ if (!atomic_dec_and_test(&rctx->req_pending)) {
+ /* Call the parent cipher's completion function */
+ skcipher_request_complete(req, error);
+ }
+}
+
+static unsigned int geniv_get_sectors(struct scatterlist *sg1,
+ struct scatterlist *sg2,
+ unsigned int segments)
+{
+ unsigned int i, n1, n2, nents;
+
+ n1 = n2 = 0;
+ for (i = 0; i < segments ; i++) {
+ n1 += sg1[i].length >> SECTOR_SHIFT;
+ n1 += (sg1[i].length & ~SECTOR_MASK) ? 1 : 0;
+ }
+
+ for (i = 0; i < segments ; i++) {
+ n2 += sg2[i].length >> SECTOR_SHIFT;
+ n2 += (sg2[i].length & ~SECTOR_MASK) ? 1 : 0;
+ }
+
+ nents = n1 > n2 ? n1 : n2;
+ return nents;
+}
+
+/* Iterate scatterlist of segments to retrieve the 512-byte sectors so that
+ * unique IVs could be generated for each 512-byte sector. This split may not
+ * be necessary e.g. when these ciphers are modelled in hardware, where it can
+ * make use of the hardware's IV generation capabilities.
+ */
+
+static int geniv_iter_block(struct skcipher_request *req,
+ struct geniv_subreq *subreq,
+ struct geniv_req_ctx *rctx,
+ unsigned int *seg_no,
+ unsigned int *done)
+
+{
+ unsigned int srcoff, dstoff, len, rem;
+ struct scatterlist *src1, *dst1, *src2, *dst2;
+
+ if (unlikely(*seg_no >= rctx->nents))
+ return 0; /* done */
+
+ src1 = &req->src[*seg_no];
+ dst1 = &req->dst[*seg_no];
+ src2 = &subreq->src;
+ dst2 = &subreq->dst;
+
+ if (*done >= src1->length) {
+ (*seg_no)++;
+
+ if (*seg_no >= rctx->nents)
+ return 0; /* done */
+
+ src1 = &req->src[*seg_no];
+ dst1 = &req->dst[*seg_no];
+ *done = 0;
+ }
+
+ srcoff = src1->offset + *done;
+ dstoff = dst1->offset + *done;
+ rem = src1->length - *done;
+
+ len = rem > SECTOR_SIZE ? SECTOR_SIZE : rem;
+
+ DMDEBUG("segment:(%d/%u), srcoff:%d, dstoff:%d, done:%d, rem:%d\n",
+ *seg_no + 1, rctx->nents, srcoff, dstoff, *done, rem);
+
+ sg_init_table(src2, 1);
+ sg_set_page(src2, sg_page(src1), len, srcoff);
+ sg_init_table(dst2, 1);
+ sg_set_page(dst2, sg_page(dst1), len, dstoff);
+
+ *done += len;
+
+ return len; /* bytes returned */
+}
+
+/* Common encryt/decrypt function for geniv template cipher. Before the crypto
+ * operation, it splits the memory segments (in the scatterlist) into 512 byte
+ * sectors. The initialization vector(IV) used is based on a unique sector
+ * number which is generated here.
+ */
+static int geniv_crypt(struct skcipher_request *req, int encrypt)
+{
+ struct crypto_skcipher *tfm = crypto_skcipher_reqtfm(req);
+ struct geniv_ctx *ctx = crypto_skcipher_ctx(tfm);
+ struct geniv_req_ctx *rctx = geniv_req_ctx(req);
+ struct geniv_req_info *rinfo = (struct geniv_req_info *) req->iv;
+ int i, bytes, cryptlen, ret = 0;
+ unsigned int sectors, segno = 0, done = 0;
+ char *str __maybe_unused = encrypt ? "encrypt" : "decrypt";
+ u8 *iv;
+
+ /* Instance of 'struct geniv_req_info' is stored in IV ptr */
+ rctx->is_write = encrypt;
+ rctx->iv_sector = rinfo->iv_sector;
+ rctx->nents = rinfo->nents;
+ rctx->req = req;
+ rctx->subreq = NULL;
+ cryptlen = req->cryptlen;
+
+ DMDEBUG("geniv:%s: starting sector=%d, #segments=%u\n", str,
+ (unsigned int) rctx->iv_sector, rctx->nents);
+
+ sectors = geniv_get_sectors(req->src, req->dst, rctx->nents);
+
+ init_completion(&rctx->restart);
+ atomic_set(&rctx->req_pending, 1);
+
+ for (i = 0; i < sectors; i++) {
+ struct geniv_subreq *subreq;
+
+ ret = geniv_alloc_subreq(req, ctx, rctx);
+ if (ret)
+ goto end;
+
+ subreq = rctx->subreq;
+ subreq->rctx = rctx;
+ iv = iv_of_subreq(ctx, subreq);
+
+ atomic_inc(&rctx->req_pending);
+ bytes = geniv_iter_block(req, subreq, rctx, &segno, &done);
+
+ if (bytes == 0)
+ break;
+
+ cryptlen -= bytes;
+
+ if (ctx->iv_gen_ops)
+ ret = ctx->iv_gen_ops->generator(ctx, rctx, subreq, iv);
+
+ if (ret < 0) {
+ DMERR("Error in generating IV ret: %d\n", ret);
+ goto end;
+ }
+
+ skcipher_request_set_crypt(&subreq->req, &subreq->src,
+ &subreq->dst, bytes, iv);
+
+ if (encrypt)
+ ret = crypto_skcipher_encrypt(&subreq->req);
+
+ else
+ ret = crypto_skcipher_decrypt(&subreq->req);
+
+ if (!ret && ctx->iv_gen_ops && ctx->iv_gen_ops->post)
+ ret = ctx->iv_gen_ops->post(ctx, rctx, subreq, iv);
+
+ switch (ret) {
+ /*
+ * The request was queued by a crypto driver
+ * but the driver request queue is full, let's wait.
+ */
+ case -EBUSY:
+ wait_for_completion(&rctx->restart);
+ reinit_completion(&rctx->restart);
+ /* fall through */
+ /*
+ * The request is queued and processed asynchronously,
+ * completion function geniv_async_done() is called.
+ */
+ case -EINPROGRESS:
+ /* Marking this NULL lets the creation of a new sub-
+ * request when 'geniv_alloc_subreq' is called.
+ */
+ rctx->subreq = NULL;
+ rctx->iv_sector++;
+ cond_resched();
+ break;
+ /*
+ * The request was already processed (synchronously).
+ */
+ case 0:
+ atomic_dec(&rctx->req_pending);
+ rctx->iv_sector++;
+ cond_resched();
+ continue;
+
+ /* There was an error while processing the request. */
+ default:
+ atomic_dec(&rctx->req_pending);
+ return ret;
+ }
+
+ if (ret)
+ break;
+ }
+
+ if (rctx->subreq && atomic_read(&rctx->req_pending) == 1) {
+ DMDEBUG("geniv:%s: Freeing sub request\n", str);
+ mempool_free(rctx->subreq, ctx->subreq_pool);
+ }
+
+end:
+ return ret;
+}
+
+static int geniv_encrypt(struct skcipher_request *req)
+{
+ return geniv_crypt(req, 1);
+}
+
+static int geniv_decrypt(struct skcipher_request *req)
+{
+ return geniv_crypt(req, 0);
+}
+
+static int geniv_init_tfm(struct crypto_skcipher *tfm)
+{
+ struct geniv_ctx *ctx = crypto_skcipher_ctx(tfm);
+ unsigned int reqsize, align;
+ size_t iv_size_padding;
+ char *algname, *chainmode;
+ int psize, ret = 0;
+
+ algname = (char *) crypto_tfm_alg_name(crypto_skcipher_tfm(tfm));
+ ctx->ciphermode = kmalloc(CRYPTO_MAX_ALG_NAME, GFP_KERNEL);
+ if (!ctx->ciphermode) {
+ ret = -ENOMEM;
+ goto out;
+ }
+
+ ctx->algname = kmalloc(CRYPTO_MAX_ALG_NAME, GFP_KERNEL);
+ if (!ctx->algname) {
+ ret = -ENOMEM;
+ goto free_ciphermode;
+ }
+
+ strlcpy(ctx->algname, algname, CRYPTO_MAX_ALG_NAME);
+ algname = ctx->algname;
+
+ /* Parse the algorithm name 'ivmode(chainmode(cipher))' */
+ ctx->ivmode = strsep(&algname, "(");
+ chainmode = strsep(&algname, "(");
+ ctx->cipher = strsep(&algname, ")");
+
+ snprintf(ctx->ciphermode, CRYPTO_MAX_ALG_NAME, "%s(%s)",
+ chainmode, ctx->cipher);
+
+ DMDEBUG("ciphermode=%s, ivmode=%s\n", ctx->ciphermode, ctx->ivmode);
+
+ /*
+ * Usually the underlying cipher instances are spawned here, but since
+ * the value of tfms_count (which is equal to the key_count) is not
+ * known yet, create only one instance and delay the creation of the
+ * rest of the instances of the underlying cipher 'cbc(aes)' until
+ * the setkey operation is invoked.
+ * The first instance created i.e. ctx->child will later be assigned as
+ * the 1st element in the array ctx->tfms. Creation of atleast one
+ * instance of the cipher is necessary to be created here to uncover
+ * any errors earlier than during the setkey operation later where the
+ * remaining instances are created.
+ */
+ ctx->child = crypto_alloc_skcipher(ctx->ciphermode, 0, 0);
+ if (IS_ERR(ctx->child)) {
+ ret = PTR_ERR(ctx->child);
+ DMERR("Failed to create skcipher %s. err %d\n",
+ ctx->ciphermode, ret);
+ goto free_algname;
+ }
+
+ /* Setup the current cipher's request structure */
+ align = crypto_skcipher_alignmask(tfm);
+ align &= ~(crypto_tfm_ctx_alignment() - 1);
+ reqsize = align + sizeof(struct geniv_req_ctx)
+ + crypto_skcipher_reqsize(ctx->child);
+ crypto_skcipher_set_reqsize(tfm, reqsize);
+
+ ctx->iv_start = sizeof(struct geniv_subreq);
+ ctx->iv_start += crypto_skcipher_reqsize(tfm);
+
+ ctx->iv_size = crypto_skcipher_ivsize(tfm);
+ /* at least a 64 bit sector number should fit in our buffer */
+ if (ctx->iv_size)
+ ctx->iv_size = max(ctx->iv_size,
+ (unsigned int)(sizeof(u64) / sizeof(u8)));
+
+ if (crypto_skcipher_alignmask(tfm) < CRYPTO_MINALIGN) {
+ /* Allocate the padding exactly */
+ iv_size_padding = -ctx->iv_start
+ & crypto_skcipher_alignmask(tfm);
+ } else {
+ /*
+ * If the cipher requires greater alignment than kmalloc
+ * alignment, we don't know the exact position of the
+ * initialization vector. We must assume worst case.
+ */
+ iv_size_padding = crypto_skcipher_alignmask(tfm);
+ }
+
+ /* create memory pool for sub-request structure */
+ psize = ctx->iv_start + iv_size_padding + ctx->iv_size;
+
+ ctx->subreq_pool = mempool_create_kmalloc_pool(MIN_IOS, psize);
+ if (!ctx->subreq_pool) {
+ ret = -ENOMEM;
+ DMERR("Could not allocate crypt sub-request mempool\n");
+ goto free_skcipher;
+ }
+out:
+ return ret;
+
+free_skcipher:
+ crypto_free_skcipher(ctx->child);
+free_algname:
+ kfree(ctx->algname);
+free_ciphermode:
+ kfree(ctx->ciphermode);
+ goto out;
+}
+
+static void geniv_exit_tfm(struct crypto_skcipher *tfm)
+{
+ struct geniv_ctx *ctx = crypto_skcipher_ctx(tfm);
+
+ if (ctx->iv_gen_ops && ctx->iv_gen_ops->dtr)
+ ctx->iv_gen_ops->dtr(ctx);
+
+ mempool_destroy(ctx->subreq_pool);
+ geniv_free_tfms(ctx);
+ kfree(ctx->ciphermode);
+ kfree(ctx->algname);
+}
+
+static void geniv_free(struct skcipher_instance *inst)
+{
+ struct crypto_skcipher_spawn *spawn = skcipher_instance_ctx(inst);
+
+ crypto_drop_skcipher(spawn);
+ kfree(inst);
+}
+
+static int geniv_create(struct crypto_template *tmpl,
+ struct rtattr **tb, char *algname)
+{
+ struct crypto_attr_type *algt;
+ struct skcipher_instance *inst;
+ struct skcipher_alg *alg;
+ struct crypto_skcipher_spawn *spawn;
+ const char *cipher_name;
+ int err;
+
+ algt = crypto_get_attr_type(tb);
+
+ if (IS_ERR(algt))
+ return PTR_ERR(algt);
+
+ if ((algt->type ^ CRYPTO_ALG_TYPE_SKCIPHER) & algt->mask)
+ return -EINVAL;
+
+ cipher_name = crypto_attr_alg_name(tb[1]);
+
+ if (IS_ERR(cipher_name))
+ return PTR_ERR(cipher_name);
+
+ inst = kzalloc(sizeof(*inst) + sizeof(*spawn), GFP_KERNEL);
+ if (!inst)
+ return -ENOMEM;
+
+ spawn = skcipher_instance_ctx(inst);
+
+ crypto_set_skcipher_spawn(spawn, skcipher_crypto_instance(inst));
+ err = crypto_grab_skcipher(spawn, cipher_name, 0,
+ crypto_requires_sync(algt->type,
+ algt->mask));
+
+ if (err)
+ goto err_free_inst;
+
+ alg = crypto_spawn_skcipher_alg(spawn);
+
+ err = -EINVAL;
+
+ /* Only support blocks of size which is of a power of 2 */
+ if (!is_power_of_2(alg->base.cra_blocksize))
+ goto err_drop_spawn;
+
+ /* algname: essiv, base.cra_name: cbc(aes) */
+ err = -ENAMETOOLONG;
+ if (snprintf(inst->alg.base.cra_name, CRYPTO_MAX_ALG_NAME, "%s(%s)",
+ algname, alg->base.cra_name) >= CRYPTO_MAX_ALG_NAME)
+ goto err_drop_spawn;
+ if (snprintf(inst->alg.base.cra_driver_name, CRYPTO_MAX_ALG_NAME,
+ "%s(%s)", algname, alg->base.cra_driver_name) >=
+ CRYPTO_MAX_ALG_NAME)
+ goto err_drop_spawn;
+
+ inst->alg.base.cra_flags = CRYPTO_ALG_TYPE_BLKCIPHER;
+ inst->alg.base.cra_priority = alg->base.cra_priority;
+ inst->alg.base.cra_blocksize = alg->base.cra_blocksize;
+ inst->alg.base.cra_alignmask = alg->base.cra_alignmask;
+ inst->alg.base.cra_flags = alg->base.cra_flags & CRYPTO_ALG_ASYNC;
+ inst->alg.ivsize = alg->base.cra_blocksize;
+ inst->alg.chunksize = crypto_skcipher_alg_chunksize(alg);
+ inst->alg.min_keysize = crypto_skcipher_alg_min_keysize(alg);
+ inst->alg.max_keysize = crypto_skcipher_alg_max_keysize(alg);
+
+ inst->alg.setkey = geniv_setkey;
+ inst->alg.encrypt = geniv_encrypt;
+ inst->alg.decrypt = geniv_decrypt;
+
+ inst->alg.base.cra_ctxsize = sizeof(struct geniv_ctx);
+
+ inst->alg.init = geniv_init_tfm;
+ inst->alg.exit = geniv_exit_tfm;
+
+ inst->free = geniv_free;
+
+ err = skcipher_register_instance(tmpl, inst);
+ if (err)
+ goto err_drop_spawn;
+
+out:
+ return err;
+
+err_drop_spawn:
+ crypto_drop_skcipher(spawn);
+err_free_inst:
+ kfree(inst);
+ goto out;
+}
+
+static int crypto_essiv_create(struct crypto_template *tmpl,
+ struct rtattr **tb)
+{
+ return geniv_create(tmpl, tb, "essiv");
+}
+
+static struct crypto_template crypto_essiv_tmpl = {
+ .name = "essiv",
+ .create = crypto_essiv_create,
+ .module = THIS_MODULE,
+};
+
+static int __init essiv_init(void)
+{
+ return crypto_register_template(&crypto_essiv_tmpl);
+}
+
+static void __exit essiv_exit(void)
+{
+ crypto_unregister_template(&crypto_essiv_tmpl);
+}
+
+module_init(essiv_init);
+module_exit(essiv_exit);
+
+MODULE_AUTHOR("Binoy Jayan <binoy.jayan@linaro.org>");
+MODULE_DESCRIPTION(DM_NAME " essiv implementation for cryptocell 712");
+MODULE_LICENSE("GPL");
--
Binoy Jayan
^ permalink raw reply related
* [PATCH v6 1/2] crypto: Add IV generation algorithms
From: Binoy Jayan @ 2017-06-22 4:41 UTC (permalink / raw)
To: Mark Brown
Cc: Arnd Bergmann, linux-crypto, linux-kernel, dm-devel, linux-raid,
Rajendra, Binoy Jayan
In-Reply-To: <1498106510-19793-1-git-send-email-binoy.jayan@linaro.org>
Just for reference. Not for merging.
Currently, the iv generation algorithms are implemented in dm-crypt.c.
The goal is to move these algorithms from the dm layer to the kernel
crypto layer by implementing them as template ciphers so they can be
implemented in hardware for performance. As part of this patchset, the
iv-generation code is moved from the dm layer to the crypto layer and
adapt the dm-layer to send a whole 'bio' (as defined in the block layer)
at a time. Each bio contains an in memory representation of physically
contiguous disk blocks. The dm layer sets up a chained scatterlist of
these blocks split into physically contiguous segments in memory so that
DMA can be performed. Also, the key management code is moved from dm layer
to the cryto layer since the key selection for encrypting neighboring
sectors depend on the keycount.
Synchronous crypto requests to encrypt/decrypt a sector are processed
sequentially. Asynchronous requests if processed in parallel, are freed
in the async callback. The storage space for the initialization vectors
are allocated in the iv generator implementations.
Interface to the crypto layer - include/crypto/geniv.h
Signed-off-by: Binoy Jayan <binoy.jayan@linaro.org>
---
drivers/md/dm-crypt.c | 1939 ++++++++++++++++++++++++++++++++++--------------
include/crypto/geniv.h | 46 ++
2 files changed, 1432 insertions(+), 553 deletions(-)
create mode 100644 include/crypto/geniv.h
diff --git a/drivers/md/dm-crypt.c b/drivers/md/dm-crypt.c
index 389a363..bef54f5 100644
--- a/drivers/md/dm-crypt.c
+++ b/drivers/md/dm-crypt.c
@@ -32,170 +32,120 @@
#include <crypto/algapi.h>
#include <crypto/skcipher.h>
#include <keys/user-type.h>
-
#include <linux/device-mapper.h>
-
-#define DM_MSG_PREFIX "crypt"
-
-/*
- * context holding the current state of a multi-part conversion
- */
-struct convert_context {
- struct completion restart;
- struct bio *bio_in;
- struct bio *bio_out;
- struct bvec_iter iter_in;
- struct bvec_iter iter_out;
- sector_t cc_sector;
- atomic_t cc_pending;
- struct skcipher_request *req;
+#include <crypto/internal/skcipher.h>
+#include <linux/backing-dev.h>
+#include <linux/log2.h>
+#include <crypto/geniv.h>
+
+#define DM_MSG_PREFIX "crypt"
+#define MAX_SG_LIST (BIO_MAX_PAGES * 8)
+#define MIN_IOS 64
+#define LMK_SEED_SIZE 64 /* hash + 0 */
+#define TCW_WHITENING_SIZE 16
+
+struct geniv_ctx;
+struct geniv_req_ctx;
+
+/* Sub request for each of the skcipher_request's for a segment */
+struct geniv_subreq {
+ struct scatterlist src;
+ struct scatterlist dst;
+ struct geniv_req_ctx *rctx;
+ struct skcipher_request req CRYPTO_MINALIGN_ATTR;
};
-/*
- * per bio private data
- */
-struct dm_crypt_io {
- struct crypt_config *cc;
- struct bio *base_bio;
- struct work_struct work;
-
- struct convert_context ctx;
-
- atomic_t io_pending;
- int error;
- sector_t sector;
-
- struct rb_node rb_node;
-} CRYPTO_MINALIGN_ATTR;
-
-struct dm_crypt_request {
- struct convert_context *ctx;
- struct scatterlist sg_in;
- struct scatterlist sg_out;
+struct geniv_req_ctx {
+ struct geniv_subreq *subreq;
+ int is_write;
sector_t iv_sector;
+ unsigned int nents;
+ struct completion restart;
+ atomic_t req_pending;
+ struct skcipher_request *req;
};
-struct crypt_config;
-
struct crypt_iv_operations {
- int (*ctr)(struct crypt_config *cc, struct dm_target *ti,
- const char *opts);
- void (*dtr)(struct crypt_config *cc);
- int (*init)(struct crypt_config *cc);
- int (*wipe)(struct crypt_config *cc);
- int (*generator)(struct crypt_config *cc, u8 *iv,
- struct dm_crypt_request *dmreq);
- int (*post)(struct crypt_config *cc, u8 *iv,
- struct dm_crypt_request *dmreq);
+ int (*ctr)(struct geniv_ctx *ctx);
+ void (*dtr)(struct geniv_ctx *ctx);
+ int (*init)(struct geniv_ctx *ctx);
+ int (*wipe)(struct geniv_ctx *ctx);
+ int (*generator)(struct geniv_ctx *ctx,
+ struct geniv_req_ctx *rctx,
+ struct geniv_subreq *subreq, u8 *iv);
+ int (*post)(struct geniv_ctx *ctx,
+ struct geniv_req_ctx *rctx,
+ struct geniv_subreq *subreq, u8 *iv);
};
-struct iv_essiv_private {
+struct geniv_essiv_private {
struct crypto_ahash *hash_tfm;
u8 *salt;
};
-struct iv_benbi_private {
+struct geniv_benbi_private {
int shift;
};
-#define LMK_SEED_SIZE 64 /* hash + 0 */
-struct iv_lmk_private {
+struct geniv_lmk_private {
struct crypto_shash *hash_tfm;
u8 *seed;
};
-#define TCW_WHITENING_SIZE 16
-struct iv_tcw_private {
+struct geniv_tcw_private {
struct crypto_shash *crc32_tfm;
u8 *iv_seed;
u8 *whitening;
};
-/*
- * Crypt: maps a linear range of a block device
- * and encrypts / decrypts at the same time.
- */
-enum flags { DM_CRYPT_SUSPENDED, DM_CRYPT_KEY_VALID,
- DM_CRYPT_SAME_CPU, DM_CRYPT_NO_OFFLOAD };
-
-/*
- * The fields in here must be read only after initialization.
- */
-struct crypt_config {
- struct dm_dev *dev;
- sector_t start;
-
- /*
- * pool for per bio private data, crypto requests and
- * encryption requeusts/buffer pages
- */
- mempool_t *req_pool;
- mempool_t *page_pool;
- struct bio_set *bs;
- struct mutex bio_alloc_lock;
-
- struct workqueue_struct *io_queue;
- struct workqueue_struct *crypt_queue;
-
- struct task_struct *write_thread;
- wait_queue_head_t write_thread_wait;
- struct rb_root write_tree;
-
+struct geniv_ctx {
+ unsigned int tfms_count;
+ struct crypto_skcipher *child;
+ struct crypto_skcipher **tfms;
+ char *ivmode;
+ unsigned int iv_size;
+ unsigned int iv_start;
+ char *algname;
+ char *ivopts;
char *cipher;
- char *cipher_string;
- char *key_string;
-
+ char *ciphermode;
const struct crypt_iv_operations *iv_gen_ops;
union {
- struct iv_essiv_private essiv;
- struct iv_benbi_private benbi;
- struct iv_lmk_private lmk;
- struct iv_tcw_private tcw;
+ struct geniv_essiv_private essiv;
+ struct geniv_benbi_private benbi;
+ struct geniv_lmk_private lmk;
+ struct geniv_tcw_private tcw;
} iv_gen_private;
- sector_t iv_offset;
- unsigned int iv_size;
-
- /* ESSIV: struct crypto_cipher *essiv_tfm */
void *iv_private;
- struct crypto_skcipher **tfms;
- unsigned tfms_count;
-
- /*
- * Layout of each crypto request:
- *
- * struct skcipher_request
- * context
- * padding
- * struct dm_crypt_request
- * padding
- * IV
- *
- * The padding is added so that dm_crypt_request and the IV are
- * correctly aligned.
- */
- unsigned int dmreq_start;
-
- unsigned int per_bio_data_size;
-
- unsigned long flags;
+ struct crypto_skcipher *tfm;
+ mempool_t *subreq_pool;
unsigned int key_size;
+ unsigned int key_extra_size;
unsigned int key_parts; /* independent parts in key buffer */
- unsigned int key_extra_size; /* additional keys length */
- u8 key[0];
+ enum setkey_op keyop;
+ char *msg;
+ u8 *key;
};
-#define MIN_IOS 64
+static struct crypto_skcipher *any_tfm(struct geniv_ctx *ctx)
+{
+ return ctx->tfms[0];
+}
-static void clone_init(struct dm_crypt_io *, struct bio *);
-static void kcryptd_queue_crypt(struct dm_crypt_io *io);
-static u8 *iv_of_dmreq(struct crypt_config *cc, struct dm_crypt_request *dmreq);
+static inline
+struct geniv_req_ctx *geniv_req_ctx(struct skcipher_request *req)
+{
+ struct crypto_skcipher *tfm = crypto_skcipher_reqtfm(req);
+ unsigned long align = crypto_skcipher_alignmask(tfm);
-/*
- * Use this to access cipher attributes that are the same for each CPU.
- */
-static struct crypto_skcipher *any_tfm(struct crypt_config *cc)
+ return (void *) PTR_ALIGN((u8 *) skcipher_request_ctx(req), align + 1);
+}
+
+static u8 *iv_of_subreq(struct geniv_ctx *ctx, struct geniv_subreq *subreq)
{
- return cc->tfms[0];
+ struct crypto_skcipher *tfm = crypto_skcipher_reqtfm(subreq->rctx->req);
+ return (u8*) ALIGN((unsigned long) ((char *) subreq + ctx->iv_start),
+ crypto_skcipher_alignmask(tfm) + 1);
}
/*
@@ -245,44 +195,46 @@ static struct crypto_skcipher *any_tfm(struct crypt_config *cc)
* http://article.gmane.org/gmane.linux.kernel.device-mapper.dm-crypt/454
*/
-static int crypt_iv_plain_gen(struct crypt_config *cc, u8 *iv,
- struct dm_crypt_request *dmreq)
+static int crypt_iv_plain_gen(struct geniv_ctx *ctx,
+ struct geniv_req_ctx *rctx,
+ struct geniv_subreq *subreq, u8 *iv)
{
- memset(iv, 0, cc->iv_size);
- *(__le32 *)iv = cpu_to_le32(dmreq->iv_sector & 0xffffffff);
+ memset(iv, 0, ctx->iv_size);
+ *(__le32 *)iv = cpu_to_le32(rctx->iv_sector & 0xffffffff);
return 0;
}
-static int crypt_iv_plain64_gen(struct crypt_config *cc, u8 *iv,
- struct dm_crypt_request *dmreq)
+static int crypt_iv_plain64_gen(struct geniv_ctx *ctx,
+ struct geniv_req_ctx *rctx,
+ struct geniv_subreq *subreq, u8 *iv)
{
- memset(iv, 0, cc->iv_size);
- *(__le64 *)iv = cpu_to_le64(dmreq->iv_sector);
+ memset(iv, 0, ctx->iv_size);
+ *(__le64 *)iv = cpu_to_le64(rctx->iv_sector);
return 0;
}
/* Initialise ESSIV - compute salt but no local memory allocations */
-static int crypt_iv_essiv_init(struct crypt_config *cc)
+static int crypt_iv_essiv_init(struct geniv_ctx *ctx)
{
- struct iv_essiv_private *essiv = &cc->iv_gen_private.essiv;
- AHASH_REQUEST_ON_STACK(req, essiv->hash_tfm);
+ struct geniv_essiv_private *essiv = &ctx->iv_gen_private.essiv;
struct scatterlist sg;
struct crypto_cipher *essiv_tfm;
int err;
+ AHASH_REQUEST_ON_STACK(req, essiv->hash_tfm);
- sg_init_one(&sg, cc->key, cc->key_size);
+ sg_init_one(&sg, ctx->key, ctx->key_size);
ahash_request_set_tfm(req, essiv->hash_tfm);
ahash_request_set_callback(req, CRYPTO_TFM_REQ_MAY_SLEEP, NULL, NULL);
- ahash_request_set_crypt(req, &sg, essiv->salt, cc->key_size);
+ ahash_request_set_crypt(req, &sg, essiv->salt, ctx->key_size);
err = crypto_ahash_digest(req);
ahash_request_zero(req);
if (err)
return err;
- essiv_tfm = cc->iv_private;
+ essiv_tfm = ctx->iv_private;
err = crypto_cipher_setkey(essiv_tfm, essiv->salt,
crypto_ahash_digestsize(essiv->hash_tfm));
@@ -293,16 +245,16 @@ static int crypt_iv_essiv_init(struct crypt_config *cc)
}
/* Wipe salt and reset key derived from volume key */
-static int crypt_iv_essiv_wipe(struct crypt_config *cc)
+static int crypt_iv_essiv_wipe(struct geniv_ctx *ctx)
{
- struct iv_essiv_private *essiv = &cc->iv_gen_private.essiv;
- unsigned salt_size = crypto_ahash_digestsize(essiv->hash_tfm);
+ struct geniv_essiv_private *essiv = &ctx->iv_gen_private.essiv;
+ unsigned int salt_size = crypto_ahash_digestsize(essiv->hash_tfm);
struct crypto_cipher *essiv_tfm;
int r, err = 0;
memset(essiv->salt, 0, salt_size);
- essiv_tfm = cc->iv_private;
+ essiv_tfm = ctx->iv_private;
r = crypto_cipher_setkey(essiv_tfm, essiv->salt, salt_size);
if (r)
err = r;
@@ -311,42 +263,40 @@ static int crypt_iv_essiv_wipe(struct crypt_config *cc)
}
/* Set up per cpu cipher state */
-static struct crypto_cipher *setup_essiv_cpu(struct crypt_config *cc,
- struct dm_target *ti,
- u8 *salt, unsigned saltsize)
+static struct crypto_cipher *setup_essiv_cpu(struct geniv_ctx *ctx,
+ u8 *salt, unsigned int saltsize)
{
struct crypto_cipher *essiv_tfm;
int err;
/* Setup the essiv_tfm with the given salt */
- essiv_tfm = crypto_alloc_cipher(cc->cipher, 0, CRYPTO_ALG_ASYNC);
+ essiv_tfm = crypto_alloc_cipher(ctx->cipher, 0, CRYPTO_ALG_ASYNC);
+
if (IS_ERR(essiv_tfm)) {
- ti->error = "Error allocating crypto tfm for ESSIV";
+ DMERR("Error allocating crypto tfm for ESSIV\n");
return essiv_tfm;
}
if (crypto_cipher_blocksize(essiv_tfm) !=
- crypto_skcipher_ivsize(any_tfm(cc))) {
- ti->error = "Block size of ESSIV cipher does "
- "not match IV size of block cipher";
+ crypto_skcipher_ivsize(any_tfm(ctx))) {
+ DMERR("Block size of ESSIV cipher does not match IV size of block cipher\n");
crypto_free_cipher(essiv_tfm);
return ERR_PTR(-EINVAL);
}
err = crypto_cipher_setkey(essiv_tfm, salt, saltsize);
if (err) {
- ti->error = "Failed to set key for ESSIV cipher";
+ DMERR("Failed to set key for ESSIV cipher\n");
crypto_free_cipher(essiv_tfm);
return ERR_PTR(err);
}
-
return essiv_tfm;
}
-static void crypt_iv_essiv_dtr(struct crypt_config *cc)
+static void crypt_iv_essiv_dtr(struct geniv_ctx *ctx)
{
struct crypto_cipher *essiv_tfm;
- struct iv_essiv_private *essiv = &cc->iv_gen_private.essiv;
+ struct geniv_essiv_private *essiv = &ctx->iv_gen_private.essiv;
crypto_free_ahash(essiv->hash_tfm);
essiv->hash_tfm = NULL;
@@ -354,52 +304,50 @@ static void crypt_iv_essiv_dtr(struct crypt_config *cc)
kzfree(essiv->salt);
essiv->salt = NULL;
- essiv_tfm = cc->iv_private;
+ essiv_tfm = ctx->iv_private;
if (essiv_tfm)
crypto_free_cipher(essiv_tfm);
- cc->iv_private = NULL;
+ ctx->iv_private = NULL;
}
-static int crypt_iv_essiv_ctr(struct crypt_config *cc, struct dm_target *ti,
- const char *opts)
+static int crypt_iv_essiv_ctr(struct geniv_ctx *ctx)
{
struct crypto_cipher *essiv_tfm = NULL;
struct crypto_ahash *hash_tfm = NULL;
u8 *salt = NULL;
int err;
- if (!opts) {
- ti->error = "Digest algorithm missing for ESSIV mode";
+ if (!ctx->ivopts) {
+ DMERR("Digest algorithm missing for ESSIV mode\n");
return -EINVAL;
}
/* Allocate hash algorithm */
- hash_tfm = crypto_alloc_ahash(opts, 0, CRYPTO_ALG_ASYNC);
+ hash_tfm = crypto_alloc_ahash(ctx->ivopts, 0, CRYPTO_ALG_ASYNC);
if (IS_ERR(hash_tfm)) {
- ti->error = "Error initializing ESSIV hash";
err = PTR_ERR(hash_tfm);
+ DMERR("Error initializing ESSIV hash. err=%d\n", err);
goto bad;
}
salt = kzalloc(crypto_ahash_digestsize(hash_tfm), GFP_KERNEL);
if (!salt) {
- ti->error = "Error kmallocing salt storage in ESSIV";
err = -ENOMEM;
goto bad;
}
- cc->iv_gen_private.essiv.salt = salt;
- cc->iv_gen_private.essiv.hash_tfm = hash_tfm;
+ ctx->iv_gen_private.essiv.salt = salt;
+ ctx->iv_gen_private.essiv.hash_tfm = hash_tfm;
- essiv_tfm = setup_essiv_cpu(cc, ti, salt,
+ essiv_tfm = setup_essiv_cpu(ctx, salt,
crypto_ahash_digestsize(hash_tfm));
if (IS_ERR(essiv_tfm)) {
- crypt_iv_essiv_dtr(cc);
+ crypt_iv_essiv_dtr(ctx);
return PTR_ERR(essiv_tfm);
}
- cc->iv_private = essiv_tfm;
+ ctx->iv_private = essiv_tfm;
return 0;
@@ -410,70 +358,69 @@ static int crypt_iv_essiv_ctr(struct crypt_config *cc, struct dm_target *ti,
return err;
}
-static int crypt_iv_essiv_gen(struct crypt_config *cc, u8 *iv,
- struct dm_crypt_request *dmreq)
+static int crypt_iv_essiv_gen(struct geniv_ctx *ctx,
+ struct geniv_req_ctx *rctx,
+ struct geniv_subreq *subreq, u8 *iv)
{
- struct crypto_cipher *essiv_tfm = cc->iv_private;
+ struct crypto_cipher *essiv_tfm = ctx->iv_private;
- memset(iv, 0, cc->iv_size);
- *(__le64 *)iv = cpu_to_le64(dmreq->iv_sector);
+ memset(iv, 0, ctx->iv_size);
+ *(__le64 *)iv = cpu_to_le64(rctx->iv_sector);
crypto_cipher_encrypt_one(essiv_tfm, iv, iv);
return 0;
}
-static int crypt_iv_benbi_ctr(struct crypt_config *cc, struct dm_target *ti,
- const char *opts)
+static int crypt_iv_benbi_ctr(struct geniv_ctx *ctx)
{
- unsigned bs = crypto_skcipher_blocksize(any_tfm(cc));
+ unsigned int bs = crypto_skcipher_blocksize(any_tfm(ctx));
int log = ilog2(bs);
/* we need to calculate how far we must shift the sector count
- * to get the cipher block count, we use this shift in _gen */
+ * to get the cipher block count, we use this shift in _gen
+ */
if (1 << log != bs) {
- ti->error = "cypher blocksize is not a power of 2";
+ DMERR("cypher blocksize is not a power of 2\n");
return -EINVAL;
}
if (log > 9) {
- ti->error = "cypher blocksize is > 512";
+ DMERR("cypher blocksize is > 512\n");
return -EINVAL;
}
- cc->iv_gen_private.benbi.shift = 9 - log;
+ ctx->iv_gen_private.benbi.shift = 9 - log;
return 0;
}
-static void crypt_iv_benbi_dtr(struct crypt_config *cc)
-{
-}
-
-static int crypt_iv_benbi_gen(struct crypt_config *cc, u8 *iv,
- struct dm_crypt_request *dmreq)
+static int crypt_iv_benbi_gen(struct geniv_ctx *ctx,
+ struct geniv_req_ctx *rctx,
+ struct geniv_subreq *subreq, u8 *iv)
{
__be64 val;
- memset(iv, 0, cc->iv_size - sizeof(u64)); /* rest is cleared below */
+ memset(iv, 0, ctx->iv_size - sizeof(u64)); /* rest is cleared below */
- val = cpu_to_be64(((u64)dmreq->iv_sector << cc->iv_gen_private.benbi.shift) + 1);
- put_unaligned(val, (__be64 *)(iv + cc->iv_size - sizeof(u64)));
+ val = cpu_to_be64(((u64) rctx->iv_sector <<
+ ctx->iv_gen_private.benbi.shift) + 1);
+ put_unaligned(val, (__be64 *)(iv + ctx->iv_size - sizeof(u64)));
return 0;
}
-static int crypt_iv_null_gen(struct crypt_config *cc, u8 *iv,
- struct dm_crypt_request *dmreq)
+static int crypt_iv_null_gen(struct geniv_ctx *ctx,
+ struct geniv_req_ctx *rctx,
+ struct geniv_subreq *subreq, u8 *iv)
{
- memset(iv, 0, cc->iv_size);
-
+ memset(iv, 0, ctx->iv_size);
return 0;
}
-static void crypt_iv_lmk_dtr(struct crypt_config *cc)
+static void crypt_iv_lmk_dtr(struct geniv_ctx *ctx)
{
- struct iv_lmk_private *lmk = &cc->iv_gen_private.lmk;
+ struct geniv_lmk_private *lmk = &ctx->iv_gen_private.lmk;
if (lmk->hash_tfm && !IS_ERR(lmk->hash_tfm))
crypto_free_shash(lmk->hash_tfm);
@@ -483,49 +430,49 @@ static void crypt_iv_lmk_dtr(struct crypt_config *cc)
lmk->seed = NULL;
}
-static int crypt_iv_lmk_ctr(struct crypt_config *cc, struct dm_target *ti,
- const char *opts)
+static int crypt_iv_lmk_ctr(struct geniv_ctx *ctx)
{
- struct iv_lmk_private *lmk = &cc->iv_gen_private.lmk;
+ struct geniv_lmk_private *lmk = &ctx->iv_gen_private.lmk;
lmk->hash_tfm = crypto_alloc_shash("md5", 0, 0);
if (IS_ERR(lmk->hash_tfm)) {
- ti->error = "Error initializing LMK hash";
+ DMERR("Error initializing LMK hash; err=%ld\n",
+ PTR_ERR(lmk->hash_tfm));
return PTR_ERR(lmk->hash_tfm);
}
/* No seed in LMK version 2 */
- if (cc->key_parts == cc->tfms_count) {
+ if (ctx->key_parts == ctx->tfms_count) {
lmk->seed = NULL;
return 0;
}
lmk->seed = kzalloc(LMK_SEED_SIZE, GFP_KERNEL);
if (!lmk->seed) {
- crypt_iv_lmk_dtr(cc);
- ti->error = "Error kmallocing seed storage in LMK";
+ crypt_iv_lmk_dtr(ctx);
+ DMERR("Error kmallocing seed storage in LMK\n");
return -ENOMEM;
}
return 0;
}
-static int crypt_iv_lmk_init(struct crypt_config *cc)
+static int crypt_iv_lmk_init(struct geniv_ctx *ctx)
{
- struct iv_lmk_private *lmk = &cc->iv_gen_private.lmk;
- int subkey_size = cc->key_size / cc->key_parts;
+ struct geniv_lmk_private *lmk = &ctx->iv_gen_private.lmk;
+ int subkey_size = ctx->key_size / ctx->key_parts;
/* LMK seed is on the position of LMK_KEYS + 1 key */
if (lmk->seed)
- memcpy(lmk->seed, cc->key + (cc->tfms_count * subkey_size),
+ memcpy(lmk->seed, ctx->key + (ctx->tfms_count * subkey_size),
crypto_shash_digestsize(lmk->hash_tfm));
return 0;
}
-static int crypt_iv_lmk_wipe(struct crypt_config *cc)
+static int crypt_iv_lmk_wipe(struct geniv_ctx *ctx)
{
- struct iv_lmk_private *lmk = &cc->iv_gen_private.lmk;
+ struct geniv_lmk_private *lmk = &ctx->iv_gen_private.lmk;
if (lmk->seed)
memset(lmk->seed, 0, LMK_SEED_SIZE);
@@ -533,15 +480,14 @@ static int crypt_iv_lmk_wipe(struct crypt_config *cc)
return 0;
}
-static int crypt_iv_lmk_one(struct crypt_config *cc, u8 *iv,
- struct dm_crypt_request *dmreq,
- u8 *data)
+static int crypt_iv_lmk_one(struct geniv_ctx *ctx, u8 *iv,
+ struct geniv_req_ctx *rctx, u8 *data)
{
- struct iv_lmk_private *lmk = &cc->iv_gen_private.lmk;
- SHASH_DESC_ON_STACK(desc, lmk->hash_tfm);
+ struct geniv_lmk_private *lmk = &ctx->iv_gen_private.lmk;
struct md5_state md5state;
__le32 buf[4];
int i, r;
+ SHASH_DESC_ON_STACK(desc, lmk->hash_tfm);
desc->tfm = lmk->hash_tfm;
desc->flags = CRYPTO_TFM_REQ_MAY_SLEEP;
@@ -562,8 +508,9 @@ static int crypt_iv_lmk_one(struct crypt_config *cc, u8 *iv,
return r;
/* Sector is cropped to 56 bits here */
- buf[0] = cpu_to_le32(dmreq->iv_sector & 0xFFFFFFFF);
- buf[1] = cpu_to_le32((((u64)dmreq->iv_sector >> 32) & 0x00FFFFFF) | 0x80000000);
+ buf[0] = cpu_to_le32(rctx->iv_sector & 0xFFFFFFFF);
+ buf[1] = cpu_to_le32((((u64)rctx->iv_sector >> 32) & 0x00FFFFFF)
+ | 0x80000000);
buf[2] = cpu_to_le32(4024);
buf[3] = 0;
r = crypto_shash_update(desc, (u8 *)buf, sizeof(buf));
@@ -577,50 +524,52 @@ static int crypt_iv_lmk_one(struct crypt_config *cc, u8 *iv,
for (i = 0; i < MD5_HASH_WORDS; i++)
__cpu_to_le32s(&md5state.hash[i]);
- memcpy(iv, &md5state.hash, cc->iv_size);
+ memcpy(iv, &md5state.hash, ctx->iv_size);
return 0;
}
-static int crypt_iv_lmk_gen(struct crypt_config *cc, u8 *iv,
- struct dm_crypt_request *dmreq)
+static int crypt_iv_lmk_gen(struct geniv_ctx *ctx,
+ struct geniv_req_ctx *rctx,
+ struct geniv_subreq *subreq, u8 *iv)
{
u8 *src;
int r = 0;
- if (bio_data_dir(dmreq->ctx->bio_in) == WRITE) {
- src = kmap_atomic(sg_page(&dmreq->sg_in));
- r = crypt_iv_lmk_one(cc, iv, dmreq, src + dmreq->sg_in.offset);
+ if (rctx->is_write) {
+ src = kmap_atomic(sg_page(&subreq->src));
+ r = crypt_iv_lmk_one(ctx, iv, rctx, src + subreq->src.offset);
kunmap_atomic(src);
} else
- memset(iv, 0, cc->iv_size);
+ memset(iv, 0, ctx->iv_size);
return r;
}
-static int crypt_iv_lmk_post(struct crypt_config *cc, u8 *iv,
- struct dm_crypt_request *dmreq)
+static int crypt_iv_lmk_post(struct geniv_ctx *ctx,
+ struct geniv_req_ctx *rctx,
+ struct geniv_subreq *subreq, u8 *iv)
{
u8 *dst;
int r;
- if (bio_data_dir(dmreq->ctx->bio_in) == WRITE)
+ if (rctx->is_write)
return 0;
- dst = kmap_atomic(sg_page(&dmreq->sg_out));
- r = crypt_iv_lmk_one(cc, iv, dmreq, dst + dmreq->sg_out.offset);
+ dst = kmap_atomic(sg_page(&subreq->dst));
+ r = crypt_iv_lmk_one(ctx, iv, rctx, dst + subreq->dst.offset);
/* Tweak the first block of plaintext sector */
if (!r)
- crypto_xor(dst + dmreq->sg_out.offset, iv, cc->iv_size);
+ crypto_xor(dst + subreq->dst.offset, iv, ctx->iv_size);
kunmap_atomic(dst);
return r;
}
-static void crypt_iv_tcw_dtr(struct crypt_config *cc)
+static void crypt_iv_tcw_dtr(struct geniv_ctx *ctx)
{
- struct iv_tcw_private *tcw = &cc->iv_gen_private.tcw;
+ struct geniv_tcw_private *tcw = &ctx->iv_gen_private.tcw;
kzfree(tcw->iv_seed);
tcw->iv_seed = NULL;
@@ -632,64 +581,65 @@ static void crypt_iv_tcw_dtr(struct crypt_config *cc)
tcw->crc32_tfm = NULL;
}
-static int crypt_iv_tcw_ctr(struct crypt_config *cc, struct dm_target *ti,
- const char *opts)
+static int crypt_iv_tcw_ctr(struct geniv_ctx *ctx)
{
- struct iv_tcw_private *tcw = &cc->iv_gen_private.tcw;
+ struct geniv_tcw_private *tcw = &ctx->iv_gen_private.tcw;
- if (cc->key_size <= (cc->iv_size + TCW_WHITENING_SIZE)) {
- ti->error = "Wrong key size for TCW";
+ if (ctx->key_size <= (ctx->iv_size + TCW_WHITENING_SIZE)) {
+ DMERR("Wrong key size (%d) for TCW. Choose a value > %d bytes\n",
+ ctx->key_size,
+ ctx->iv_size + TCW_WHITENING_SIZE);
return -EINVAL;
}
tcw->crc32_tfm = crypto_alloc_shash("crc32", 0, 0);
if (IS_ERR(tcw->crc32_tfm)) {
- ti->error = "Error initializing CRC32 in TCW";
+ DMERR("Error initializing CRC32 in TCW; err=%ld\n",
+ PTR_ERR(tcw->crc32_tfm));
return PTR_ERR(tcw->crc32_tfm);
}
- tcw->iv_seed = kzalloc(cc->iv_size, GFP_KERNEL);
+ tcw->iv_seed = kzalloc(ctx->iv_size, GFP_KERNEL);
tcw->whitening = kzalloc(TCW_WHITENING_SIZE, GFP_KERNEL);
if (!tcw->iv_seed || !tcw->whitening) {
- crypt_iv_tcw_dtr(cc);
- ti->error = "Error allocating seed storage in TCW";
+ crypt_iv_tcw_dtr(ctx);
+ DMERR("Error allocating seed storage in TCW\n");
return -ENOMEM;
}
return 0;
}
-static int crypt_iv_tcw_init(struct crypt_config *cc)
+static int crypt_iv_tcw_init(struct geniv_ctx *ctx)
{
- struct iv_tcw_private *tcw = &cc->iv_gen_private.tcw;
- int key_offset = cc->key_size - cc->iv_size - TCW_WHITENING_SIZE;
+ struct geniv_tcw_private *tcw = &ctx->iv_gen_private.tcw;
+ int key_offset = ctx->key_size - ctx->iv_size - TCW_WHITENING_SIZE;
- memcpy(tcw->iv_seed, &cc->key[key_offset], cc->iv_size);
- memcpy(tcw->whitening, &cc->key[key_offset + cc->iv_size],
+ memcpy(tcw->iv_seed, &ctx->key[key_offset], ctx->iv_size);
+ memcpy(tcw->whitening, &ctx->key[key_offset + ctx->iv_size],
TCW_WHITENING_SIZE);
return 0;
}
-static int crypt_iv_tcw_wipe(struct crypt_config *cc)
+static int crypt_iv_tcw_wipe(struct geniv_ctx *ctx)
{
- struct iv_tcw_private *tcw = &cc->iv_gen_private.tcw;
+ struct geniv_tcw_private *tcw = &ctx->iv_gen_private.tcw;
- memset(tcw->iv_seed, 0, cc->iv_size);
+ memset(tcw->iv_seed, 0, ctx->iv_size);
memset(tcw->whitening, 0, TCW_WHITENING_SIZE);
return 0;
}
-static int crypt_iv_tcw_whitening(struct crypt_config *cc,
- struct dm_crypt_request *dmreq,
- u8 *data)
+static int crypt_iv_tcw_whitening(struct geniv_ctx *ctx,
+ struct geniv_req_ctx *rctx, u8 *data)
{
- struct iv_tcw_private *tcw = &cc->iv_gen_private.tcw;
- __le64 sector = cpu_to_le64(dmreq->iv_sector);
+ struct geniv_tcw_private *tcw = &ctx->iv_gen_private.tcw;
+ __le64 sector = cpu_to_le64(rctx->iv_sector);
u8 buf[TCW_WHITENING_SIZE];
- SHASH_DESC_ON_STACK(desc, tcw->crc32_tfm);
int i, r;
+ SHASH_DESC_ON_STACK(desc, tcw->crc32_tfm);
/* xor whitening with sector number */
memcpy(buf, tcw->whitening, TCW_WHITENING_SIZE);
@@ -714,97 +664,1053 @@ static int crypt_iv_tcw_whitening(struct crypt_config *cc,
crypto_xor(&buf[4], &buf[8], 4);
/* apply whitening (8 bytes) to whole sector */
- for (i = 0; i < ((1 << SECTOR_SHIFT) / 8); i++)
+ for (i = 0; i < (SECTOR_SIZE / 8); i++)
crypto_xor(data + i * 8, buf, 8);
out:
memzero_explicit(buf, sizeof(buf));
return r;
}
-static int crypt_iv_tcw_gen(struct crypt_config *cc, u8 *iv,
- struct dm_crypt_request *dmreq)
-{
- struct iv_tcw_private *tcw = &cc->iv_gen_private.tcw;
- __le64 sector = cpu_to_le64(dmreq->iv_sector);
- u8 *src;
- int r = 0;
+static int crypt_iv_tcw_gen(struct geniv_ctx *ctx,
+ struct geniv_req_ctx *rctx,
+ struct geniv_subreq *subreq, u8 *iv)
+{
+ struct geniv_tcw_private *tcw = &ctx->iv_gen_private.tcw;
+ __le64 sector = cpu_to_le64(rctx->iv_sector);
+ u8 *src;
+ int r = 0;
+
+ /* Remove whitening from ciphertext */
+ if (!rctx->is_write) {
+ src = kmap_atomic(sg_page(&subreq->src));
+ r = crypt_iv_tcw_whitening(ctx, rctx,
+ src + subreq->src.offset);
+ kunmap_atomic(src);
+ }
+
+ /* Calculate IV */
+ memcpy(iv, tcw->iv_seed, ctx->iv_size);
+ crypto_xor(iv, (u8 *)§or, 8);
+ if (ctx->iv_size > 8)
+ crypto_xor(&iv[8], (u8 *)§or, ctx->iv_size - 8);
+
+ return r;
+}
+
+static int crypt_iv_tcw_post(struct geniv_ctx *ctx,
+ struct geniv_req_ctx *rctx,
+ struct geniv_subreq *subreq, u8 *iv)
+{
+ u8 *dst;
+ int r;
+
+ if (!rctx->is_write)
+ return 0;
+
+ /* Apply whitening on ciphertext */
+ dst = kmap_atomic(sg_page(&subreq->dst));
+ r = crypt_iv_tcw_whitening(ctx, rctx, dst + subreq->dst.offset);
+ kunmap_atomic(dst);
+
+ return r;
+}
+
+static const struct crypt_iv_operations crypt_iv_plain_ops = {
+ .generator = crypt_iv_plain_gen
+};
+
+static const struct crypt_iv_operations crypt_iv_plain64_ops = {
+ .generator = crypt_iv_plain64_gen
+};
+
+static const struct crypt_iv_operations crypt_iv_essiv_ops = {
+ .ctr = crypt_iv_essiv_ctr,
+ .dtr = crypt_iv_essiv_dtr,
+ .init = crypt_iv_essiv_init,
+ .wipe = crypt_iv_essiv_wipe,
+ .generator = crypt_iv_essiv_gen
+};
+
+static const struct crypt_iv_operations crypt_iv_benbi_ops = {
+ .ctr = crypt_iv_benbi_ctr,
+ .generator = crypt_iv_benbi_gen
+};
+
+static const struct crypt_iv_operations crypt_iv_null_ops = {
+ .generator = crypt_iv_null_gen
+};
+
+static const struct crypt_iv_operations crypt_iv_lmk_ops = {
+ .ctr = crypt_iv_lmk_ctr,
+ .dtr = crypt_iv_lmk_dtr,
+ .init = crypt_iv_lmk_init,
+ .wipe = crypt_iv_lmk_wipe,
+ .generator = crypt_iv_lmk_gen,
+ .post = crypt_iv_lmk_post
+};
+
+static const struct crypt_iv_operations crypt_iv_tcw_ops = {
+ .ctr = crypt_iv_tcw_ctr,
+ .dtr = crypt_iv_tcw_dtr,
+ .init = crypt_iv_tcw_init,
+ .wipe = crypt_iv_tcw_wipe,
+ .generator = crypt_iv_tcw_gen,
+ .post = crypt_iv_tcw_post
+};
+
+static int geniv_setkey_set(struct geniv_ctx *ctx)
+{
+ int ret = 0;
+
+ if (ctx->iv_gen_ops && ctx->iv_gen_ops->init)
+ ret = ctx->iv_gen_ops->init(ctx);
+ return ret;
+}
+
+static int geniv_setkey_wipe(struct geniv_ctx *ctx)
+{
+ int ret = 0;
+
+ if (ctx->iv_gen_ops && ctx->iv_gen_ops->wipe) {
+ ret = ctx->iv_gen_ops->wipe(ctx);
+ if (ret)
+ return ret;
+ }
+ return ret;
+}
+
+static int geniv_init_iv(struct geniv_ctx *ctx)
+{
+ int ret = -EINVAL;
+
+ DMDEBUG("IV Generation algorithm : %s\n", ctx->ivmode);
+
+ if (ctx->ivmode == NULL)
+ ctx->iv_gen_ops = NULL;
+ else if (strcmp(ctx->ivmode, "plain") == 0)
+ ctx->iv_gen_ops = &crypt_iv_plain_ops;
+ else if (strcmp(ctx->ivmode, "plain64") == 0)
+ ctx->iv_gen_ops = &crypt_iv_plain64_ops;
+ else if (strcmp(ctx->ivmode, "essiv") == 0)
+ ctx->iv_gen_ops = &crypt_iv_essiv_ops;
+ else if (strcmp(ctx->ivmode, "benbi") == 0)
+ ctx->iv_gen_ops = &crypt_iv_benbi_ops;
+ else if (strcmp(ctx->ivmode, "null") == 0)
+ ctx->iv_gen_ops = &crypt_iv_null_ops;
+ else if (strcmp(ctx->ivmode, "lmk") == 0)
+ ctx->iv_gen_ops = &crypt_iv_lmk_ops;
+ else if (strcmp(ctx->ivmode, "tcw") == 0) {
+ ctx->iv_gen_ops = &crypt_iv_tcw_ops;
+ ctx->key_parts += 2; /* IV + whitening */
+ ctx->key_extra_size = ctx->iv_size + TCW_WHITENING_SIZE;
+ } else {
+ ret = -EINVAL;
+ DMERR("Invalid IV mode %s\n", ctx->ivmode);
+ goto end;
+ }
+
+ /* Allocate IV */
+ if (ctx->iv_gen_ops && ctx->iv_gen_ops->ctr) {
+ ret = ctx->iv_gen_ops->ctr(ctx);
+ if (ret < 0) {
+ DMERR("Error creating IV for %s\n", ctx->ivmode);
+ goto end;
+ }
+ }
+
+ /* Initialize IV (set keys for ESSIV etc) */
+ if (ctx->iv_gen_ops && ctx->iv_gen_ops->init) {
+ ret = ctx->iv_gen_ops->init(ctx);
+ if (ret < 0)
+ DMERR("Error creating IV for %s\n", ctx->ivmode);
+ }
+ ret = 0;
+end:
+ return ret;
+}
+
+static void geniv_free_tfms(struct geniv_ctx *ctx)
+{
+ unsigned int i;
+
+ if (!ctx->tfms)
+ return;
+
+ for (i = 0; i < ctx->tfms_count; i++)
+ if (ctx->tfms[i] && !IS_ERR(ctx->tfms[i])) {
+ crypto_free_skcipher(ctx->tfms[i]);
+ ctx->tfms[i] = NULL;
+ }
+
+ kfree(ctx->tfms);
+ ctx->tfms = NULL;
+}
+
+/* Allocate memory for the underlying cipher algorithm. Ex: cbc(aes)
+ */
+
+static int geniv_alloc_tfms(struct crypto_skcipher *parent,
+ struct geniv_ctx *ctx)
+{
+ unsigned int i, reqsize, align;
+ int err = 0;
+
+ ctx->tfms = kcalloc(ctx->tfms_count, sizeof(struct crypto_skcipher *),
+ GFP_KERNEL);
+ if (!ctx->tfms) {
+ err = -ENOMEM;
+ goto end;
+ }
+
+ /* First instance is already allocated in geniv_init_tfm */
+ ctx->tfms[0] = ctx->child;
+ for (i = 1; i < ctx->tfms_count; i++) {
+ ctx->tfms[i] = crypto_alloc_skcipher(ctx->ciphermode, 0, 0);
+ if (IS_ERR(ctx->tfms[i])) {
+ err = PTR_ERR(ctx->tfms[i]);
+ geniv_free_tfms(ctx);
+ goto end;
+ }
+
+ /* Setup the current cipher's request structure */
+ align = crypto_skcipher_alignmask(parent);
+ align &= ~(crypto_tfm_ctx_alignment() - 1);
+ reqsize = align + sizeof(struct geniv_req_ctx) +
+ crypto_skcipher_reqsize(ctx->tfms[i]);
+ crypto_skcipher_set_reqsize(parent, reqsize);
+ }
+
+end:
+ return err;
+}
+
+/* Initialize the cipher's context with the key, ivmode and other parameters.
+ * Also allocate IV generation template ciphers and initialize them.
+ */
+
+static int geniv_setkey_init(struct crypto_skcipher *parent,
+ struct geniv_key_info *info)
+{
+ struct geniv_ctx *ctx = crypto_skcipher_ctx(parent);
+ int ret = -ENOMEM;
+
+ ctx->tfms_count = info->tfms_count;
+ ctx->key = info->key;
+ ctx->key_size = info->key_size;
+ ctx->key_parts = info->key_parts;
+ ctx->ivopts = info->ivopts;
+
+ ret = geniv_alloc_tfms(parent, ctx);
+ if (ret)
+ goto end;
+
+ ret = geniv_init_iv(ctx);
+
+end:
+ return ret;
+}
+
+static int geniv_setkey_tfms(struct crypto_skcipher *parent,
+ struct geniv_ctx *ctx,
+ struct geniv_key_info *info)
+{
+ unsigned int subkey_size;
+ int ret = 0, i;
+
+ /* Ignore extra keys (which are used for IV etc) */
+ subkey_size = (ctx->key_size - ctx->key_extra_size)
+ >> ilog2(ctx->tfms_count);
+
+ for (i = 0; i < ctx->tfms_count; i++) {
+ struct crypto_skcipher *child = ctx->tfms[i];
+ char *subkey = ctx->key + (subkey_size) * i;
+
+ crypto_skcipher_clear_flags(child, CRYPTO_TFM_REQ_MASK);
+ crypto_skcipher_set_flags(child,
+ crypto_skcipher_get_flags(parent) &
+ CRYPTO_TFM_REQ_MASK);
+ ret = crypto_skcipher_setkey(child, subkey, subkey_size);
+ if (ret) {
+ DMERR("Error setting key for tfms[%d]\n", i);
+ break;
+ }
+ crypto_skcipher_set_flags(parent,
+ crypto_skcipher_get_flags(child) &
+ CRYPTO_TFM_RES_MASK);
+ }
+
+ return ret;
+}
+
+static int geniv_setkey(struct crypto_skcipher *parent,
+ const u8 *key, unsigned int keylen)
+{
+ int err = 0;
+ struct geniv_ctx *ctx = crypto_skcipher_ctx(parent);
+ struct geniv_key_info *info = (struct geniv_key_info *) key;
+
+ DMDEBUG("SETKEY Operation : %d\n", info->keyop);
+
+ switch (info->keyop) {
+ case SETKEY_OP_INIT:
+ err = geniv_setkey_init(parent, info);
+ break;
+ case SETKEY_OP_SET:
+ err = geniv_setkey_set(ctx);
+ break;
+ case SETKEY_OP_WIPE:
+ err = geniv_setkey_wipe(ctx);
+ break;
+ }
+
+ if (err)
+ goto end;
+
+ err = geniv_setkey_tfms(parent, ctx, info);
+
+end:
+ return err;
+}
+
+static void geniv_async_done(struct crypto_async_request *async_req, int error);
+
+static int geniv_alloc_subreq(struct skcipher_request *req,
+ struct geniv_ctx *ctx,
+ struct geniv_req_ctx *rctx)
+{
+ int key_index, r = 0;
+ struct skcipher_request *sreq;
+
+ if (!rctx->subreq) {
+ rctx->subreq = mempool_alloc(ctx->subreq_pool, GFP_NOIO);
+ if (!rctx->subreq)
+ r = -ENOMEM;
+ }
+
+ sreq = &rctx->subreq->req;
+ rctx->subreq->rctx = rctx;
+
+ key_index = rctx->iv_sector & (ctx->tfms_count - 1);
+
+ skcipher_request_set_tfm(sreq, ctx->tfms[key_index]);
+ skcipher_request_set_callback(sreq, req->base.flags,
+ geniv_async_done, rctx->subreq);
+ return r;
+}
+
+/* Asynchronous IO completion callback for each sector in a segment. When all
+ * pending i/o are completed the parent cipher's async function is called.
+ */
+
+static void geniv_async_done(struct crypto_async_request *async_req, int error)
+{
+ struct geniv_subreq *subreq =
+ (struct geniv_subreq *) async_req->data;
+ struct geniv_req_ctx *rctx = subreq->rctx;
+ struct skcipher_request *req = rctx->req;
+ struct crypto_skcipher *tfm = crypto_skcipher_reqtfm(req);
+ struct geniv_ctx *ctx = crypto_skcipher_ctx(tfm);
+ u8 *iv;
+
+ /*
+ * A request from crypto driver backlog is going to be processed now,
+ * finish the completion and continue in crypt_convert().
+ * (Callback will be called for the second time for this request.)
+ */
+
+ if (error == -EINPROGRESS) {
+ complete(&rctx->restart);
+ return;
+ }
+
+ iv = iv_of_subreq(ctx, subreq);
+ if (!error && ctx->iv_gen_ops && ctx->iv_gen_ops->post)
+ error = ctx->iv_gen_ops->post(ctx, rctx, subreq, iv);
+
+ mempool_free(subreq, ctx->subreq_pool);
+
+ /* req_pending needs to be checked before req->base.complete is called
+ * as we need 'req_pending' to be equal to 1 to ensure all subrequests
+ * are processed.
+ */
+ if (!atomic_dec_and_test(&rctx->req_pending)) {
+ /* Call the parent cipher's completion function */
+ skcipher_request_complete(req, error);
+ }
+}
+
+static unsigned int geniv_get_sectors(struct scatterlist *sg1,
+ struct scatterlist *sg2,
+ unsigned int segments)
+{
+ unsigned int i, n1, n2, nents;
+
+ n1 = n2 = 0;
+ for (i = 0; i < segments ; i++) {
+ n1 += sg1[i].length >> SECTOR_SHIFT;
+ n1 += (sg1[i].length & ~SECTOR_MASK) ? 1 : 0;
+ }
+
+ for (i = 0; i < segments ; i++) {
+ n2 += sg2[i].length >> SECTOR_SHIFT;
+ n2 += (sg2[i].length & ~SECTOR_MASK) ? 1 : 0;
+ }
+
+ nents = n1 > n2 ? n1 : n2;
+ return nents;
+}
+
+/* Iterate scatterlist of segments to retrieve the 512-byte sectors so that
+ * unique IVs could be generated for each 512-byte sector. This split may not
+ * be necessary e.g. when these ciphers are modelled in hardware, where it can
+ * make use of the hardware's IV generation capabilities.
+ */
+
+static int geniv_iter_block(struct skcipher_request *req,
+ struct geniv_subreq *subreq,
+ struct geniv_req_ctx *rctx,
+ unsigned int *seg_no,
+ unsigned int *done)
+
+{
+ unsigned int srcoff, dstoff, len, rem;
+ struct scatterlist *src1, *dst1, *src2, *dst2;
+
+ if (unlikely(*seg_no >= rctx->nents))
+ return 0; /* done */
+
+ src1 = &req->src[*seg_no];
+ dst1 = &req->dst[*seg_no];
+ src2 = &subreq->src;
+ dst2 = &subreq->dst;
+
+ if (*done >= src1->length) {
+ (*seg_no)++;
+
+ if (*seg_no >= rctx->nents)
+ return 0; /* done */
+
+ src1 = &req->src[*seg_no];
+ dst1 = &req->dst[*seg_no];
+ *done = 0;
+ }
+
+ srcoff = src1->offset + *done;
+ dstoff = dst1->offset + *done;
+ rem = src1->length - *done;
+
+ len = rem > SECTOR_SIZE ? SECTOR_SIZE : rem;
+
+ DMDEBUG("segment:(%d/%u), srcoff:%d, dstoff:%d, done:%d, rem:%d\n",
+ *seg_no + 1, rctx->nents, srcoff, dstoff, *done, rem);
+
+ sg_init_table(src2, 1);
+ sg_set_page(src2, sg_page(src1), len, srcoff);
+ sg_init_table(dst2, 1);
+ sg_set_page(dst2, sg_page(dst1), len, dstoff);
+
+ *done += len;
+
+ return len; /* bytes returned */
+}
+
+/* Common encryt/decrypt function for geniv template cipher. Before the crypto
+ * operation, it splits the memory segments (in the scatterlist) into 512 byte
+ * sectors. The initialization vector(IV) used is based on a unique sector
+ * number which is generated here.
+ */
+static int geniv_crypt(struct skcipher_request *req, int encrypt)
+{
+ struct crypto_skcipher *tfm = crypto_skcipher_reqtfm(req);
+ struct geniv_ctx *ctx = crypto_skcipher_ctx(tfm);
+ struct geniv_req_ctx *rctx = geniv_req_ctx(req);
+ struct geniv_req_info *rinfo = (struct geniv_req_info *) req->iv;
+ int i, bytes, cryptlen, ret = 0;
+ unsigned int sectors, segno = 0, done = 0;
+ char *str __maybe_unused = encrypt ? "encrypt" : "decrypt";
+ u8 *iv;
+
+ /* Instance of 'struct geniv_req_info' is stored in IV ptr */
+ rctx->is_write = encrypt;
+ rctx->iv_sector = rinfo->iv_sector;
+ rctx->nents = rinfo->nents;
+ rctx->req = req;
+ rctx->subreq = NULL;
+ cryptlen = req->cryptlen;
+
+ DMDEBUG("geniv:%s: starting sector=%d, #segments=%u\n", str,
+ (unsigned int) rctx->iv_sector, rctx->nents);
+
+ sectors = geniv_get_sectors(req->src, req->dst, rctx->nents);
+
+ init_completion(&rctx->restart);
+ atomic_set(&rctx->req_pending, 1);
+
+ for (i = 0; i < sectors; i++) {
+ struct geniv_subreq *subreq;
+
+ ret = geniv_alloc_subreq(req, ctx, rctx);
+ if (ret)
+ goto end;
+
+ subreq = rctx->subreq;
+ subreq->rctx = rctx;
+ iv = iv_of_subreq(ctx, subreq);
+
+ atomic_inc(&rctx->req_pending);
+ bytes = geniv_iter_block(req, subreq, rctx, &segno, &done);
+
+ if (bytes == 0)
+ break;
+
+ cryptlen -= bytes;
+
+ if (ctx->iv_gen_ops)
+ ret = ctx->iv_gen_ops->generator(ctx, rctx, subreq, iv);
+
+ if (ret < 0) {
+ DMERR("Error in generating IV ret: %d\n", ret);
+ goto end;
+ }
+
+ skcipher_request_set_crypt(&subreq->req, &subreq->src,
+ &subreq->dst, bytes, iv);
+
+ if (encrypt)
+ ret = crypto_skcipher_encrypt(&subreq->req);
+
+ else
+ ret = crypto_skcipher_decrypt(&subreq->req);
+
+ if (!ret && ctx->iv_gen_ops && ctx->iv_gen_ops->post)
+ ret = ctx->iv_gen_ops->post(ctx, rctx, subreq, iv);
+
+ switch (ret) {
+ /*
+ * The request was queued by a crypto driver
+ * but the driver request queue is full, let's wait.
+ */
+ case -EBUSY:
+ wait_for_completion(&rctx->restart);
+ reinit_completion(&rctx->restart);
+ /* fall through */
+ /*
+ * The request is queued and processed asynchronously,
+ * completion function geniv_async_done() is called.
+ */
+ case -EINPROGRESS:
+ /* Marking this NULL lets the creation of a new sub-
+ * request when 'geniv_alloc_subreq' is called.
+ */
+ rctx->subreq = NULL;
+ rctx->iv_sector++;
+ cond_resched();
+ break;
+ /*
+ * The request was already processed (synchronously).
+ */
+ case 0:
+ atomic_dec(&rctx->req_pending);
+ rctx->iv_sector++;
+ cond_resched();
+ continue;
+
+ /* There was an error while processing the request. */
+ default:
+ atomic_dec(&rctx->req_pending);
+ return ret;
+ }
+
+ if (ret)
+ break;
+ }
+
+ if (rctx->subreq && atomic_read(&rctx->req_pending) == 1) {
+ DMDEBUG("geniv:%s: Freeing sub request\n", str);
+ mempool_free(rctx->subreq, ctx->subreq_pool);
+ }
+
+end:
+ return ret;
+}
+
+static int geniv_encrypt(struct skcipher_request *req)
+{
+ return geniv_crypt(req, 1);
+}
+
+static int geniv_decrypt(struct skcipher_request *req)
+{
+ return geniv_crypt(req, 0);
+}
+
+static int geniv_init_tfm(struct crypto_skcipher *tfm)
+{
+ struct geniv_ctx *ctx = crypto_skcipher_ctx(tfm);
+ unsigned int reqsize, align;
+ size_t iv_size_padding;
+ char *algname, *chainmode;
+ int psize, ret = 0;
+
+ algname = (char *) crypto_tfm_alg_name(crypto_skcipher_tfm(tfm));
+ ctx->ciphermode = kmalloc(CRYPTO_MAX_ALG_NAME, GFP_KERNEL);
+ if (!ctx->ciphermode) {
+ ret = -ENOMEM;
+ goto out;
+ }
+
+ ctx->algname = kmalloc(CRYPTO_MAX_ALG_NAME, GFP_KERNEL);
+ if (!ctx->algname) {
+ ret = -ENOMEM;
+ goto free_ciphermode;
+ }
+
+ strlcpy(ctx->algname, algname, CRYPTO_MAX_ALG_NAME);
+ algname = ctx->algname;
+
+ /* Parse the algorithm name 'ivmode(chainmode(cipher))' */
+ ctx->ivmode = strsep(&algname, "(");
+ chainmode = strsep(&algname, "(");
+ ctx->cipher = strsep(&algname, ")");
+
+ snprintf(ctx->ciphermode, CRYPTO_MAX_ALG_NAME, "%s(%s)",
+ chainmode, ctx->cipher);
+
+ DMDEBUG("ciphermode=%s, ivmode=%s\n", ctx->ciphermode, ctx->ivmode);
+
+ /*
+ * Usually the underlying cipher instances are spawned here, but since
+ * the value of tfms_count (which is equal to the key_count) is not
+ * known yet, create only one instance and delay the creation of the
+ * rest of the instances of the underlying cipher 'cbc(aes)' until
+ * the setkey operation is invoked.
+ * The first instance created i.e. ctx->child will later be assigned as
+ * the 1st element in the array ctx->tfms. Creation of atleast one
+ * instance of the cipher is necessary to be created here to uncover
+ * any errors earlier than during the setkey operation later where the
+ * remaining instances are created.
+ */
+ ctx->child = crypto_alloc_skcipher(ctx->ciphermode, 0, 0);
+ if (IS_ERR(ctx->child)) {
+ ret = PTR_ERR(ctx->child);
+ DMERR("Failed to create skcipher %s. err %d\n",
+ ctx->ciphermode, ret);
+ goto free_algname;
+ }
+
+ /* Setup the current cipher's request structure */
+ align = crypto_skcipher_alignmask(tfm);
+ align &= ~(crypto_tfm_ctx_alignment() - 1);
+ reqsize = align + sizeof(struct geniv_req_ctx)
+ + crypto_skcipher_reqsize(ctx->child);
+ crypto_skcipher_set_reqsize(tfm, reqsize);
+
+ ctx->iv_start = sizeof(struct geniv_subreq);
+ ctx->iv_start += crypto_skcipher_reqsize(tfm);
+
+ ctx->iv_size = crypto_skcipher_ivsize(tfm);
+ /* at least a 64 bit sector number should fit in our buffer */
+ if (ctx->iv_size)
+ ctx->iv_size = max(ctx->iv_size,
+ (unsigned int)(sizeof(u64) / sizeof(u8)));
+
+ if (crypto_skcipher_alignmask(tfm) < CRYPTO_MINALIGN) {
+ /* Allocate the padding exactly */
+ iv_size_padding = -ctx->iv_start
+ & crypto_skcipher_alignmask(tfm);
+ } else {
+ /*
+ * If the cipher requires greater alignment than kmalloc
+ * alignment, we don't know the exact position of the
+ * initialization vector. We must assume worst case.
+ */
+ iv_size_padding = crypto_skcipher_alignmask(tfm);
+ }
+
+ /* create memory pool for sub-request structure */
+ psize = ctx->iv_start + iv_size_padding + ctx->iv_size;
+
+ ctx->subreq_pool = mempool_create_kmalloc_pool(MIN_IOS, psize);
+ if (!ctx->subreq_pool) {
+ ret = -ENOMEM;
+ DMERR("Could not allocate crypt sub-request mempool\n");
+ goto free_skcipher;
+ }
+out:
+ return ret;
+
+free_skcipher:
+ crypto_free_skcipher(ctx->child);
+free_algname:
+ kfree(ctx->algname);
+free_ciphermode:
+ kfree(ctx->ciphermode);
+ goto out;
+}
+
+static void geniv_exit_tfm(struct crypto_skcipher *tfm)
+{
+ struct geniv_ctx *ctx = crypto_skcipher_ctx(tfm);
+
+ if (ctx->iv_gen_ops && ctx->iv_gen_ops->dtr)
+ ctx->iv_gen_ops->dtr(ctx);
+
+ mempool_destroy(ctx->subreq_pool);
+ geniv_free_tfms(ctx);
+ kfree(ctx->ciphermode);
+ kfree(ctx->algname);
+}
+
+static void geniv_free(struct skcipher_instance *inst)
+{
+ struct crypto_skcipher_spawn *spawn = skcipher_instance_ctx(inst);
+
+ crypto_drop_skcipher(spawn);
+ kfree(inst);
+}
+
+static int geniv_create(struct crypto_template *tmpl,
+ struct rtattr **tb, char *algname)
+{
+ struct crypto_attr_type *algt;
+ struct skcipher_instance *inst;
+ struct skcipher_alg *alg;
+ struct crypto_skcipher_spawn *spawn;
+ const char *cipher_name;
+ int err;
+
+ algt = crypto_get_attr_type(tb);
+
+ if (IS_ERR(algt))
+ return PTR_ERR(algt);
+
+ if ((algt->type ^ CRYPTO_ALG_TYPE_SKCIPHER) & algt->mask)
+ return -EINVAL;
+
+ cipher_name = crypto_attr_alg_name(tb[1]);
+
+ if (IS_ERR(cipher_name))
+ return PTR_ERR(cipher_name);
+
+ inst = kzalloc(sizeof(*inst) + sizeof(*spawn), GFP_KERNEL);
+ if (!inst)
+ return -ENOMEM;
+
+ spawn = skcipher_instance_ctx(inst);
+
+ crypto_set_skcipher_spawn(spawn, skcipher_crypto_instance(inst));
+ err = crypto_grab_skcipher(spawn, cipher_name, 0,
+ crypto_requires_sync(algt->type,
+ algt->mask));
+
+ if (err)
+ goto err_free_inst;
+
+ alg = crypto_spawn_skcipher_alg(spawn);
+
+ err = -EINVAL;
+
+ /* Only support blocks of size which is of a power of 2 */
+ if (!is_power_of_2(alg->base.cra_blocksize))
+ goto err_drop_spawn;
+
+ /* algname: essiv, base.cra_name: cbc(aes) */
+ err = -ENAMETOOLONG;
+ if (snprintf(inst->alg.base.cra_name, CRYPTO_MAX_ALG_NAME, "%s(%s)",
+ algname, alg->base.cra_name) >= CRYPTO_MAX_ALG_NAME)
+ goto err_drop_spawn;
+ if (snprintf(inst->alg.base.cra_driver_name, CRYPTO_MAX_ALG_NAME,
+ "%s(%s)", algname, alg->base.cra_driver_name) >=
+ CRYPTO_MAX_ALG_NAME)
+ goto err_drop_spawn;
+
+ inst->alg.base.cra_flags = CRYPTO_ALG_TYPE_BLKCIPHER;
+ inst->alg.base.cra_priority = alg->base.cra_priority;
+ inst->alg.base.cra_blocksize = alg->base.cra_blocksize;
+ inst->alg.base.cra_alignmask = alg->base.cra_alignmask;
+ inst->alg.base.cra_flags = alg->base.cra_flags & CRYPTO_ALG_ASYNC;
+ inst->alg.ivsize = alg->base.cra_blocksize;
+ inst->alg.chunksize = crypto_skcipher_alg_chunksize(alg);
+ inst->alg.min_keysize = crypto_skcipher_alg_min_keysize(alg);
+ inst->alg.max_keysize = crypto_skcipher_alg_max_keysize(alg);
+
+ inst->alg.setkey = geniv_setkey;
+ inst->alg.encrypt = geniv_encrypt;
+ inst->alg.decrypt = geniv_decrypt;
+
+ inst->alg.base.cra_ctxsize = sizeof(struct geniv_ctx);
+
+ inst->alg.init = geniv_init_tfm;
+ inst->alg.exit = geniv_exit_tfm;
+
+ inst->free = geniv_free;
+
+ err = skcipher_register_instance(tmpl, inst);
+ if (err)
+ goto err_drop_spawn;
+
+out:
+ return err;
+
+err_drop_spawn:
+ crypto_drop_skcipher(spawn);
+err_free_inst:
+ kfree(inst);
+ goto out;
+}
+
+static int crypto_plain_create(struct crypto_template *tmpl,
+ struct rtattr **tb)
+{
+ return geniv_create(tmpl, tb, "plain");
+}
+
+static int crypto_plain64_create(struct crypto_template *tmpl,
+ struct rtattr **tb)
+{
+ return geniv_create(tmpl, tb, "plain64");
+}
+
+static int crypto_essiv_create(struct crypto_template *tmpl,
+ struct rtattr **tb)
+{
+ return geniv_create(tmpl, tb, "essiv");
+}
+
+static int crypto_benbi_create(struct crypto_template *tmpl,
+ struct rtattr **tb)
+{
+ return geniv_create(tmpl, tb, "benbi");
+}
+
+static int crypto_null_create(struct crypto_template *tmpl,
+ struct rtattr **tb)
+{
+ return geniv_create(tmpl, tb, "null");
+}
+
+static int crypto_lmk_create(struct crypto_template *tmpl,
+ struct rtattr **tb)
+{
+ return geniv_create(tmpl, tb, "lmk");
+}
+
+static int crypto_tcw_create(struct crypto_template *tmpl,
+ struct rtattr **tb)
+{
+ return geniv_create(tmpl, tb, "tcw");
+}
+
+static struct crypto_template crypto_plain_tmpl = {
+ .name = "plain",
+ .create = crypto_plain_create,
+ .module = THIS_MODULE,
+};
+
+static struct crypto_template crypto_plain64_tmpl = {
+ .name = "plain64",
+ .create = crypto_plain64_create,
+ .module = THIS_MODULE,
+};
+
+static struct crypto_template crypto_essiv_tmpl = {
+ .name = "essiv",
+ .create = crypto_essiv_create,
+ .module = THIS_MODULE,
+};
+
+static struct crypto_template crypto_benbi_tmpl = {
+ .name = "benbi",
+ .create = crypto_benbi_create,
+ .module = THIS_MODULE,
+};
+
+static struct crypto_template crypto_null_tmpl = {
+ .name = "null",
+ .create = crypto_null_create,
+ .module = THIS_MODULE,
+};
+
+static struct crypto_template crypto_lmk_tmpl = {
+ .name = "lmk",
+ .create = crypto_lmk_create,
+ .module = THIS_MODULE,
+};
+
+static struct crypto_template crypto_tcw_tmpl = {
+ .name = "tcw",
+ .create = crypto_tcw_create,
+ .module = THIS_MODULE,
+};
+
+static int __init geniv_register_algs(void)
+{
+ int err;
+
+ err = crypto_register_template(&crypto_plain_tmpl);
+ if (err)
+ goto out;
+
+ err = crypto_register_template(&crypto_plain64_tmpl);
+ if (err)
+ goto out_undo_plain;
+
+ err = crypto_register_template(&crypto_essiv_tmpl);
+ if (err)
+ goto out_undo_plain64;
+
+ err = crypto_register_template(&crypto_benbi_tmpl);
+ if (err)
+ goto out_undo_essiv;
+
+ err = crypto_register_template(&crypto_null_tmpl);
+ if (err)
+ goto out_undo_benbi;
+
+ err = crypto_register_template(&crypto_lmk_tmpl);
+ if (err)
+ goto out_undo_null;
+
+ err = crypto_register_template(&crypto_tcw_tmpl);
+ if (!err)
+ goto out;
+
+ crypto_unregister_template(&crypto_lmk_tmpl);
+out_undo_null:
+ crypto_unregister_template(&crypto_null_tmpl);
+out_undo_benbi:
+ crypto_unregister_template(&crypto_benbi_tmpl);
+out_undo_essiv:
+ crypto_unregister_template(&crypto_essiv_tmpl);
+out_undo_plain64:
+ crypto_unregister_template(&crypto_plain64_tmpl);
+out_undo_plain:
+ crypto_unregister_template(&crypto_plain_tmpl);
+out:
+ return err;
+}
+
+static void __exit geniv_deregister_algs(void)
+{
+ crypto_unregister_template(&crypto_plain_tmpl);
+ crypto_unregister_template(&crypto_plain64_tmpl);
+ crypto_unregister_template(&crypto_essiv_tmpl);
+ crypto_unregister_template(&crypto_benbi_tmpl);
+ crypto_unregister_template(&crypto_null_tmpl);
+ crypto_unregister_template(&crypto_lmk_tmpl);
+ crypto_unregister_template(&crypto_tcw_tmpl);
+}
+
+/* End of geniv template cipher algorithms */
+
+/*
+ * context holding the current state of a multi-part conversion
+ */
+struct convert_context {
+ struct completion restart;
+ struct bio *bio_in;
+ struct bio *bio_out;
+ struct bvec_iter iter_in;
+ struct bvec_iter iter_out;
+ sector_t cc_sector;
+ atomic_t cc_pending;
+ struct skcipher_request *req;
+};
+
+/*
+ * per bio private data
+ */
+struct dm_crypt_io {
+ struct crypt_config *cc;
+ struct bio *base_bio;
+ struct work_struct work;
+
+ struct convert_context ctx;
+
+ atomic_t io_pending;
+ int error;
+ sector_t sector;
+
+ struct rb_node rb_node;
+} CRYPTO_MINALIGN_ATTR;
- /* Remove whitening from ciphertext */
- if (bio_data_dir(dmreq->ctx->bio_in) != WRITE) {
- src = kmap_atomic(sg_page(&dmreq->sg_in));
- r = crypt_iv_tcw_whitening(cc, dmreq, src + dmreq->sg_in.offset);
- kunmap_atomic(src);
- }
+struct dm_crypt_request {
+ struct convert_context *ctx;
+ struct scatterlist *sg_in;
+ struct scatterlist *sg_out;
+ sector_t iv_sector;
+};
- /* Calculate IV */
- memcpy(iv, tcw->iv_seed, cc->iv_size);
- crypto_xor(iv, (u8 *)§or, 8);
- if (cc->iv_size > 8)
- crypto_xor(&iv[8], (u8 *)§or, cc->iv_size - 8);
+struct crypt_config;
- return r;
-}
+/*
+ * Crypt: maps a linear range of a block device
+ * and encrypts / decrypts at the same time.
+ */
+enum flags { DM_CRYPT_SUSPENDED, DM_CRYPT_KEY_VALID,
+ DM_CRYPT_SAME_CPU, DM_CRYPT_NO_OFFLOAD };
-static int crypt_iv_tcw_post(struct crypt_config *cc, u8 *iv,
- struct dm_crypt_request *dmreq)
-{
- u8 *dst;
- int r;
+/*
+ * The fields in here must be read only after initialization.
+ */
+struct crypt_config {
+ struct dm_dev *dev;
+ sector_t start;
- if (bio_data_dir(dmreq->ctx->bio_in) != WRITE)
- return 0;
+ /*
+ * pool for per bio private data, crypto requests and
+ * encryption requeusts/buffer pages
+ */
+ mempool_t *req_pool;
+ mempool_t *page_pool;
+ struct bio_set *bs;
+ struct mutex bio_alloc_lock;
- /* Apply whitening on ciphertext */
- dst = kmap_atomic(sg_page(&dmreq->sg_out));
- r = crypt_iv_tcw_whitening(cc, dmreq, dst + dmreq->sg_out.offset);
- kunmap_atomic(dst);
+ struct workqueue_struct *io_queue;
+ struct workqueue_struct *crypt_queue;
- return r;
-}
+ struct task_struct *write_thread;
+ wait_queue_head_t write_thread_wait;
+ struct rb_root write_tree;
-static const struct crypt_iv_operations crypt_iv_plain_ops = {
- .generator = crypt_iv_plain_gen
-};
+ char *cipher;
+ char *cipher_string;
+ char *key_string;
-static const struct crypt_iv_operations crypt_iv_plain64_ops = {
- .generator = crypt_iv_plain64_gen
-};
+ sector_t iv_offset;
+ unsigned int iv_size;
-static const struct crypt_iv_operations crypt_iv_essiv_ops = {
- .ctr = crypt_iv_essiv_ctr,
- .dtr = crypt_iv_essiv_dtr,
- .init = crypt_iv_essiv_init,
- .wipe = crypt_iv_essiv_wipe,
- .generator = crypt_iv_essiv_gen
-};
+ /* ESSIV: struct crypto_cipher *essiv_tfm */
+ void *iv_private;
+ struct crypto_skcipher *tfm;
+ unsigned int tfms_count;
-static const struct crypt_iv_operations crypt_iv_benbi_ops = {
- .ctr = crypt_iv_benbi_ctr,
- .dtr = crypt_iv_benbi_dtr,
- .generator = crypt_iv_benbi_gen
-};
+ /*
+ * Layout of each crypto request:
+ *
+ * struct skcipher_request
+ * context
+ * padding
+ * struct dm_crypt_request
+ * padding
+ * IV
+ *
+ * The padding is added so that dm_crypt_request and the IV are
+ * correctly aligned.
+ */
+ unsigned int dmreq_start;
-static const struct crypt_iv_operations crypt_iv_null_ops = {
- .generator = crypt_iv_null_gen
-};
+ unsigned int per_bio_data_size;
-static const struct crypt_iv_operations crypt_iv_lmk_ops = {
- .ctr = crypt_iv_lmk_ctr,
- .dtr = crypt_iv_lmk_dtr,
- .init = crypt_iv_lmk_init,
- .wipe = crypt_iv_lmk_wipe,
- .generator = crypt_iv_lmk_gen,
- .post = crypt_iv_lmk_post
+ unsigned long flags;
+ unsigned int key_size;
+ unsigned int key_parts; /* independent parts in key buffer */
+ unsigned int key_extra_size; /* additional keys length */
+ u8 key[0];
};
-static const struct crypt_iv_operations crypt_iv_tcw_ops = {
- .ctr = crypt_iv_tcw_ctr,
- .dtr = crypt_iv_tcw_dtr,
- .init = crypt_iv_tcw_init,
- .wipe = crypt_iv_tcw_wipe,
- .generator = crypt_iv_tcw_gen,
- .post = crypt_iv_tcw_post
-};
+static void clone_init(struct dm_crypt_io *, struct bio *);
+static void kcryptd_queue_crypt(struct dm_crypt_io *io);
static void crypt_convert_init(struct crypt_config *cc,
struct convert_context *ctx,
@@ -833,71 +1739,16 @@ static struct skcipher_request *req_of_dmreq(struct crypt_config *cc,
return (struct skcipher_request *)((char *)dmreq - cc->dmreq_start);
}
-static u8 *iv_of_dmreq(struct crypt_config *cc,
- struct dm_crypt_request *dmreq)
-{
- return (u8 *)ALIGN((unsigned long)(dmreq + 1),
- crypto_skcipher_alignmask(any_tfm(cc)) + 1);
-}
-
-static int crypt_convert_block(struct crypt_config *cc,
- struct convert_context *ctx,
- struct skcipher_request *req)
-{
- struct bio_vec bv_in = bio_iter_iovec(ctx->bio_in, ctx->iter_in);
- struct bio_vec bv_out = bio_iter_iovec(ctx->bio_out, ctx->iter_out);
- struct dm_crypt_request *dmreq;
- u8 *iv;
- int r;
-
- dmreq = dmreq_of_req(cc, req);
- iv = iv_of_dmreq(cc, dmreq);
-
- dmreq->iv_sector = ctx->cc_sector;
- dmreq->ctx = ctx;
- sg_init_table(&dmreq->sg_in, 1);
- sg_set_page(&dmreq->sg_in, bv_in.bv_page, 1 << SECTOR_SHIFT,
- bv_in.bv_offset);
-
- sg_init_table(&dmreq->sg_out, 1);
- sg_set_page(&dmreq->sg_out, bv_out.bv_page, 1 << SECTOR_SHIFT,
- bv_out.bv_offset);
-
- bio_advance_iter(ctx->bio_in, &ctx->iter_in, 1 << SECTOR_SHIFT);
- bio_advance_iter(ctx->bio_out, &ctx->iter_out, 1 << SECTOR_SHIFT);
-
- if (cc->iv_gen_ops) {
- r = cc->iv_gen_ops->generator(cc, iv, dmreq);
- if (r < 0)
- return r;
- }
-
- skcipher_request_set_crypt(req, &dmreq->sg_in, &dmreq->sg_out,
- 1 << SECTOR_SHIFT, iv);
-
- if (bio_data_dir(ctx->bio_in) == WRITE)
- r = crypto_skcipher_encrypt(req);
- else
- r = crypto_skcipher_decrypt(req);
-
- if (!r && cc->iv_gen_ops && cc->iv_gen_ops->post)
- r = cc->iv_gen_ops->post(cc, iv, dmreq);
-
- return r;
-}
-
static void kcryptd_async_done(struct crypto_async_request *async_req,
int error);
static void crypt_alloc_req(struct crypt_config *cc,
struct convert_context *ctx)
{
- unsigned key_index = ctx->cc_sector & (cc->tfms_count - 1);
-
if (!ctx->req)
ctx->req = mempool_alloc(cc->req_pool, GFP_NOIO);
- skcipher_request_set_tfm(ctx->req, cc->tfms[key_index]);
+ skcipher_request_set_tfm(ctx->req, cc->tfm);
/*
* Use REQ_MAY_BACKLOG so a cipher driver internally backlogs
@@ -920,57 +1771,94 @@ static void crypt_free_req(struct crypt_config *cc,
/*
* Encrypt / decrypt data from one bio to another one (can be the same one)
*/
-static int crypt_convert(struct crypt_config *cc,
- struct convert_context *ctx)
+
+static int crypt_convert_bio(struct crypt_config *cc,
+ struct convert_context *ctx)
{
+ unsigned int cryptlen, n1, n2, nents, i = 0, bytes = 0;
+ struct skcipher_request *req;
+ struct dm_crypt_request *dmreq;
+ struct geniv_req_info rinfo;
+ struct bio_vec bv_in, bv_out;
int r;
atomic_set(&ctx->cc_pending, 1);
+ crypt_alloc_req(cc, ctx);
- while (ctx->iter_in.bi_size && ctx->iter_out.bi_size) {
+ req = ctx->req;
+ dmreq = dmreq_of_req(cc, req);
- crypt_alloc_req(cc, ctx);
+ n1 = bio_segments(ctx->bio_in);
+ n2 = bio_segments(ctx->bio_out);
+ nents = n1 > n2 ? n1 : n2;
+ nents = nents > MAX_SG_LIST ? MAX_SG_LIST : nents;
+ cryptlen = ctx->iter_in.bi_size;
- atomic_inc(&ctx->cc_pending);
+ DMDEBUG("dm-crypt:%s: segments:[in=%u, out=%u] bi_size=%u\n",
+ bio_data_dir(ctx->bio_in) == WRITE ? "write" : "read",
+ n1, n2, cryptlen);
- r = crypt_convert_block(cc, ctx, ctx->req);
+ dmreq->sg_in = kcalloc(nents, sizeof(struct scatterlist), GFP_KERNEL);
+ dmreq->sg_out = kcalloc(nents, sizeof(struct scatterlist), GFP_KERNEL);
+ if (!dmreq->sg_in || !dmreq->sg_out) {
+ DMERR("dm-crypt: Failed to allocate scatterlist\n");
+ r = -ENOMEM;
+ goto end;
+ }
+ dmreq->ctx = ctx;
- switch (r) {
- /*
- * The request was queued by a crypto driver
- * but the driver request queue is full, let's wait.
- */
- case -EBUSY:
- wait_for_completion(&ctx->restart);
- reinit_completion(&ctx->restart);
- /* fall through */
- /*
- * The request is queued and processed asynchronously,
- * completion function kcryptd_async_done() will be called.
- */
- case -EINPROGRESS:
- ctx->req = NULL;
- ctx->cc_sector++;
- continue;
- /*
- * The request was already processed (synchronously).
- */
- case 0:
- atomic_dec(&ctx->cc_pending);
- ctx->cc_sector++;
- cond_resched();
- continue;
+ sg_init_table(dmreq->sg_in, nents);
+ sg_init_table(dmreq->sg_out, nents);
- /* There was an error while processing the request. */
- default:
- atomic_dec(&ctx->cc_pending);
- return r;
- }
+ while (ctx->iter_in.bi_size && ctx->iter_out.bi_size && i < nents) {
+ bv_in = bio_iter_iovec(ctx->bio_in, ctx->iter_in);
+ bv_out = bio_iter_iovec(ctx->bio_out, ctx->iter_out);
+
+ sg_set_page(&dmreq->sg_in[i], bv_in.bv_page, bv_in.bv_len,
+ bv_in.bv_offset);
+ sg_set_page(&dmreq->sg_out[i], bv_out.bv_page, bv_out.bv_len,
+ bv_out.bv_offset);
+
+ bio_advance_iter(ctx->bio_in, &ctx->iter_in, bv_in.bv_len);
+ bio_advance_iter(ctx->bio_out, &ctx->iter_out, bv_out.bv_len);
+
+ bytes += bv_in.bv_len;
+ i++;
}
- return 0;
+ DMDEBUG("dm-crypt: Processed %u of %u bytes\n", bytes, cryptlen);
+
+ rinfo.iv_sector = ctx->cc_sector;
+ rinfo.nents = nents;
+
+ skcipher_request_set_crypt(req, dmreq->sg_in, dmreq->sg_out,
+ bytes, &rinfo);
+
+ if (bio_data_dir(ctx->bio_in) == WRITE)
+ r = crypto_skcipher_encrypt(req);
+ else
+ r = crypto_skcipher_decrypt(req);
+
+ switch (r) {
+ /* The request was queued so wait. */
+ case -EBUSY:
+ wait_for_completion(&ctx->restart);
+ reinit_completion(&ctx->restart);
+ /* fall through */
+ /*
+ * The request is queued and processed asynchronously,
+ * completion function kcryptd_async_done() is called.
+ */
+ case -EINPROGRESS:
+ ctx->req = NULL;
+ cond_resched();
+ break;
+ }
+end:
+ return r;
}
+
static void crypt_free_buffer_pages(struct crypt_config *cc, struct bio *clone);
/*
@@ -1070,11 +1958,17 @@ static void crypt_dec_pending(struct dm_crypt_io *io)
{
struct crypt_config *cc = io->cc;
struct bio *base_bio = io->base_bio;
+ struct dm_crypt_request *dmreq;
int error = io->error;
if (!atomic_dec_and_test(&io->io_pending))
return;
+ dmreq = dmreq_of_req(cc, io->ctx.req);
+ DMDEBUG("dm-crypt: Freeing scatterlists [sync]\n");
+ kfree(dmreq->sg_in);
+ kfree(dmreq->sg_out);
+
if (io->ctx.req)
crypt_free_req(cc, io->ctx.req, base_bio);
@@ -1313,7 +2207,7 @@ static void kcryptd_crypt_write_convert(struct dm_crypt_io *io)
sector += bio_sectors(clone);
crypt_inc_pending(io);
- r = crypt_convert(cc, &io->ctx);
+ r = crypt_convert_bio(cc, &io->ctx);
if (r)
io->error = -EIO;
crypt_finished = atomic_dec_and_test(&io->ctx.cc_pending);
@@ -1343,7 +2237,8 @@ static void kcryptd_crypt_read_convert(struct dm_crypt_io *io)
crypt_convert_init(cc, &io->ctx, io->base_bio, io->base_bio,
io->sector);
- r = crypt_convert(cc, &io->ctx);
+ r = crypt_convert_bio(cc, &io->ctx);
+
if (r < 0)
io->error = -EIO;
@@ -1371,12 +2266,13 @@ static void kcryptd_async_done(struct crypto_async_request *async_req,
return;
}
- if (!error && cc->iv_gen_ops && cc->iv_gen_ops->post)
- error = cc->iv_gen_ops->post(cc, iv_of_dmreq(cc, dmreq), dmreq);
-
if (error < 0)
io->error = -EIO;
+ DMDEBUG("dm-crypt: Freeing scatterlists and request struct [async]\n");
+ kfree(dmreq->sg_in);
+ kfree(dmreq->sg_out);
+
crypt_free_req(cc, req_of_dmreq(cc, dmreq), io->base_bio);
if (!atomic_dec_and_test(&ctx->cc_pending))
@@ -1430,62 +2326,38 @@ static int crypt_decode_key(u8 *key, char *hex, unsigned int size)
return 0;
}
-static void crypt_free_tfms(struct crypt_config *cc)
+static void crypt_free_tfm(struct crypt_config *cc)
{
- unsigned i;
-
- if (!cc->tfms)
+ if (!cc->tfm)
return;
- for (i = 0; i < cc->tfms_count; i++)
- if (cc->tfms[i] && !IS_ERR(cc->tfms[i])) {
- crypto_free_skcipher(cc->tfms[i]);
- cc->tfms[i] = NULL;
- }
+ if (cc->tfm && !IS_ERR(cc->tfm))
+ crypto_free_skcipher(cc->tfm);
- kfree(cc->tfms);
- cc->tfms = NULL;
+ cc->tfm = NULL;
}
-static int crypt_alloc_tfms(struct crypt_config *cc, char *ciphermode)
+static int crypt_alloc_tfm(struct crypt_config *cc, char *ciphermode)
{
- unsigned i;
int err;
- cc->tfms = kzalloc(cc->tfms_count * sizeof(struct crypto_skcipher *),
- GFP_KERNEL);
- if (!cc->tfms)
- return -ENOMEM;
-
- for (i = 0; i < cc->tfms_count; i++) {
- cc->tfms[i] = crypto_alloc_skcipher(ciphermode, 0, 0);
- if (IS_ERR(cc->tfms[i])) {
- err = PTR_ERR(cc->tfms[i]);
- crypt_free_tfms(cc);
- return err;
- }
+ cc->tfm = crypto_alloc_skcipher(ciphermode, 0, 0);
+ if (IS_ERR(cc->tfm)) {
+ err = PTR_ERR(cc->tfm);
+ crypt_free_tfm(cc);
+ return err;
}
return 0;
}
-static int crypt_setkey(struct crypt_config *cc)
+static inline int crypt_setkey(struct crypt_config *cc, enum setkey_op keyop,
+ char *ivopts)
{
- unsigned subkey_size;
- int err = 0, i, r;
-
- /* Ignore extra keys (which are used for IV etc) */
- subkey_size = (cc->key_size - cc->key_extra_size) >> ilog2(cc->tfms_count);
+ DECLARE_GENIV_KEY(kinfo, keyop, cc->tfms_count, cc->key, cc->key_size,
+ cc->key_parts, ivopts);
- for (i = 0; i < cc->tfms_count; i++) {
- r = crypto_skcipher_setkey(cc->tfms[i],
- cc->key + (i * subkey_size),
- subkey_size);
- if (r)
- err = r;
- }
-
- return err;
+ return crypto_skcipher_setkey(cc->tfm, (u8 *) &kinfo, sizeof(kinfo));
}
#ifdef CONFIG_KEYS
@@ -1498,7 +2370,9 @@ static bool contains_whitespace(const char *str)
return false;
}
-static int crypt_set_keyring_key(struct crypt_config *cc, const char *key_string)
+static int crypt_set_keyring_key(struct crypt_config *cc,
+ const char *key_string,
+ enum setkey_op keyop, char *ivopts)
{
char *new_key_string, *key_desc;
int ret;
@@ -1559,7 +2433,7 @@ static int crypt_set_keyring_key(struct crypt_config *cc, const char *key_string
/* clear the flag since following operations may invalidate previously valid key */
clear_bit(DM_CRYPT_KEY_VALID, &cc->flags);
- ret = crypt_setkey(cc);
+ ret = crypt_setkey(cc, keyop, ivopts);
/* wipe the kernel key payload copy in each case */
memset(cc->key, 0, cc->key_size * sizeof(u8));
@@ -1599,7 +2473,9 @@ static int get_key_size(char **key_string)
#else
-static int crypt_set_keyring_key(struct crypt_config *cc, const char *key_string)
+static int crypt_set_keyring_key(struct crypt_config *cc,
+ const char *key_string,
+ enum setkey_op keyop, char *ivopts)
{
return -EINVAL;
}
@@ -1611,7 +2487,8 @@ static int get_key_size(char **key_string)
#endif
-static int crypt_set_key(struct crypt_config *cc, char *key)
+static int crypt_set_key(struct crypt_config *cc, enum setkey_op keyop,
+ char *key, char *ivopts)
{
int r = -EINVAL;
int key_string_len = strlen(key);
@@ -1622,7 +2499,7 @@ static int crypt_set_key(struct crypt_config *cc, char *key)
/* ':' means the key is in kernel keyring, short-circuit normal key processing */
if (key[0] == ':') {
- r = crypt_set_keyring_key(cc, key + 1);
+ r = crypt_set_keyring_key(cc, key + 1, keyop, ivopts);
goto out;
}
@@ -1636,7 +2513,7 @@ static int crypt_set_key(struct crypt_config *cc, char *key)
if (cc->key_size && crypt_decode_key(cc->key, key, cc->key_size) < 0)
goto out;
- r = crypt_setkey(cc);
+ r = crypt_setkey(cc, keyop, ivopts);
if (!r)
set_bit(DM_CRYPT_KEY_VALID, &cc->flags);
@@ -1647,6 +2524,17 @@ static int crypt_set_key(struct crypt_config *cc, char *key)
return r;
}
+static int crypt_init_key(struct dm_target *ti, char *key, char *ivopts)
+{
+ struct crypt_config *cc = ti->private;
+ int ret;
+
+ ret = crypt_set_key(cc, SETKEY_OP_INIT, key, ivopts);
+ if (ret < 0)
+ ti->error = "Error decoding and setting key";
+ return ret;
+}
+
static int crypt_wipe_key(struct crypt_config *cc)
{
clear_bit(DM_CRYPT_KEY_VALID, &cc->flags);
@@ -1654,7 +2542,7 @@ static int crypt_wipe_key(struct crypt_config *cc)
kzfree(cc->key_string);
cc->key_string = NULL;
- return crypt_setkey(cc);
+ return crypt_setkey(cc, SETKEY_OP_WIPE, NULL);
}
static void crypt_dtr(struct dm_target *ti)
@@ -1674,7 +2562,7 @@ static void crypt_dtr(struct dm_target *ti)
if (cc->crypt_queue)
destroy_workqueue(cc->crypt_queue);
- crypt_free_tfms(cc);
+ crypt_free_tfm(cc);
if (cc->bs)
bioset_free(cc->bs);
@@ -1682,9 +2570,6 @@ static void crypt_dtr(struct dm_target *ti)
mempool_destroy(cc->page_pool);
mempool_destroy(cc->req_pool);
- if (cc->iv_gen_ops && cc->iv_gen_ops->dtr)
- cc->iv_gen_ops->dtr(cc);
-
if (cc->dev)
dm_put_device(ti, cc->dev);
@@ -1762,22 +2647,30 @@ static int crypt_ctr_cipher(struct dm_target *ti,
if (!cipher_api)
goto bad_mem;
- ret = snprintf(cipher_api, CRYPTO_MAX_ALG_NAME,
- "%s(%s)", chainmode, cipher);
+create_cipher:
+ /* For those ciphers which do not support IVs,
+ * use the 'null' template cipher
+ */
+
+ if (!ivmode)
+ ivmode = "null";
+
+ ret = snprintf(cipher_api, CRYPTO_MAX_ALG_NAME, "%s(%s(%s))",
+ ivmode, chainmode, cipher);
if (ret < 0) {
kfree(cipher_api);
goto bad_mem;
}
/* Allocate cipher */
- ret = crypt_alloc_tfms(cc, cipher_api);
+ ret = crypt_alloc_tfm(cc, cipher_api);
if (ret < 0) {
ti->error = "Error allocating crypto tfm";
goto bad;
}
/* Initialize IV */
- cc->iv_size = crypto_skcipher_ivsize(any_tfm(cc));
+ cc->iv_size = crypto_skcipher_ivsize(cc->tfm);
if (cc->iv_size)
/* at least a 64 bit sector number should fit in our buffer */
cc->iv_size = max(cc->iv_size,
@@ -1785,23 +2678,10 @@ static int crypt_ctr_cipher(struct dm_target *ti,
else if (ivmode) {
DMWARN("Selected cipher does not support IVs");
ivmode = NULL;
+ goto create_cipher;
}
- /* Choose ivmode, see comments at iv code. */
- if (ivmode == NULL)
- cc->iv_gen_ops = NULL;
- else if (strcmp(ivmode, "plain") == 0)
- cc->iv_gen_ops = &crypt_iv_plain_ops;
- else if (strcmp(ivmode, "plain64") == 0)
- cc->iv_gen_ops = &crypt_iv_plain64_ops;
- else if (strcmp(ivmode, "essiv") == 0)
- cc->iv_gen_ops = &crypt_iv_essiv_ops;
- else if (strcmp(ivmode, "benbi") == 0)
- cc->iv_gen_ops = &crypt_iv_benbi_ops;
- else if (strcmp(ivmode, "null") == 0)
- cc->iv_gen_ops = &crypt_iv_null_ops;
- else if (strcmp(ivmode, "lmk") == 0) {
- cc->iv_gen_ops = &crypt_iv_lmk_ops;
+ if (strcmp(ivmode, "lmk") == 0) {
/*
* Version 2 and 3 is recognised according
* to length of provided multi-key string.
@@ -1813,39 +2693,14 @@ static int crypt_ctr_cipher(struct dm_target *ti,
cc->key_extra_size = cc->key_size / cc->key_parts;
}
} else if (strcmp(ivmode, "tcw") == 0) {
- cc->iv_gen_ops = &crypt_iv_tcw_ops;
cc->key_parts += 2; /* IV + whitening */
cc->key_extra_size = cc->iv_size + TCW_WHITENING_SIZE;
- } else {
- ret = -EINVAL;
- ti->error = "Invalid IV mode";
- goto bad;
}
/* Initialize and set key */
- ret = crypt_set_key(cc, key);
- if (ret < 0) {
- ti->error = "Error decoding and setting key";
+ ret = crypt_init_key(ti, key, ivopts);
+ if (ret < 0)
goto bad;
- }
-
- /* Allocate IV */
- if (cc->iv_gen_ops && cc->iv_gen_ops->ctr) {
- ret = cc->iv_gen_ops->ctr(cc, ti, ivopts);
- if (ret < 0) {
- ti->error = "Error creating IV";
- goto bad;
- }
- }
-
- /* Initialize IV (set keys for ESSIV etc) */
- if (cc->iv_gen_ops && cc->iv_gen_ops->init) {
- ret = cc->iv_gen_ops->init(cc);
- if (ret < 0) {
- ti->error = "Error initialising IV";
- goto bad;
- }
- }
ret = 0;
bad:
@@ -1868,7 +2723,6 @@ static int crypt_ctr(struct dm_target *ti, unsigned int argc, char **argv)
unsigned int opt_params;
unsigned long long tmpll;
int ret;
- size_t iv_size_padding;
struct dm_arg_set as;
const char *opt_string;
char dummy;
@@ -1901,25 +2755,12 @@ static int crypt_ctr(struct dm_target *ti, unsigned int argc, char **argv)
goto bad;
cc->dmreq_start = sizeof(struct skcipher_request);
- cc->dmreq_start += crypto_skcipher_reqsize(any_tfm(cc));
+ cc->dmreq_start += crypto_skcipher_reqsize(cc->tfm);
cc->dmreq_start = ALIGN(cc->dmreq_start, __alignof__(struct dm_crypt_request));
- if (crypto_skcipher_alignmask(any_tfm(cc)) < CRYPTO_MINALIGN) {
- /* Allocate the padding exactly */
- iv_size_padding = -(cc->dmreq_start + sizeof(struct dm_crypt_request))
- & crypto_skcipher_alignmask(any_tfm(cc));
- } else {
- /*
- * If the cipher requires greater alignment than kmalloc
- * alignment, we don't know the exact position of the
- * initialization vector. We must assume worst case.
- */
- iv_size_padding = crypto_skcipher_alignmask(any_tfm(cc));
- }
-
ret = -ENOMEM;
cc->req_pool = mempool_create_kmalloc_pool(MIN_IOS, cc->dmreq_start +
- sizeof(struct dm_crypt_request) + iv_size_padding + cc->iv_size);
+ sizeof(struct dm_crypt_request));
if (!cc->req_pool) {
ti->error = "Cannot allocate crypt request mempool";
goto bad;
@@ -1927,8 +2768,7 @@ static int crypt_ctr(struct dm_target *ti, unsigned int argc, char **argv)
cc->per_bio_data_size = ti->per_io_data_size =
ALIGN(sizeof(struct dm_crypt_io) + cc->dmreq_start +
- sizeof(struct dm_crypt_request) + iv_size_padding + cc->iv_size,
- ARCH_KMALLOC_MINALIGN);
+ sizeof(struct dm_crypt_request), ARCH_KMALLOC_MINALIGN);
cc->page_pool = mempool_create_page_pool(BIO_MAX_PAGES, 0);
if (!cc->page_pool) {
@@ -2072,8 +2912,9 @@ static int crypt_map(struct dm_target *ti, struct bio *bio)
if (bio_data_dir(io->base_bio) == READ) {
if (kcryptd_io_read(io, GFP_NOWAIT))
kcryptd_queue_read(io);
- } else
+ } else {
kcryptd_queue_crypt(io);
+ }
return DM_MAPIO_SUBMITTED;
}
@@ -2155,7 +2996,7 @@ static void crypt_resume(struct dm_target *ti)
static int crypt_message(struct dm_target *ti, unsigned argc, char **argv)
{
struct crypt_config *cc = ti->private;
- int key_size, ret = -EINVAL;
+ int key_size;
if (argc < 2)
goto error;
@@ -2173,19 +3014,9 @@ static int crypt_message(struct dm_target *ti, unsigned argc, char **argv)
return -EINVAL;
}
- ret = crypt_set_key(cc, argv[2]);
- if (ret)
- return ret;
- if (cc->iv_gen_ops && cc->iv_gen_ops->init)
- ret = cc->iv_gen_ops->init(cc);
- return ret;
+ return crypt_set_key(cc, SETKEY_OP_SET, argv[2], NULL);
}
if (argc == 2 && !strcasecmp(argv[1], "wipe")) {
- if (cc->iv_gen_ops && cc->iv_gen_ops->wipe) {
- ret = cc->iv_gen_ops->wipe(cc);
- if (ret)
- return ret;
- }
return crypt_wipe_key(cc);
}
}
@@ -2216,7 +3047,7 @@ static void crypt_io_hints(struct dm_target *ti, struct queue_limits *limits)
static struct target_type crypt_target = {
.name = "crypt",
- .version = {1, 15, 0},
+ .version = {1, 16, 0},
.module = THIS_MODULE,
.ctr = crypt_ctr,
.dtr = crypt_dtr,
@@ -2234,6 +3065,7 @@ static int __init dm_crypt_init(void)
{
int r;
+ geniv_register_algs();
r = dm_register_target(&crypt_target);
if (r < 0)
DMERR("register failed %d", r);
@@ -2244,6 +3076,7 @@ static int __init dm_crypt_init(void)
static void __exit dm_crypt_exit(void)
{
dm_unregister_target(&crypt_target);
+ geniv_deregister_algs();
}
module_init(dm_crypt_init);
diff --git a/include/crypto/geniv.h b/include/crypto/geniv.h
new file mode 100644
index 0000000..3c518fe
--- /dev/null
+++ b/include/crypto/geniv.h
@@ -0,0 +1,46 @@
+/*
+ * geniv: common interface for IV generation algorithms
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms of the GNU General Public License as published by the Free
+ * Software Foundation; either version 2 of the License, or (at your option)
+ * any later version.
+ *
+ */
+#ifndef _CRYPTO_GENIV_
+#define _CRYPTO_GENIV_
+
+#define SECTOR_SIZE (1 << SECTOR_SHIFT)
+#define SECTOR_MASK (~((1 << SECTOR_SHIFT) - 1))
+
+enum setkey_op {
+ SETKEY_OP_INIT,
+ SETKEY_OP_SET,
+ SETKEY_OP_WIPE,
+};
+
+struct geniv_key_info {
+ enum setkey_op keyop;
+ unsigned int tfms_count;
+ u8 *key;
+ unsigned int key_size;
+ unsigned int key_parts;
+ char *ivopts;
+};
+
+#define DECLARE_GENIV_KEY(c, op, n, k, sz, kp, opts) \
+ struct geniv_key_info c = { \
+ .keyop = op, \
+ .tfms_count = n, \
+ .key = k, \
+ .key_size = sz, \
+ .key_parts = kp, \
+ .ivopts = opts, \
+ }
+
+struct geniv_req_info {
+ sector_t iv_sector;
+ unsigned int nents;
+};
+
+#endif
--
Binoy Jayan
^ permalink raw reply related
* [PATCH v6 0/2] IV Generation algorithms for dm-crypt
From: Binoy Jayan @ 2017-06-22 4:41 UTC (permalink / raw)
To: Mark Brown
Cc: Arnd Bergmann, linux-crypto, linux-kernel, dm-devel, linux-raid,
Rajendra, Binoy Jayan
===============================================================================
dm-crypt optimization for larger block sizes
===============================================================================
Currently, the iv generation algorithms are implemented in dm-crypt.c. The goal
is to move these algorithms from the dm layer to the kernel crypto layer by
implementing them as template ciphers so they can be used in relation with
algorithms like aes, and with multiple modes like cbc, ecb etc. As part of this
patchset, the iv-generation code is moved from the dm layer to the crypto layer
and adapt the dm-layer to send a whole 'bio' (as defined in the block layer)
at a time. Each bio contains the in memory representation of physically
contiguous disk blocks. Since the bio itself may not be contiguous in main
memory, the dm layer sets up a chained scatterlist of these blocks split into
physically contiguous segments in memory so that DMA can be performed.
One challenge in doing so is that the IVs are generated based on a 512-byte
sector number. This infact limits the block sizes to 512 bytes. But this should
not be a problem if a hardware with iv generation support is used. The geniv
itself splits the segments into sectors so it could choose the IV based on
sector number. But it could be modelled in hardware effectively by not
splitting up the segments in the bio.
Another challenge faced is that dm-crypt has an option to use multiple keys.
The key selection is done based on the sector number. If the whole bio is
encrypted / decrypted with the same key, the encrypted volumes will not be
compatible with the original dm-crypt [without the changes]. So, the key
selection code is moved to crypto layer so the neighboring sectors are
encrypted with a different key.
The dm layer allocates space for iv. The hardware drivers can choose to make
use of this space to generate their IVs sequentially or allocate it on their
own. This can be moved to crypto layer too. Postponing this decision until
the requirement to integrate milan's changes are clear.
Interface to the crypto layer - include/crypto/geniv.h
More information on test procedure can be found in v1.
Results of performance tests with software crypto in v5.
The patch 'crypto: Multikey template for essiv' depends on
the following patches by Gilad:
MAINTAINERS: add Gilad BY as maintainer for ccree
staging: ccree: add devicetree bindings
staging: ccree: add TODO list
staging: add ccree crypto driver
Revisions:
----------
v1: https://patchwork.kernel.org/patch/9439175
v2: https://patchwork.kernel.org/patch/9471923
v3: https://lkml.org/lkml/2017/1/18/170
v4: https://patchwork.kernel.org/patch/9559665
v5: https://patchwork.kernel.org/patch/9669237
v5 --> v6:
----------
1. Moved allocation of initialization vectors to the iv-generator
2. Few consmetic changes as the consequence of the above
3. Few logical to boolean expressions for faster calculation
4. Included multikey template for splitting keys.
This needs testing with real hardware (juno with ccree)
and also modification. It is only for testing and not
for inclusion upstream.
v4 --> v5
----------
1. Fix for the multiple instance issue in /proc/crypto
2. Few cosmetic changes including struct alignment
3. Simplified 'struct geniv_req_info'
v3 --> v4
----------
Fix for the bug reported by Gilad Ben-Yossef.
The element '__ctx' in 'struct skcipher_request req' overflowed into the
element 'struct scatterlist src' which immediately follows 'req' in
'struct geniv_subreq' and corrupted src.
v2 --> v3
----------
1. Moved iv algorithms in dm-crypt.c for control
2. Key management code moved from dm layer to cryto layer
so that cipher instance selection can be made depending on key_index
3. The revision v2 had scatterlist nodes created for every sector in the bio.
It is modified to create only once scatterlist node to reduce memory
foot print. Synchronous requests are processed sequentially. Asynchronous
requests are processed in parallel and is freed in the async callback.
4. Changed allocation for sub-requests using mempool
v1 --> v2
----------
1. dm-crypt changes to process larger block sizes (one segment in a bio)
2. Incorporated changes w.r.t. comments from Herbert.
Binoy Jayan (2):
crypto: Add IV generation algorithms
crypto: Multikey template for essiv
drivers/md/dm-crypt.c | 1940 +++++++++++++++++++++++++++-----------
drivers/staging/ccree/Makefile | 2 +-
drivers/staging/ccree/essiv.c | 777 +++++++++++++++
drivers/staging/ccree/essiv_sw.c | 1040 ++++++++++++++++++++
include/crypto/geniv.h | 46 +
5 files changed, 3251 insertions(+), 554 deletions(-)
create mode 100644 drivers/staging/ccree/essiv.c
create mode 100644 drivers/staging/ccree/essiv_sw.c
create mode 100644 include/crypto/geniv.h
--
Binoy Jayan
^ permalink raw reply
* AW: Wir geben jährlich Darlehen für 2% Zinsen aus
From: Bernhard Stöckl @ 2017-06-21 22:51 UTC (permalink / raw)
Wir vergeben Kredite mit einem Zinssatz von jährlich 2%.
Die Bearbeitung des Antrags erfolgt rasch, wir verlangen keine Gebühren, was sie beantragen werden wir annehmen. Wir bewilligen Kredite von bis zu 40 Millionen Euro und von mindestens 15.000 Euro. Sie können einen geschäftlichen oder privaten Kredit beantrage, wobei Sie bei Geschäftskrediten eine Zahlungsfrist von einem Jahr erhalten.
Anfragen bitte per Mail an: sunrisefunding01@hotmail.com
^ permalink raw reply
* Assemble the previously replaced components of a previously created array into an active array
From: LM @ 2017-06-21 20:15 UTC (permalink / raw)
To: Linux RAID
Hi,
I replaced all component devices of my RAID5 like in the example below:
mdadm /dev/md/test --add-spare /dev/loop20 --replace /dev/loop10 --with /dev/loop20
mdadm /dev/md/test --add-spare /dev/loop21 --replace /dev/loop11 --with /dev/loop21
mdadm /dev/md/test --add-spare /dev/loop22 --replace /dev/loop12 --with /dev/loop22
mdadm /dev/md/test --add-spare /dev/loop23 --replace /dev/loop13 --with /dev/loop23
mdadm /dev/md/test --remove /dev/loop10 /dev/loop11 /dev/loop12 /dev/loop13
Now one of my new disks failed. I currently rebuilding the RAID with the
failed disk hoping that the reason for failure has been resolved.
But I wonder if I could just:
1) power down the machine
2) replace all for disks by the removed old (and functioning) disks
3) assemble the old RAID
This would be a great relief, because no relevant data changed since the
replacement. However, the RAID has been in use during the replacement
and this makes me worry that the RAID cannot be assembled using the old
disks due to data inconsistency.
Best,
Lars
^ permalink raw reply
* Re: raid-1 starting with 1 drive after brief hiatus with cruddy controller.
From: r23 @ 2017-06-21 18:29 UTC (permalink / raw)
To: NeilBrown; +Cc: linux-raid
In-Reply-To: <871sqh57x5.fsf@notabene.neil.brown.name>
Hi All,
mdadm --assemble /dev/md/debian:0 --update=name -v /dev/sdb2 --run
started the container with the 1 drive and I was able to mount it !!
Thank you all very much. Happy day :)
root@mars:~/raid/bin/util# mdadm --assemble /dev/md/debian:0 --update=name -v /dev/sdb2 --run
mdadm: looking for devices for /dev/md/debian:0
mdadm: /dev/sdb2 is identified as a member of /dev/md/debian:0, slot 1.
mdadm: no uptodate device for slot 0 of /dev/md/debian:0
mdadm: added /dev/sdb2 to /dev/md/debian:0 as 1
mdadm: /dev/md/debian:0 has been started with 1 drive (out of 2).
root@mars:~/raid/bin/util# mkdir /mnt/thanks
root@mars:~/raid/bin/util# mount -o ro /dev/md/debian:0 /mnt/thanks
root@mars:~/raid/bin/util# cd /mnt/thanks
root@mars:/mnt/thanks# ls -l
total 63900
drwxr-xr-x 19 root userme 4096 Nov 7....
::
::
::
----------------
On Mon, 19 Jun 2017 06:59:18 +1000
NeilBrown <neilb@suse.com> wrote:
> On Sun, Jun 18 2017, r23@anchordrop.net wrote:
> >
> > root@mars:~/raid/bin# mdadm --misc --examine /dev/sdb2
> > /dev/sdb2:
> > Magic : a92b4efc
> > Version : 1.2
> > Feature Map : 0x1
> > Array UUID : 2a3489a6:b430c744:2c89a792:98521913
> > Name : debian:0
> > Creation Time : Sat May 9 17:44:25 2015
> > Raid Level : raid1
> > Raid Devices : 2
> >
> > Avail Dev Size : 9765179392 (4656.40 GiB 4999.77 GB)
> > Array Size : 4882589696 (4656.40 GiB 4999.77 GB)
> > Data Offset : 262144 sectors
> > Super Offset : 8 sectors
> > Unused Space : before=262056 sectors, after=0 sectors
> > State : clean
> > Device UUID : 4697b088:5d3b1ae5:55d30f65:516df63d
> >
> > Internal Bitmap : 8 sectors from superblock
> > Update Time : Wed Jun 14 23:53:27 2017
> > Bad Block Log : 512 entries available at offset 72 sectors
> > Checksum : 1f245410 - expected 1f249410
>
> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
>
> checksum is wrong.
>
> > root@mars:~/raid/bin/util/lsdrv# mdadm
> > --assemble /dev/md/debian:0 -v /dev/sdb2 --run mdadm: looking for
> > devices for /dev/md/debian:0 mdadm: /dev/sdb2 is identified as a
> > member of /dev/md/debian:0, slot 1. mdadm: no uptodate device for
> > slot 0 of /dev/md/debian:0 mdadm: failed to add /dev/sdb2
> > to /dev/md/debian:0: Invalid argument
>
> Bad checksum caused kernel to reject the device.
>
> There is no obvious way to correct the checksum, but I think you can
> force it by using the --update option to --assemble
> e.g.
>
> mdadm --assemble /dev/md/debian:0 --update=name -v /dev/sdb2
>
> When doing that, you should check that other fields all look correct
> first, because something must have changed to affect the bitmap.
> (I cannot see anything that looks wrong)
>
> NeilBrown
^ permalink raw reply
* Re: [PATCH] md: use a separate bio_set for synchronous IO.
From: Shaohua Li @ 2017-06-21 17:49 UTC (permalink / raw)
To: NeilBrown; +Cc: linux-raid
In-Reply-To: <87wp861cfe.fsf@notabene.neil.brown.name>
On Wed, Jun 21, 2017 at 09:12:21AM +1000, Neil Brown wrote:
>
> md devices allocate a bio_set and use it for two
> distinct purposes.
> mddev->bio_set is used to clone bios as part of sending
> upper level requests down to lower level devices,
> and it is also use for synchronous IO such as superblock
> and bitmap updates, and for correcting read errors.
>
> This multiple usage can lead to deadlocks. It is likely
> that cloned bios might be queued for write and to be
> waiting for a metadata update before the write can be permitted.
> If the cloning exhausted mddev->bio_set, the metadata update
> may not be able to proceed.
>
> This scenario has been seen during heavy testing, with lots of IO and
> lots of memory pressure.
>
> Address this by adding a new bio_set specifically for synchronous IO.
> All synchronous IO goes directly to the underlying device and is not
> queued at the md level, so request using entries from the new
> mddev->sync_set will complete in a timely fashion.
> Requests that use mddev->bio_set will sometimes need to wait
> for synchronous IO, but will no longer risk deadlocking that iO.
>
> Also: small simplification in mddev_put(): there is no need to
> wait until the spinlock is released before calling bioset_free().
applied, thanks
> Signed-off-by: NeilBrown <neilb@suse.com>
> ---
> drivers/md/md.c | 27 ++++++++++++++++++++-------
> drivers/md/md.h | 3 +++
> 2 files changed, 23 insertions(+), 7 deletions(-)
>
> diff --git a/drivers/md/md.c b/drivers/md/md.c
> index 87edc342ccb3..9d0eb50c458e 100644
> --- a/drivers/md/md.c
> +++ b/drivers/md/md.c
> @@ -203,6 +203,14 @@ struct bio *bio_alloc_mddev(gfp_t gfp_mask, int nr_iovecs,
> }
> EXPORT_SYMBOL_GPL(bio_alloc_mddev);
>
> +static struct bio *md_bio_alloc_sync(struct mddev *mddev)
> +{
> + if (!mddev->sync_set)
> + return bio_alloc(GFP_NOIO, 1);
> +
> + return bio_alloc_bioset(GFP_NOIO, 1, mddev->sync_set);
> +}
> +
> /*
> * We have a system wide 'event count' that is incremented
> * on any 'interesting' event, and readers of /proc/mdstat
> @@ -462,8 +470,6 @@ static void mddev_delayed_delete(struct work_struct *ws);
>
> static void mddev_put(struct mddev *mddev)
> {
> - struct bio_set *bs = NULL;
> -
> if (!atomic_dec_and_lock(&mddev->active, &all_mddevs_lock))
> return;
> if (!mddev->raid_disks && list_empty(&mddev->disks) &&
> @@ -471,8 +477,12 @@ static void mddev_put(struct mddev *mddev)
> /* Array is not configured at all, and not held active,
> * so destroy it */
> list_del_init(&mddev->all_mddevs);
> - bs = mddev->bio_set;
> + if (mddev->bio_set)
> + bioset_free(mddev->bio_set);
> + if (mddev->sync_set)
> + bioset_free(mddev->sync_set);
> mddev->bio_set = NULL;
> + mddev->sync_set = NULL;
> if (mddev->gendisk) {
> /* We did a probe so need to clean up. Call
> * queue_work inside the spinlock so that
> @@ -485,8 +495,6 @@ static void mddev_put(struct mddev *mddev)
> kfree(mddev);
> }
> spin_unlock(&all_mddevs_lock);
> - if (bs)
> - bioset_free(bs);
> }
>
> static void md_safemode_timeout(unsigned long data);
> @@ -751,7 +759,7 @@ void md_super_write(struct mddev *mddev, struct md_rdev *rdev,
> if (test_bit(Faulty, &rdev->flags))
> return;
>
> - bio = bio_alloc_mddev(GFP_NOIO, 1, mddev);
> + bio = md_bio_alloc_sync(mddev);
>
> atomic_inc(&rdev->nr_pending);
>
> @@ -783,7 +791,7 @@ int md_super_wait(struct mddev *mddev)
> int sync_page_io(struct md_rdev *rdev, sector_t sector, int size,
> struct page *page, int op, int op_flags, bool metadata_op)
> {
> - struct bio *bio = bio_alloc_mddev(GFP_NOIO, 1, rdev->mddev);
> + struct bio *bio = md_bio_alloc_sync(rdev->mddev);
> int ret;
>
> bio->bi_bdev = (metadata_op && rdev->meta_bdev) ?
> @@ -5432,6 +5440,11 @@ int md_run(struct mddev *mddev)
> if (!mddev->bio_set)
> return -ENOMEM;
> }
> + if (mddev->sync_set == NULL) {
> + mddev->sync_set = bioset_create(BIO_POOL_SIZE, 0);
> + if (!mddev->sync_set)
> + return -ENOMEM;
> + }
>
> spin_lock(&pers_lock);
> pers = find_pers(mddev->level, mddev->clevel);
> diff --git a/drivers/md/md.h b/drivers/md/md.h
> index 0fa1de42c42b..a1c7aa89bb67 100644
> --- a/drivers/md/md.h
> +++ b/drivers/md/md.h
> @@ -444,6 +444,9 @@ struct mddev {
> struct attribute_group *to_remove;
>
> struct bio_set *bio_set;
> + struct bio_set *sync_set; /* for sync operations like
> + * metadata and bitmap writes
> + */
>
> /* Generic flush handling.
> * The last to finish preflush schedules a worker to submit
> --
> 2.12.2
>
^ permalink raw reply
* Re: [PATCH] md/bitmap: don't read page from device with Bitmap_sync
From: Shaohua Li @ 2017-06-21 17:47 UTC (permalink / raw)
To: NeilBrown; +Cc: Guoqing Jiang, linux-raid
In-Reply-To: <87o9tj2tse.fsf@notabene.neil.brown.name>
On Tue, Jun 20, 2017 at 01:59:45PM +1000, Neil Brown wrote:
> On Mon, Jun 19 2017, Shaohua Li wrote:
>
> > On Mon, Jun 19, 2017 at 09:11:23AM +1000, Neil Brown wrote:
> >> On Fri, Jun 16 2017, Shaohua Li wrote:
> >>
> >> > On Fri, Jun 16, 2017 at 05:19:27PM +0800, Guoqing Jiang wrote:
> >> >> The device owns Bitmap_sync flag needs recovery
> >> >> to become in sync, and read page from this type
> >> >> device could get stale status.
> >> >>
> >> >> Signed-off-by: Guoqing Jiang <gqjiang@suse.com>
> >> >> ---
> >> >> When develop for clustered raid10 feature, if add a
> >> >> disk under grow mode in master node, I could get
> >> >> the "bitmap superblock UUID mismatch" warning due
> >> >> to the page is read from Bitmap_sync device.
> >> >>
> >> >> drivers/md/bitmap.c | 3 ++-
> >> >> 1 file changed, 2 insertions(+), 1 deletion(-)
> >> >>
> >> >> diff --git a/drivers/md/bitmap.c b/drivers/md/bitmap.c
> >> >> index bf7419a..bf34cd8 100644
> >> >> --- a/drivers/md/bitmap.c
> >> >> +++ b/drivers/md/bitmap.c
> >> >> @@ -156,7 +156,8 @@ static int read_sb_page(struct mddev *mddev, loff_t offset,
> >> >>
> >> >> rdev_for_each(rdev, mddev) {
> >> >> if (! test_bit(In_sync, &rdev->flags)
> >> >
> >> > Why In_sync is set for the rdev? I think it shoudn't.
> >>
> >> There are several states a device can be in.
> >> If ->raid_disk is < 0, then the device is a spare, and doesn't contain
> >> and valid data.
> >> Otherwise ->raid_disk >=0 and:
> >> In_sync is clear, which means that blocks before ->recovery_offset
> >> contain valid data, and blocks after there don't
> >> In_sync is set, which means ->recovery_offset is irrelevant and
> >> should be treated as MaxSector
> >> Bitmap_sync is set, which could apply in either of the above cases,
> >> and means blocks corresponding to bits that are set in the bitmap
> >> may not be up to date.
> >>
> >> Bitmap_sync is only relevant before a device has been handed to the
> >> personality. After it has been added, ->recovery_cp ensures that the
> >> blocks that might be wrong are not read until until the bitmap-based
> >> recovery has fixed them up.
> >>
> >> Bitmap_sync is only use to stop the device from being given to the
> >> personality if a recovery won't be started because the array is
> >> read-only.
> >>
> >> So it is perfectly valid for both In_sync and Bitmap_sync to be set.
> >> If they are, it makes sense to avoid reading bitmap information from the
> >> Bitmap_sync device, as that will be out-of-date.
> >>
> >> I'm not quite sure why Guoqing is getting a UUID mismatch ... it
> >> suggests that the new device wasn't initialized properly. So there
> >> might be another bug. But I think this is definitely a bug.
> >
> > Can you describe a scenario a disk has both bits set? I had hard time to
> > imagine it.
>
> An array with a bitmap has a device fail (cable break?).
> So the bitmap stops clearing bits and ->events_cleared stays
> where it was, with the same value as events in the superblock
> of the failed device.
>
> The device then starts working (cable is replaced) and
> it device is added back to the array.
> super_1_validate notices that the array has a bitmap and that the event
> count in the new device is >= mddev->bitmap->events_cleared, but <
> mddev->events.
> i.e. it is in the range for which a bitmap-based recovery is both
> possible and needed.
> So it sets Bitmap_sync.
> Then it notices that the 'role' in a valid role for the array,
> so that role is stored in rdev->saved_raid_disk and, as there is no
> MD_FEATURE_RECOVERY_OFFSET, In_sync is set.
> A similar sequence can happen in super_90_validate.
This part is quite confusing actually. The failed device is kicked out, so the
array is degraded. When we readd the device, we shouldn't set In_sync.
Actually we set In_sync in super_1_validate, we then clear it in add_bound_rdev
and reset it till recovery finishes. Set it till recovery finishes makes sense
to me, the super_1_validate is confusing.
> >
> > md_update_sb will update super and bitmap super at almost the same time. It's
> > possible we update super but not bitmap super and there is power loss. Is this
> > the case we have the both bits set? There seems no mechanism to prevent the
> > first disk has older bitmap super than the second disk in this scenario.
>
> This isn't really about the superblock and bitmap on the one device
> being in sync or not.
> It is the superblock on one (failed) device having a different event
> count than the bitmap (on the other devices) that is actively being used
> by the array.
My point is we could use a disk's superblock, but its bitmap superblock is
stale (even is old)
Anyway, I still didn't understand when the problem could happen. Maybe Guoqing
can clarify.
Thanks,
Shaohua
^ permalink raw reply
* Re: [PATCH] fs: buffer: Modify alloc_page_buffers.
From: Sean Fu @ 2017-06-21 17:07 UTC (permalink / raw)
To: Al Viro
Cc: shli, anton, jack, axboe, ebiggers, rpeterso, bmarzins,
linux-raid, linux-kernel, linux-fsdevel, linux-ntfs-dev
In-Reply-To: <20170619160315.GD10672@ZenIV.linux.org.uk>
[-- Attachment #1: Type: text/plain, Size: 1926 bytes --]
On Mon, Jun 19, 2017 at 05:03:16PM +0100, Al Viro wrote:
> On Mon, Jun 19, 2017 at 09:01:36PM +0800, Sean Fu wrote:
> > Make alloc_page_buffers support circular buffer list and initialise
> > b_state field.
> > Optimize the performance by removing the buffer list traversal to create
> > circular buffer list.
>
> > - bh = head = alloc_page_buffers(page, blocksize, 1);
> > + bh = head = alloc_page_buffers(page, blocksize, 1, 0, 0);
>
> Frankly, I don't like that change of calling conventions; it's very easy to
> mess the order of arguments when using interfaces like that and it's hell
> to find when trying to debug the resulting mess.
>
> Do you really get an observable change in performance? What loads are
> triggering it?
Yes, I have got the performance change with ext3 file system which block
size is 1024 bytes. It has at least %5 performance improvement.
I have found the performance improvements when writting/reading a 800M
size of file on ext3 file system with 1024 block size.
In this case, Each page has four buffer. In other word, the buffer list
has 4 elements.
I have compared the time that the process spent in kernel mode.
Improvements via this patch
Before After
Write:
0m5.604s 0m4.116s
0m4.408s 0m3.924s
0m4.184s 0m3.708s
0m4.352s 0m3.656s
0m4.380s 0m3.608s
0m4.240s 0m3.612s
0m4.460s 0m3.552s
0m4.072s 0m3.832s
0m4.300s 0m3.736s
0m4.400s 0m3.480s
Read:
0m3.128s 0m3.036s
0m2.976s 0m2.568s
0m3.384s 0m2.544s
0m3.112s 0m2.752s
0m2.924s 0m2.684s
0m3.084s 0m2.856s
0m3.348s 0m2.576s
0m3.000s 0m2.968s
0m3.012s 0m2.560s
0m2.768s 0m2.752s
Reproduce steps:
1 mkfs.ext3 -b 1024 /dev/sdb1
2 ./test_write.sh ./writetest 10
Test shell script:
#!/bin/bash
i=$2;
while test $((i)) -ge 1; do
mount /dev/sdb1 /mnt/sdb1/
time $1 -b 800 -o /mnt/sdb1/fileout
rm /mnt/sdb1/fileout
sync
sleep 1
umount /mnt/sdb1/
echo "aaa"
i=$i-1
done
The attachment are the code for writetest and test result.
[-- Attachment #2: writetest.c --]
[-- Type: text/x-c, Size: 7610 bytes --]
/*
*
* Copyright (c) CHANG Industry, Inc., 2004
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of the GNU General Public License as published by
* the Free Software Foundation; either version 2 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See
* the GNU General Public License for more details.
*
* You should have received a copy of the GNU General Public License
* along with this program; if not, write to the Free Software
* Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
*/
/*
* FILE : writetest.c
* DESCRIPTION : The purpose of this test is to verify that writes to
* disk occur without corruption. It writes one 1MB
* buffer at a time, where each byte in the buffer is
* generated from a random number. Once done
* completed, the file is re-opened, the random number
* generator is re-seeded, and the file is verified.
*
* HISTORY:
* 05/12/2004 : Written by Danny Sung <dannys@changind.com> to
* verify integrity of disk writes.
*
*/
#include <fcntl.h>
#include <getopt.h>
#include <stdarg.h>
#include <stdint.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <time.h>
#include <unistd.h>
#include <sys/types.h>
#include <sys/stat.h>
#include <fcntl.h>
//#include "test.h"
#define BLOCKSIZE (1024*1024)
#define FILE_OUT "fileout"
#define FILE_MODE 0644
#define MAX_FILENAME_LEN 1024
#define tst_resm(prefix, fmt, args...) \
do { \
printf(fmt, ##args); \
} while (0)
int Verbosity = 0;
int DefaultSeed = 0;
char Filename[MAX_FILENAME_LEN] = FILE_OUT;
off_t NumBlocks = 1;
char *TCID = "writetest";
int TST_TOTAL = 2;
void buf_init(void)
{
static int seed = 0;
if (seed == 0)
seed = DefaultSeed;
srand(seed);
}
void buf_fill(uint8_t * buf)
{
int i;
for (i = 0; i < BLOCKSIZE; i++) {
*buf = (rand() & 0xff);
buf++;
}
}
int write_file(off_t num_blocks, const char *filename)
{
int fd;
int ret = 0;
off_t block;
uint8_t buf[BLOCKSIZE];
fd = open(filename, O_RDWR | O_CREAT | O_TRUNC/* | O_LARGEFILE*/,
FILE_MODE);
if (fd < 0) {
perror(TCID);
return (-1);
}
for (block = 0; block < num_blocks; block++) {
int rv;
if (Verbosity > 2)
tst_resm(TINFO, "Block: %lld/%lld (%3lld%%)\r",
(long long int)block,
(long long int)num_blocks,
(long long int)(block * 100 / num_blocks));
buf_fill(buf);
rv = write(fd, buf, BLOCKSIZE);
if (rv != BLOCKSIZE) {
ret = -1;
break;
}
}
if (Verbosity > 2)
tst_resm(TINFO, "Block: %lld/%lld (%3lld%%)\r",
(long long int)block, (long long int)num_blocks,
(long long int)(block * 100 / num_blocks));
close(fd);
return (ret);
}
int read_file(off_t num_blocks, const char *filename)
{
int fd;
int ret = 0;
off_t block;
uint8_t buf_actual[BLOCKSIZE];
char buf_read[BLOCKSIZE];
fd = open(filename, O_RDONLY);
if (fd < 0) {
perror(TCID);
return (-1);
}
for (block = 0; block < num_blocks; block++) {
int rv;
int n;
if (Verbosity > 2)
tst_resm(TINFO, "Block: %lld/%lld (%3lld%%)\r",
(long long int)block,
(long long int)num_blocks,
(long long int)(block * 100 / num_blocks));
buf_fill(buf_actual);
rv = read(fd, buf_read, BLOCKSIZE);
if (rv != BLOCKSIZE) {
ret = -1;
break;
}
}
close(fd);
if (Verbosity > 2)
tst_resm(TINFO, "Block: %lld/%lld (%3lld%%)\r",
(long long int)block, (long long int)num_blocks,
(long long int)(block * 100 / num_blocks));
return (ret);
}
int verify_file(off_t num_blocks, const char *filename)
{
int fd;
int ret = 0;
off_t block;
uint8_t buf_actual[BLOCKSIZE];
char buf_read[BLOCKSIZE];
fd = open(filename, O_RDONLY);
if (fd < 0) {
perror(TCID);
return (-1);
}
for (block = 0; block < num_blocks; block++) {
int rv;
int n;
if (Verbosity > 2)
tst_resm(TINFO, "Block: %lld/%lld (%3lld%%)\r",
(long long int)block,
(long long int)num_blocks,
(long long int)(block * 100 / num_blocks));
buf_fill(buf_actual);
rv = read(fd, buf_read, BLOCKSIZE);
if (rv != BLOCKSIZE) {
ret = -1;
break;
}
for (n = 0; n < BLOCKSIZE; n++) {
int ba, br;
ba = buf_actual[n] & 0xff;
br = buf_read[n] & 0xff;
if (ba != br) {
if (Verbosity > 2)
tst_resm(TINFO,
"Mismatch: block=%lld +%d bytes offset=%lld read: %02xh actual: %02xh\n",
(long long int)block, n,
(long long
int)((block * BLOCKSIZE) + n),
br, ba);
ret++;
}
}
}
close(fd);
if (Verbosity > 2)
tst_resm(TINFO, "Block: %lld/%lld (%3lld%%)\r",
(long long int)block, (long long int)num_blocks,
(long long int)(block * 100 / num_blocks));
return (ret);
}
void usage(void)
{
printf("%s [-v] [-b blocks] [-s seed] [-o filename]\n", TCID);
printf("\n"
" -v - increase verbosity level\n"
" blocks - number of blocks to write\n"
" seed - seed to use (0 to use timestamp)\n"
" filename - name of output file\n");
}
void parse_args(int argc, char **argv)
{
int c;
TCID = argv[0];
while (1) {
int option_index = 0;
static struct option long_options[] = {
{"blocks", 1, 0, 'b'},
{"out", 1, 0, 'o'},
{"seed", 1, 0, 's'},
{"verbose", 0, 0, 'v'},
{"help", 0, 0, 'h'},
{0, 0, 0, 0}
};
c = getopt_long(argc, argv, "hvb:o:s:", long_options,
&option_index);
if (c == -1)
break;
switch (c) {
case 'b':
NumBlocks = strtoul(optarg, NULL, 0);
break;
case 'o':
strncpy(Filename, optarg, MAX_FILENAME_LEN);
break;
case 's':
DefaultSeed = strtoul(optarg, NULL, 0);
break;
case 'v':
Verbosity++;
break;
case 'h':
default:
usage();
exit(-1);
}
}
}
/*
void setup()
{
tst_tmpdir();
}
void cleanup(void)
{
tst_rmdir();
tst_exit();
}
*/
int main(int argc, char *argv[])
{
int rv;
// setup();
DefaultSeed = time(NULL);
parse_args(argc, argv);
tst_resm(TINFO, "Blocks: %lld\n", (long long int)NumBlocks);
tst_resm(TINFO, "Seed: %d\n", DefaultSeed);
tst_resm(TINFO, "Output file: '%s'\n", Filename);
tst_resm(TINFO, "Writing %lld blocks of %d bytes to '%s'\n",
(long long int)NumBlocks, BLOCKSIZE, Filename);
buf_init();
rv = write_file(NumBlocks, Filename);
if (rv == 0) {
tst_resm(TPASS, "Write: Success");
} else {
tst_resm(TFAIL, "Write: Failure");
}
/*
rv = read_file(NumBlocks, Filename);
if (rv == 0) {
tst_resm(TPASS, "Read: Success");
} else {
tst_resm(TFAIL, "Read: Failure");
}
*/
/*
tst_resm(TINFO, "Verifying %lld blocks in '%s'\n",
(long long int)NumBlocks, Filename);
buf_init();
rv = verify_file(NumBlocks, Filename);
if (rv == 0) {
tst_resm(TPASS, "Verify: Success\n");
} else {
tst_resm(TFAIL, "Verify: Failure");
tst_resm(TINFO, "Total mismatches: %d bytes\n", rv);
}
cleanup();
*/
return 0;
}
[-- Attachment #3: patched_data.txt --]
[-- Type: text/plain, Size: 4202 bytes --]
patched kernel：
============================================================================================================================
linux-g6ih:/home/sean/vfs_test # ./test_write.sh ./writetest 10
Blocks: 800
Seed: 1498036495
Output file: '/mnt/sdb1/fileout'
Writing 800 blocks of 1048576 bytes to '/mnt/sdb1/fileout'
Write: Success
real 0m18.466s
user 0m13.581s
sys 0m4.116s
aaa
Blocks: 800
Seed: 1498036515
Output file: '/mnt/sdb1/fileout'
Writing 800 blocks of 1048576 bytes to '/mnt/sdb1/fileout'
Write: Success
real 0m16.188s
user 0m12.013s
sys 0m3.924s
aaa
Blocks: 800
Seed: 1498036532
Output file: '/mnt/sdb1/fileout'
Writing 800 blocks of 1048576 bytes to '/mnt/sdb1/fileout'
Write: Success
real 0m15.700s
user 0m11.481s
sys 0m3.708s
aaa
Blocks: 800
Seed: 1498036549
Output file: '/mnt/sdb1/fileout'
Writing 800 blocks of 1048576 bytes to '/mnt/sdb1/fileout'
Write: Success
real 0m15.284s
user 0m11.493s
sys 0m3.656s
aaa
Blocks: 800
Seed: 1498036567
Output file: '/mnt/sdb1/fileout'
Writing 800 blocks of 1048576 bytes to '/mnt/sdb1/fileout'
Write: Success
real 0m16.049s
user 0m11.933s
sys 0m3.608s
aaa
Blocks: 800
Seed: 1498036585
Output file: '/mnt/sdb1/fileout'
Writing 800 blocks of 1048576 bytes to '/mnt/sdb1/fileout'
Write: Success
real 0m16.489s
user 0m12.441s
sys 0m3.612s
aaa
Blocks: 800
Seed: 1498036603
Output file: '/mnt/sdb1/fileout'
Writing 800 blocks of 1048576 bytes to '/mnt/sdb1/fileout'
Write: Success
real 0m15.466s
user 0m11.605s
sys 0m3.552s
aaa
Blocks: 800
Seed: 1498036620
Output file: '/mnt/sdb1/fileout'
Writing 800 blocks of 1048576 bytes to '/mnt/sdb1/fileout'
Write: Success
real 0m16.381s
user 0m11.749s
sys 0m3.832s
aaa
Blocks: 800
Seed: 1498036638
Output file: '/mnt/sdb1/fileout'
Writing 800 blocks of 1048576 bytes to '/mnt/sdb1/fileout'
Write: Success
real 0m16.163s
user 0m12.241s
sys 0m3.736s
aaa
Blocks: 800
Seed: 1498036655
Output file: '/mnt/sdb1/fileout'
Writing 800 blocks of 1048576 bytes to '/mnt/sdb1/fileout'
Write: Success
real 0m15.523s
user 0m11.537s
sys 0m3.480s
aaa
linux-g6ih:/home/sean/vfs_test # ./test_write.sh ./readtest 10
Blocks: 800
Seed: 1498036833
Output file: '/mnt/sdb1/fileout'
Writing 800 blocks of 1048576 bytes to '/mnt/sdb1/fileout'
Read: Success
real 0m14.719s
user 0m10.305s
sys 0m3.036s
aaa
Blocks: 800
Seed: 1498036849
Output file: '/mnt/sdb1/fileout'
Writing 800 blocks of 1048576 bytes to '/mnt/sdb1/fileout'
Read: Success
real 0m13.756s
user 0m10.933s
sys 0m2.568s
aaa
Blocks: 800
Seed: 1498036864
Output file: '/mnt/sdb1/fileout'
Writing 800 blocks of 1048576 bytes to '/mnt/sdb1/fileout'
Read: Success
real 0m13.743s
user 0m10.841s
sys 0m2.544s
aaa
Blocks: 800
Seed: 1498036879
Output file: '/mnt/sdb1/fileout'
Writing 800 blocks of 1048576 bytes to '/mnt/sdb1/fileout'
Read: Success
real 0m14.808s
user 0m11.545s
sys 0m2.752s
aaa
Blocks: 800
Seed: 1498036895
Output file: '/mnt/sdb1/fileout'
Writing 800 blocks of 1048576 bytes to '/mnt/sdb1/fileout'
Read: Success
real 0m13.863s
user 0m10.777s
sys 0m2.684s
aaa
Blocks: 800
Seed: 1498036910
Output file: '/mnt/sdb1/fileout'
Writing 800 blocks of 1048576 bytes to '/mnt/sdb1/fileout'
Read: Success
real 0m13.377s
user 0m10.301s
sys 0m2.856s
aaa
Blocks: 800
Seed: 1498036925
Output file: '/mnt/sdb1/fileout'
Writing 800 blocks of 1048576 bytes to '/mnt/sdb1/fileout'
Read: Success
real 0m13.394s
user 0m10.465s
sys 0m2.576s
aaa
Blocks: 800
Seed: 1498036939
Output file: '/mnt/sdb1/fileout'
Writing 800 blocks of 1048576 bytes to '/mnt/sdb1/fileout'
Read: Success
real 0m13.645s
user 0m10.457s
sys 0m2.968s
aaa
Blocks: 800
Seed: 1498036954
Output file: '/mnt/sdb1/fileout'
Writing 800 blocks of 1048576 bytes to '/mnt/sdb1/fileout'
Read: Success
real 0m13.420s
user 0m10.297s
sys 0m2.560s
aaa
Blocks: 800
Seed: 1498036968
Output file: '/mnt/sdb1/fileout'
Writing 800 blocks of 1048576 bytes to '/mnt/sdb1/fileout'
Read: Success
real 0m13.394s
user 0m10.345s
sys 0m2.752s
aaa
[-- Attachment #4: no_patched_data.txt --]
[-- Type: text/plain, Size: 4059 bytes --]
linux-g6ih:/home/sean/vfs_test # ./test_write.sh ./writetest 10
Blocks: 800
Seed: 1498027939
Output file: '/mnt/sdb1/fileout'
Writing 800 blocks of 1048576 bytes to '/mnt/sdb1/fileout'
Write: Success
real 0m19.816s
user 0m10.381s
sys 0m5.604s
aaa
Blocks: 800
Seed: 1498027962
Output file: '/mnt/sdb1/fileout'
Writing 800 blocks of 1048576 bytes to '/mnt/sdb1/fileout'
Write: Success
real 0m16.153s
user 0m11.237s
sys 0m4.408s
aaa
Blocks: 800
Seed: 1498027980
Output file: '/mnt/sdb1/fileout'
Writing 800 blocks of 1048576 bytes to '/mnt/sdb1/fileout'
Write: Success
real 0m16.226s
user 0m11.157s
sys 0m4.184s
aaa
Blocks: 800
Seed: 1498027998
Output file: '/mnt/sdb1/fileout'
Writing 800 blocks of 1048576 bytes to '/mnt/sdb1/fileout'
Write: Success
real 0m17.178s
user 0m10.965s
sys 0m4.352s
aaa
Blocks: 800
Seed: 1498028018
Output file: '/mnt/sdb1/fileout'
Writing 800 blocks of 1048576 bytes to '/mnt/sdb1/fileout'
Write: Success
real 0m16.454s
user 0m11.461s
sys 0m4.380s
aaa
Blocks: 800
Seed: 1498028036
Output file: '/mnt/sdb1/fileout'
Writing 800 blocks of 1048576 bytes to '/mnt/sdb1/fileout'
Write: Success
real 0m16.356s
user 0m11.349s
sys 0m4.240s
aaa
Blocks: 800
Seed: 1498028054
Output file: '/mnt/sdb1/fileout'
Writing 800 blocks of 1048576 bytes to '/mnt/sdb1/fileout'
Write: Success
real 0m16.450s
user 0m10.905s
sys 0m4.460s
aaa
Blocks: 800
Seed: 1498028072
Output file: '/mnt/sdb1/fileout'
Writing 800 blocks of 1048576 bytes to '/mnt/sdb1/fileout'
Write: Success
real 0m15.444s
user 0m10.877s
sys 0m4.072s
aaa
Blocks: 800
Seed: 1498028089
Output file: '/mnt/sdb1/fileout'
Writing 800 blocks of 1048576 bytes to '/mnt/sdb1/fileout'
Write: Success
real 0m15.619s
user 0m10.693s
sys 0m4.300s
aaa
Blocks: 800
Seed: 1498028106
Output file: '/mnt/sdb1/fileout'
Writing 800 blocks of 1048576 bytes to '/mnt/sdb1/fileout'
Write: Success
real 0m16.778s
user 0m11.669s
sys 0m4.400s
aaa
linux-g6ih:/home/sean/vfs_test # ./test_write.sh ./readtest 10
Blocks: 800
Seed: 1498028442
Output file: '/mnt/sdb1/fileout'
Writing 800 blocks of 1048576 bytes to '/mnt/sdb1/fileout'
Read: Success
real 0m15.799s
user 0m11.345s
sys 0m3.128s
aaa
Blocks: 800
Seed: 1498028459
Output file: '/mnt/sdb1/fileout'
Writing 800 blocks of 1048576 bytes to '/mnt/sdb1/fileout'
Read: Success
real 0m14.821s
user 0m11.589s
sys 0m2.976s
aaa
Blocks: 800
Seed: 1498028475
Output file: '/mnt/sdb1/fileout'
Writing 800 blocks of 1048576 bytes to '/mnt/sdb1/fileout'
Read: Success
real 0m15.164s
user 0m11.481s
sys 0m3.384s
aaa
Blocks: 800
Seed: 1498028491
Output file: '/mnt/sdb1/fileout'
Writing 800 blocks of 1048576 bytes to '/mnt/sdb1/fileout'
Read: Success
real 0m15.282s
user 0m11.753s
sys 0m3.112s
aaa
Blocks: 800
Seed: 1498028508
Output file: '/mnt/sdb1/fileout'
Writing 800 blocks of 1048576 bytes to '/mnt/sdb1/fileout'
Read: Success
real 0m15.064s
user 0m11.465s
sys 0m2.924s
aaa
Blocks: 800
Seed: 1498028524
Output file: '/mnt/sdb1/fileout'
Writing 800 blocks of 1048576 bytes to '/mnt/sdb1/fileout'
Read: Success
real 0m15.921s
user 0m12.497s
sys 0m3.084s
aaa
Blocks: 800
Seed: 1498028541
Output file: '/mnt/sdb1/fileout'
Writing 800 blocks of 1048576 bytes to '/mnt/sdb1/fileout'
Read: Success
real 0m15.663s
user 0m11.965s
sys 0m3.348s
aaa
Blocks: 800
Seed: 1498028558
Output file: '/mnt/sdb1/fileout'
Writing 800 blocks of 1048576 bytes to '/mnt/sdb1/fileout'
Read: Success
real 0m14.935s
user 0m11.657s
sys 0m3.000s
aaa
Blocks: 800
Seed: 1498028574
Output file: '/mnt/sdb1/fileout'
Writing 800 blocks of 1048576 bytes to '/mnt/sdb1/fileout'
Read: Success
real 0m15.061s
user 0m11.789s
sys 0m3.012s
aaa
Blocks: 800
Seed: 1498028590
Output file: '/mnt/sdb1/fileout'
Writing 800 blocks of 1048576 bytes to '/mnt/sdb1/fileout'
Read: Success
real 0m14.734s
user 0m11.729s
sys 0m2.768s
aaa
^ permalink raw reply
* Re: [PATCH] md: make suspend range wait timed out
From: Shaohua Li @ 2017-06-21 16:07 UTC (permalink / raw)
To: Mikulas Patocka; +Cc: NeilBrown, linux-raid, Shaohua Li
In-Reply-To: <alpine.LRH.2.02.1706211007560.29243@file01.intranet.prod.int.rdu2.redhat.com>
On Wed, Jun 21, 2017 at 10:09:08AM -0400, Mikulas Patocka wrote:
>
>
> On Mon, 19 Jun 2017, Shaohua Li wrote:
>
> > > Write errors only get back to the application if it calls fsync(), and
> > > many don't do that. Write errors can easily cause a filesystem to go
> > > read-only, and require an fsck. I think we should be very cautious
> > > about triggering write errors.
> > >
> > > NFS will hang indefinitely rather then return an error if the server is
> > > not available. That can certainly be annoying, but the alternative has
> > > been tried, and it leads to random data corruption.
> > > The two cases are only comparable at a very high level, but I think
> > > this result should encourage substantial caution.
> >
> > It's hard to say if an IO error or an infinite wait is better, but since there
> > is better option in this case, I don't want to argue. I'll repost a patch to
> > reset suspend range after a timeout, assume this is your suggestion.
> >
> > Thanks,
> > Shaohua
>
> Automatically resetting the suspend range could result in data corruption,
> so it is even worse than a deadlock.
depending on how you look at this. a deadlock means you will eventually hard
reset the system, and that will result in data corruption.
^ permalink raw reply
* Re: [PATCH] md: make suspend range wait timed out
From: Mikulas Patocka @ 2017-06-21 14:09 UTC (permalink / raw)
To: Shaohua Li; +Cc: NeilBrown, linux-raid, Shaohua Li
In-Reply-To: <20170620005443.hil23ffizavctgkq@kernel.org>
On Mon, 19 Jun 2017, Shaohua Li wrote:
> > Write errors only get back to the application if it calls fsync(), and
> > many don't do that. Write errors can easily cause a filesystem to go
> > read-only, and require an fsck. I think we should be very cautious
> > about triggering write errors.
> >
> > NFS will hang indefinitely rather then return an error if the server is
> > not available. That can certainly be annoying, but the alternative has
> > been tried, and it leads to random data corruption.
> > The two cases are only comparable at a very high level, but I think
> > this result should encourage substantial caution.
>
> It's hard to say if an IO error or an infinite wait is better, but since there
> is better option in this case, I don't want to argue. I'll repost a patch to
> reset suspend range after a timeout, assume this is your suggestion.
>
> Thanks,
> Shaohua
Automatically resetting the suspend range could result in data corruption,
so it is even worse than a deadlock.
Mikulas
^ permalink raw reply
* Re: 9 second recovery when re-adding a drive that got kicked out?
From: Nix @ 2017-06-21 11:08 UTC (permalink / raw)
To: NeilBrown; +Cc: Marc MERLIN, linux-raid@vger.kernel.org
In-Reply-To: <87efue2x06.fsf@notabene.neil.brown.name>
On 20 Jun 2017, NeilBrown verbalised:
> On Tue, Jun 20 2017, Marc MERLIN wrote:
>> The progress meter was wrong, it recovered in 3mn:
>> Jun 20 11:19:28 gargamel kernel: [ 916.007017] md: recovery of RAID array md8
>> Jun 20 11:22:41 gargamel kernel: [ 1108.395580] md: md8: recovery done.
>
> It is a progress bar - haven't you learned by now that they are *always*
> wrong :-)
I'd call this progress bar remarkably right by the standards of the
field -- it never jumps backwards :)
--
NULL && (void)
^ permalink raw reply
* Re: [PATCH] md/bitmap: don't read page from device with Bitmap_sync
From: Guoqing Jiang @ 2017-06-21 9:05 UTC (permalink / raw)
To: NeilBrown, Shaohua Li; +Cc: linux-raid
In-Reply-To: <87o9tj2tse.fsf@notabene.neil.brown.name>
On 06/20/2017 11:59 AM, NeilBrown wrote:
> On Mon, Jun 19 2017, Shaohua Li wrote:
>
>> On Mon, Jun 19, 2017 at 09:11:23AM +1000, Neil Brown wrote:
>>> On Fri, Jun 16 2017, Shaohua Li wrote:
>>>
>>>> On Fri, Jun 16, 2017 at 05:19:27PM +0800, Guoqing Jiang wrote:
>>>>> The device owns Bitmap_sync flag needs recovery
>>>>> to become in sync, and read page from this type
>>>>> device could get stale status.
>>>>>
>>>>> Signed-off-by: Guoqing Jiang <gqjiang@suse.com>
>>>>> ---
>>>>> When develop for clustered raid10 feature, if add a
>>>>> disk under grow mode in master node, I could get
>>>>> the "bitmap superblock UUID mismatch" warning due
>>>>> to the page is read from Bitmap_sync device.
>>>>>
>>>>> drivers/md/bitmap.c | 3 ++-
>>>>> 1 file changed, 2 insertions(+), 1 deletion(-)
>>>>>
>>>>> diff --git a/drivers/md/bitmap.c b/drivers/md/bitmap.c
>>>>> index bf7419a..bf34cd8 100644
>>>>> --- a/drivers/md/bitmap.c
>>>>> +++ b/drivers/md/bitmap.c
>>>>> @@ -156,7 +156,8 @@ static int read_sb_page(struct mddev *mddev, loff_t offset,
>>>>>
>>>>> rdev_for_each(rdev, mddev) {
>>>>> if (! test_bit(In_sync, &rdev->flags)
>>>> Why In_sync is set for the rdev? I think it shoudn't.
>>> There are several states a device can be in.
>>> If ->raid_disk is < 0, then the device is a spare, and doesn't contain
>>> and valid data.
>>> Otherwise ->raid_disk >=0 and:
>>> In_sync is clear, which means that blocks before ->recovery_offset
>>> contain valid data, and blocks after there don't
>>> In_sync is set, which means ->recovery_offset is irrelevant and
>>> should be treated as MaxSector
>>> Bitmap_sync is set, which could apply in either of the above cases,
>>> and means blocks corresponding to bits that are set in the bitmap
>>> may not be up to date.
>>>
>>> Bitmap_sync is only relevant before a device has been handed to the
>>> personality. After it has been added, ->recovery_cp ensures that the
>>> blocks that might be wrong are not read until until the bitmap-based
>>> recovery has fixed them up.
>>>
>>> Bitmap_sync is only use to stop the device from being given to the
>>> personality if a recovery won't be started because the array is
>>> read-only.
>>>
>>> So it is perfectly valid for both In_sync and Bitmap_sync to be set.
>>> If they are, it makes sense to avoid reading bitmap information from the
>>> Bitmap_sync device, as that will be out-of-date.
>>>
>>> I'm not quite sure why Guoqing is getting a UUID mismatch ... it
>>> suggests that the new device wasn't initialized properly. So there
>>> might be another bug. But I think this is definitely a bug.
>> Can you describe a scenario a disk has both bits set? I had hard time to
>> imagine it.
> An array with a bitmap has a device fail (cable break?).
> So the bitmap stops clearing bits and ->events_cleared stays
> where it was, with the same value as events in the superblock
> of the failed device.
>
> The device then starts working (cable is replaced) and
> it device is added back to the array.
> super_1_validate notices that the array has a bitmap and that the event
> count in the new device is >= mddev->bitmap->events_cleared, but <
> mddev->events.
> i.e. it is in the range for which a bitmap-based recovery is both
> possible and needed.
> So it sets Bitmap_sync.
> Then it notices that the 'role' in a valid role for the array,
> so that role is stored in rdev->saved_raid_disk and, as there is no
> MD_FEATURE_RECOVERY_OFFSET, In_sync is set.
> A similar sequence can happen in super_90_validate.
>
>> md_update_sb will update super and bitmap super at almost the same time. It's
>> possible we update super but not bitmap super and there is power loss. Is this
>> the case we have the both bits set? There seems no mechanism to prevent the
>> first disk has older bitmap super than the second disk in this scenario.
> This isn't really about the superblock and bitmap on the one device
> being in sync or not.
> It is the superblock on one (failed) device having a different event
> count than the bitmap (on the other devices) that is actively being used
> by the array.
>
>>
>>>>> - || test_bit(Faulty, &rdev->flags))
>>>>> + || test_bit(Faulty, &rdev->flags)
>>>>> + || test_bit(Bitmap_sync, &rdev->flags))
>>>> I didn't see code clears the Bitmap_sync bit after disk is synced, so likely
>>>> there is bug in this side.
>>> There is no need to clear Bitmap_sync. It stays set until it becomes
>>> irrelevant.
>> Hmm, if we use in this way, we should add comments there to declare this bit
>> can only be used very early.
> Certainly would be appropriate to document that it is only meaningful
> before the device has been passed to ->hot_add_disk().
Ok, I will send v2 version with the comment included.
Thanks,
Guoqing
^ permalink raw reply
* mdadm goes crazy after changing chunk size
From: d tbsky @ 2017-06-21 3:31 UTC (permalink / raw)
To: linux-raid
Hi:
I want to test the performance of different chunk size, so I
create a small raid (about 20G~200G) and use command like "mdadm
--grow -c 64 /dev/md2) to change the chunk size.
after changing chunk size, I found almost every time the raid can
not re-assemble after reboot.
the error message shows like "xxx does not have a valid v1.2
superblock, not importing!".
I found I can use "mdadm --assemble --update=devicesize ....." to
correct it. so I just continue my testing.
now testing is done, so I want to grow the small raid to full size,
but I am surprised that the " Used Dev Size" stocked at some stage and
can not grow to full size. maybe I miss some command parameter?
I think I need to re-create the raid. but maybe someone is
interested to see what happened. my environment is rhel 7.3 (but
redhat backport kernel 4.x softwareraid stack to their 3.10 kernel).
I have several testing raidset. below is a 4disk raid6:
==========================================================================================
command "mdadm --detail /dev/md1":
/dev/md1:
Version : 1.2
Creation Time : Mon Jun 5 17:58:52 2017
Raid Level : raid6
Array Size : 4116416128 (3925.72 GiB 4215.21 GB)
Used Dev Size : 2058208064 (1962.86 GiB 2107.61 GB)
Raid Devices : 4
Total Devices : 4
Persistence : Superblock is persistent
Update Time : Wed Jun 21 11:06:32 2017
State : clean
Active Devices : 4
Working Devices : 4
Failed Devices : 0
Spare Devices : 0
Layout : left-symmetric
Chunk Size : 64K
Name : localhost.localdomain:pv00
UUID : cc6a6b68:3d066e91:8bac3ba0:96448f78
Events : 8180
Number Major Minor RaidDevice State
0 8 3 0 active sync /dev/sda3
1 8 19 1 active sync /dev/sdb3
5 8 51 2 active sync /dev/sdd3
4 8 35 3 active sync /dev/sdc3
==========================================================================================
==========================================================================================
command "fdisk -lu /dev/sda /dev/sdb /dev/sdc /dev/sdd":
Disk /dev/sda: 4000.8 GB, 4000787030016 bytes, 7814037168 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes
Disk label type: gpt
# Start End Size Type Name
1 2048 4095 1M BIOS boot parti
2 4096 620543 301M Linux RAID
3 620544 7814037134 3.7T Linux RAID
Disk /dev/sdb: 4000.8 GB, 4000787030016 bytes, 7814037168 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes
Disk label type: gpt
# Start End Size Type Name
1 2048 4095 1M BIOS boot parti
2 4096 620543 301M Linux RAID
3 620544 7814037134 3.7T Linux RAID
Disk /dev/sdc: 4000.8 GB, 4000787030016 bytes, 7814037168 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes
Disk label type: gpt
# Start End Size Type Name
1 2048 4095 1M BIOS boot parti
2 4096 620543 301M Linux RAID
3 620544 7814037134 3.7T Linux RAID
Disk /dev/sdd: 4000.8 GB, 4000787030016 bytes, 7814037168 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes
Disk label type: gpt
# Start End Size Type Name
1 2048 4095 1M BIOS boot parti
2 4096 620543 301M Linux RAID
3 620544 7814037134 3.7T Linux RAID
==========================================================================================
==========================================================================================
command "mdadm --grow --size=max /dev/md1":
mdadm: component size of /dev/md1 unchanged at 2058208064K
==========================================================================================
another raid5 is more strange. it assemble correctly after change
chunk size (this is unusual in my testing environment without
"--update=devicesize").
the strange part is the md2 huge bitmap chunk:
==========================================================================================
command "cat /proc/mdstat"
Personalities : [raid1] [raid6] [raid5] [raid4]
md2 : active raid5 sde1[0] sdf1[1] sdg1[3]
5263812224 blocks super 1.2 level 5, 64k chunk, algorithm 2 [3/3] [UUU]
bitmap: 0/1 pages [0KB], 18014398507384832KB chunk
md0 : active raid1 sdc2[4] sdb2[1] sda2[0] sdd2[5]
308160 blocks super 1.0 [4/4] [UUUU]
bitmap: 0/1 pages [0KB], 65536KB chunk
md1 : active raid6 sdd3[5] sdb3[1] sdc3[4] sda3[0]
4116416128 blocks super 1.2 level 6, 64k chunk, algorithm 2 [4/4] [UUUU]
unused devices: <none>
==========================================================================================
the raid5 device also can not grow to full size:
==========================================================================================
command "mdadm --detail /dev/md2"
/dev/md2:
Version : 1.2
Creation Time : Tue Jun 13 15:21:32 2017
Raid Level : raid5
Array Size : 5263812224 (5019.96 GiB 5390.14 GB)
Used Dev Size : 2631906112 (2509.98 GiB 2695.07 GB)
Raid Devices : 3
Total Devices : 3
Persistence : Superblock is persistent
Intent Bitmap : Internal
Update Time : Wed Jun 21 10:45:39 2017
State : clean
Active Devices : 3
Working Devices : 3
Failed Devices : 0
Spare Devices : 0
Layout : left-symmetric
Chunk Size : 64K
Name : love-1:3 (local to host love-1)
UUID : 5b2c25fc:b4ccc860:ba8685fe:5e0433f7
Events : 5176
Number Major Minor RaidDevice State
0 8 65 0 active sync /dev/sde1
1 8 81 1 active sync /dev/sdf1
3 8 97 2 active sync /dev/sdg1
==========================================================================================
==========================================================================================
command "fdisk -lu /dev/sde /dev/sdf /dev/sdg":
Disk /dev/sde: 4000.8 GB, 4000787030016 bytes, 7814037168 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes
Disk label type: gpt
# Start End Size Type Name
1 2048 7814037134 3.7T Linux RAID
Disk /dev/sdf: 4000.8 GB, 4000787030016 bytes, 7814037168 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes
Disk label type: gpt
# Start End Size Type Name
1 2048 7814037134 3.7T Linux RAID
Disk /dev/sdg: 4000.8 GB, 4000787030016 bytes, 7814037168 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes
Disk label type: gpt
# Start End Size Type Name
1 2048 7814037134 3.7T Linux RAID
==========================================================================================
==========================================================================================
command "mdadm --grow --size=max /dev/md2"
mdadm: component size of /dev/md2 unchanged at 2631906112K
==========================================================================================
^ permalink raw reply
* [PATCH] md: use a separate bio_set for synchronous IO.
From: NeilBrown @ 2017-06-20 23:12 UTC (permalink / raw)
To: Shaohua Li; +Cc: linux-raid
[-- Attachment #1: Type: text/plain, Size: 4398 bytes --]
md devices allocate a bio_set and use it for two
distinct purposes.
mddev->bio_set is used to clone bios as part of sending
upper level requests down to lower level devices,
and it is also use for synchronous IO such as superblock
and bitmap updates, and for correcting read errors.
This multiple usage can lead to deadlocks. It is likely
that cloned bios might be queued for write and to be
waiting for a metadata update before the write can be permitted.
If the cloning exhausted mddev->bio_set, the metadata update
may not be able to proceed.
This scenario has been seen during heavy testing, with lots of IO and
lots of memory pressure.
Address this by adding a new bio_set specifically for synchronous IO.
All synchronous IO goes directly to the underlying device and is not
queued at the md level, so request using entries from the new
mddev->sync_set will complete in a timely fashion.
Requests that use mddev->bio_set will sometimes need to wait
for synchronous IO, but will no longer risk deadlocking that iO.
Also: small simplification in mddev_put(): there is no need to
wait until the spinlock is released before calling bioset_free().
Signed-off-by: NeilBrown <neilb@suse.com>
---
drivers/md/md.c | 27 ++++++++++++++++++++-------
drivers/md/md.h | 3 +++
2 files changed, 23 insertions(+), 7 deletions(-)
diff --git a/drivers/md/md.c b/drivers/md/md.c
index 87edc342ccb3..9d0eb50c458e 100644
--- a/drivers/md/md.c
+++ b/drivers/md/md.c
@@ -203,6 +203,14 @@ struct bio *bio_alloc_mddev(gfp_t gfp_mask, int nr_iovecs,
}
EXPORT_SYMBOL_GPL(bio_alloc_mddev);
+static struct bio *md_bio_alloc_sync(struct mddev *mddev)
+{
+ if (!mddev->sync_set)
+ return bio_alloc(GFP_NOIO, 1);
+
+ return bio_alloc_bioset(GFP_NOIO, 1, mddev->sync_set);
+}
+
/*
* We have a system wide 'event count' that is incremented
* on any 'interesting' event, and readers of /proc/mdstat
@@ -462,8 +470,6 @@ static void mddev_delayed_delete(struct work_struct *ws);
static void mddev_put(struct mddev *mddev)
{
- struct bio_set *bs = NULL;
-
if (!atomic_dec_and_lock(&mddev->active, &all_mddevs_lock))
return;
if (!mddev->raid_disks && list_empty(&mddev->disks) &&
@@ -471,8 +477,12 @@ static void mddev_put(struct mddev *mddev)
/* Array is not configured at all, and not held active,
* so destroy it */
list_del_init(&mddev->all_mddevs);
- bs = mddev->bio_set;
+ if (mddev->bio_set)
+ bioset_free(mddev->bio_set);
+ if (mddev->sync_set)
+ bioset_free(mddev->sync_set);
mddev->bio_set = NULL;
+ mddev->sync_set = NULL;
if (mddev->gendisk) {
/* We did a probe so need to clean up. Call
* queue_work inside the spinlock so that
@@ -485,8 +495,6 @@ static void mddev_put(struct mddev *mddev)
kfree(mddev);
}
spin_unlock(&all_mddevs_lock);
- if (bs)
- bioset_free(bs);
}
static void md_safemode_timeout(unsigned long data);
@@ -751,7 +759,7 @@ void md_super_write(struct mddev *mddev, struct md_rdev *rdev,
if (test_bit(Faulty, &rdev->flags))
return;
- bio = bio_alloc_mddev(GFP_NOIO, 1, mddev);
+ bio = md_bio_alloc_sync(mddev);
atomic_inc(&rdev->nr_pending);
@@ -783,7 +791,7 @@ int md_super_wait(struct mddev *mddev)
int sync_page_io(struct md_rdev *rdev, sector_t sector, int size,
struct page *page, int op, int op_flags, bool metadata_op)
{
- struct bio *bio = bio_alloc_mddev(GFP_NOIO, 1, rdev->mddev);
+ struct bio *bio = md_bio_alloc_sync(rdev->mddev);
int ret;
bio->bi_bdev = (metadata_op && rdev->meta_bdev) ?
@@ -5432,6 +5440,11 @@ int md_run(struct mddev *mddev)
if (!mddev->bio_set)
return -ENOMEM;
}
+ if (mddev->sync_set == NULL) {
+ mddev->sync_set = bioset_create(BIO_POOL_SIZE, 0);
+ if (!mddev->sync_set)
+ return -ENOMEM;
+ }
spin_lock(&pers_lock);
pers = find_pers(mddev->level, mddev->clevel);
diff --git a/drivers/md/md.h b/drivers/md/md.h
index 0fa1de42c42b..a1c7aa89bb67 100644
--- a/drivers/md/md.h
+++ b/drivers/md/md.h
@@ -444,6 +444,9 @@ struct mddev {
struct attribute_group *to_remove;
struct bio_set *bio_set;
+ struct bio_set *sync_set; /* for sync operations like
+ * metadata and bitmap writes
+ */
/* Generic flush handling.
* The last to finish preflush schedules a worker to submit
--
2.12.2
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 832 bytes --]
^ permalink raw reply related
* Re: 9 second recovery when re-adding a drive that got kicked out?
From: Marc MERLIN @ 2017-06-20 21:32 UTC (permalink / raw)
To: NeilBrown; +Cc: linux-raid@vger.kernel.org
In-Reply-To: <87efue2x06.fsf@notabene.neil.brown.name>
[-- Attachment #1: Type: text/plain, Size: 1403 bytes --]
On Wed, Jun 21, 2017 at 07:02:33AM +1000, NeilBrown wrote:
> > The progress meter was wrong, it recovered in 3mn:
> > Jun 20 11:19:28 gargamel kernel: [ 916.007017] md: recovery of RAID array md8
> > Jun 20 11:22:41 gargamel kernel: [ 1108.395580] md: md8: recovery done.
>
> It is a progress bar - haven't you learned by now that they are *always*
> wrong :-)
>
> recovery always reports progress in sectors completed, and estimates
> time based on how many sectors were processes in the last 30 seconds,
> and how many are left.
>
> With a bitmap based recovery, most sectors are handled very quickly
> (instantly?), while some take milliseconds. That makes the estimate
> imprecise.
Indeed.
Sorry that I jumped the gun, I think it looked like it was going to take
hours, and then it completed a few seconds after I had sent the Email.
Sorry for the noise, and obviously it's awesome that it worked so quickly
again
(been dealing with some unstable underlying SAS card that tends to kill
drives if the cable isn't plugged in perfectly after a good amount of
compressed air sent to the connector and the cable's plug)
Marc
--
"A mouse is a device used to point at the xterm you want to type in" - A.S.R.
Microsoft is to operating systems ....
.... what McDonalds is to gourmet cooking
Home page: http://marc.merlins.org/
[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 291 bytes --]
^ permalink raw reply
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox