[dm-crypt] [charlesc@pyropus.ca: Re: Does luksFormat write the full size of the device?]

[dm-crypt] [charlesc@pyropus.ca: Re: Does luksFormat write the full size of the device?]

[Charles Cazabon]

This message of yesterday doesn't seem to have gone through to the list,
trying again...

Milan Broz <gmazyland@gmail.com> wrote:
> On 06/26/2016 05:39 AM, Charles Cazabon wrote:
> 
> No, format doesn't write the whole device.

Ok, thanks.  That explains why I don't remember it taking this long before.

> Can you please post output of running commant with added --debug switch?

Sure:

---------------------------
# cryptsetup --debug luksFormat /dev/md0
# cryptsetup 1.4.3 processing "cryptsetup --debug luksFormat /dev/md0"
# Running command luksFormat.
# Locking memory.

WARNING!
========
This will overwrite data on /dev/md0 irrevocably.

Are you sure? (Type uppercase yes): YES
# Allocating crypt device /dev/md0 context.
# Trying to open and read device /dev/md0.
# Initialising device-mapper backend, UDEV is enabled.
# Detected dm-crypt version 1.14.1, dm-ioctl version 4.34.0.
# Timeout set to 0 miliseconds.
# Iteration time set to 1000 miliseconds.
# Interactive passphrase entry requested.
Enter LUKS passphrase: 
Verify passphrase: 
# Formatting device /dev/md0 as type LUKS1.
# Crypto backend (gcrypt 1.5.0) initialized.
# Topology: IO (524288/3145728), offset = 0; Required alignment is 3145728
# bytes.
# Generating LUKS header version 1 using hash sha1, aes, cbc-essiv:sha256, MK
# 32 bytes
^C
---------------------------

I aborted it after a couple of minutes this time.

I've compiled 1.7.2 from the source tarball, and it does the same thing,
spinning on one CPU.  One difference is that it doesn't seem to respond to a
ctrl-C; it's still running after trying to abort it.  I haven't tried to kill
it with other signals yet.  Here's the output from it:

---------------------------
# ~xxxxxxx/software/cryptsetup-1.7.2/src/.libs/cryptsetup --debug luksFormat /dev/md0
# cryptsetup 1.7.2 processing
# "/home/xxxxxxx/software/cryptsetup-1.7.2/src/.libs/cryptsetup --debug luksFormat /dev/md0"
# Running command luksFormat.
# Locking memory.
# Installing SIGINT/SIGTERM handler.
# Unblocking interruption on signal.

WARNING!
========
This will overwrite data on /dev/md0 irrevocably.

Are you sure? (Type uppercase yes): YES
# Allocating crypt device /dev/md0 context.
# Trying to open and read device /dev/md0.
# Initialising device-mapper backend, UDEV is enabled.
# Detected dm-crypt version 1.14.1, dm-ioctl version 4.34.0.
# Timeout set to 0 miliseconds.
# Iteration time set to 2000 miliseconds.
# Interactive passphrase entry requested.
Enter passphrase: 
Verify passphrase: 
# Formatting device /dev/md0 as type LUKS1.
# Crypto backend (gcrypt 1.5.0) initialized.
# Topology: IO (524288/3145728), offset = 0; Required alignment is 3145728
# bytes.
# Generating LUKS header version 1 using hash sha256, aes, xts-plain64, MK 32
# bytes
---------------------------


> > Is this long runtime normal for initialization of a device of this size?
> 
> No. If the default iteration time (1 or 2 seconds) is not changes, format
> should be done in max 4-5 seconds.

Ok, then something is definitely still wrong here.  I'm guessing the newish
kernel is related?

Thanks,

Charles

-- 
------------------------------------------------------------------
Charles Cazabon                              <charlesc@pyropus.ca>
Software, consulting, and services available at http://pyropus.ca/
------------------------------------------------------------------

Re: [dm-crypt] luksFormat still spinning/hanging with large array on kernel 4.6.2

[Charles Cazabon]

Charles Cazabon <charlesc@pyropus.ca> wrote:
> 
> > Can you please post output of running commant with added --debug switch?
> 
> Sure:
> 
> ---------------------------
> # cryptsetup --debug luksFormat /dev/md0
> # cryptsetup 1.4.3 processing "cryptsetup --debug luksFormat /dev/md0"
> # Running command luksFormat.
> # Locking memory.
> 
> WARNING!
> ========
> This will overwrite data on /dev/md0 irrevocably.
> 
> Are you sure? (Type uppercase yes): YES
> # Allocating crypt device /dev/md0 context.
> # Trying to open and read device /dev/md0.
> # Initialising device-mapper backend, UDEV is enabled.
> # Detected dm-crypt version 1.14.1, dm-ioctl version 4.34.0.
> # Timeout set to 0 miliseconds.
> # Iteration time set to 1000 miliseconds.
> # Interactive passphrase entry requested.
> Enter LUKS passphrase: 
> Verify passphrase: 
> # Formatting device /dev/md0 as type LUKS1.
> # Crypto backend (gcrypt 1.5.0) initialized.
> # Topology: IO (524288/3145728), offset = 0; Required alignment is 3145728
> # bytes.
> # Generating LUKS header version 1 using hash sha1, aes, cbc-essiv:sha256, MK
> # 32 bytes
> ^C
> ---------------------------

I've since moved the array over to another machine, also running Wheezy with
the mainline 4.6.2 kernel.  cryptsetup luksFormat behaves the same there,
spinning forever and taking one full CPU.

So is this a known or new incompatibility between cryptsetup and the newer
kernel?  Or some other known issue?

Basically I'm wondering what I should be doing to get this working.  It *used*
to work, given that I have another identical array running on this particular
machine, and I did the cryptsetup luksFormat on it some six months ago with no
issues.  The only changes between then and now are that the kernel and other
system packages have been updated since.

Anyone have any advice for what to try here?  I'm willing to help compile with
debugging patches or whatever if that would help track down where the problem
lies.

Thanks,

Charles
> 
> I aborted it after a couple of minutes this time.
> 
> I've compiled 1.7.2 from the source tarball, and it does the same thing,
> spinning on one CPU.  One difference is that it doesn't seem to respond to a
> ctrl-C; it's still running after trying to abort it.  I haven't tried to kill
> it with other signals yet.  Here's the output from it:
> 
> ---------------------------
> # ~xxxxxxx/software/cryptsetup-1.7.2/src/.libs/cryptsetup --debug luksFormat /dev/md0
> # cryptsetup 1.7.2 processing
> # "/home/xxxxxxx/software/cryptsetup-1.7.2/src/.libs/cryptsetup --debug luksFormat /dev/md0"
> # Running command luksFormat.
> # Locking memory.
> # Installing SIGINT/SIGTERM handler.
> # Unblocking interruption on signal.
> 
> WARNING!
> ========
> This will overwrite data on /dev/md0 irrevocably.
> 
> Are you sure? (Type uppercase yes): YES
> # Allocating crypt device /dev/md0 context.
> # Trying to open and read device /dev/md0.
> # Initialising device-mapper backend, UDEV is enabled.
> # Detected dm-crypt version 1.14.1, dm-ioctl version 4.34.0.
> # Timeout set to 0 miliseconds.
> # Iteration time set to 2000 miliseconds.
> # Interactive passphrase entry requested.
> Enter passphrase: 
> Verify passphrase: 
> # Formatting device /dev/md0 as type LUKS1.
> # Crypto backend (gcrypt 1.5.0) initialized.
> # Topology: IO (524288/3145728), offset = 0; Required alignment is 3145728
> # bytes.
> # Generating LUKS header version 1 using hash sha256, aes, xts-plain64, MK 32
> # bytes
> ---------------------------
> 
> 
> > > Is this long runtime normal for initialization of a device of this size?
> > 
> > No. If the default iteration time (1 or 2 seconds) is not changes, format
> > should be done in max 4-5 seconds.
> 
> Ok, then something is definitely still wrong here.  I'm guessing the newish
> kernel is related?
> 
> Thanks,
> 
> Charles
> 
> -- 
> ------------------------------------------------------------------
> Charles Cazabon                              <charlesc@pyropus.ca>
> Software, consulting, and services available at http://pyropus.ca/
> ------------------------------------------------------------------
> _______________________________________________
> dm-crypt mailing list
> dm-crypt@saout.de
> http://www.saout.de/mailman/listinfo/dm-crypt
-- 
------------------------------------------------------------------
Charles Cazabon                              <charlesc@pyropus.ca>
Software, consulting, and services available at http://pyropus.ca/
------------------------------------------------------------------

Re: [dm-crypt] luksFormat still spinning/hanging with large array on kernel 4.6.2

[Milan Broz]

Hi,

thanks for the info.

On 06/28/2016 05:01 AM, Charles Cazabon wrote:
> I've since moved the array over to another machine, also running Wheezy with
> the mainline 4.6.2 kernel.  cryptsetup luksFormat behaves the same there,
> spinning forever and taking one full CPU.
> 
> So is this a known or new incompatibility between cryptsetup and the newer
> kernel?  Or some other known issue?

Not (yet). It is perhaps some recent incompatibility but we have to reproduce and
track to proper component.
 
> Basically I'm wondering what I should be doing to get this working.  It *used*
> to work, given that I have another identical array running on this particular
> machine, and I did the cryptsetup luksFormat on it some six months ago with no
> issues.  The only changes between then and now are that the kernel and other
> system packages have been updated since.
> 
> Anyone have any advice for what to try here?  I'm willing to help compile with
> debugging patches or whatever if that would help track down where the problem
> lies.

That would be great. Please if you can compile cryptsetup from the branch "bench" with
added print inside PBKDF2 benchmark (https://gitlab.com/cryptsetup/cryptsetup/tree/bench)

and run the command again with --debug switch and send me the output.

I need to verify that it is the same problem (if it is, then getrusage() call
return all zeroed values).

Thanks,
Milan

Re: [dm-crypt] luksFormat still spinning/hanging with large array on kernel 4.6.2

[Charles Cazabon]

Milan Broz <gmazyland@gmail.com> wrote:
> On 06/28/2016 05:01 AM, Charles Cazabon wrote:
> > I've since moved the array over to another machine, also running Wheezy with
> > the mainline 4.6.2 kernel.  cryptsetup luksFormat behaves the same there,
> > spinning forever and taking one full CPU.
[...]
> I need to verify that it is the same problem (if it is, then getrusage() call
> return all zeroed values).
[...]
> Just noticed this here - you are testing new cryptsetup binary but according
> to debug output it uses system old libcryptsetup library.
>
> Please use src/cryptsetup not directly src/.libs/cryptsetup ...

And this, indeed, fixed the problem.  luksFormat completed in normal time.
Sorry, I wasn't sure which of the built `cryptsetup` files was the one to use
(in fact, I had assumed without checking that they would be hardlinks to the
same file).

Thanks muchly for the help.  So whatever bug I hit is indeed the one you've
already fixed in newer versions.

Charles
-- 
------------------------------------------------------------------
Charles Cazabon                              <charlesc@pyropus.ca>
Software, consulting, and services available at http://pyropus.ca/
------------------------------------------------------------------