[dm-crypt] out of order encryption

[dm-crypt] out of order encryption

[Vasile Catalin-B50542]

Would dm-crypt execute correctly if sector encryption ended asynchronously?
For example:
If sector 1, 2, 3 are sent to be done asynchronously to the same 
algorithm instance,
and the jobs end in the following order: 2, 1, 3; does the dm-crypt 
module know to
write the data in the proper place when the encryption "done" callback 
is called?

Re: [dm-crypt] out of order encryption

[Milan Broz]

On 08/06/2015 09:00 AM, Vasile Catalin-B50542 wrote:
> Would dm-crypt execute correctly if sector encryption ended asynchronously?
> For example:
> If sector 1, 2, 3 are sent to be done asynchronously to the same 
> algorithm instance,
> and the jobs end in the following order: 2, 1, 3; does the dm-crypt 
> module know to
> write the data in the proper place when the encryption "done" callback 
> is called?

Not sure if I understand your question - encryption in dmcrypt works on sector
level, sectors are encrypted independetly, so order cannot influence result
of encryption of individual sectors.

Or if the question is about order of submitted requests:
If you need to ensure order of processing, you have to issue "flush" operation
before submitting next data content. Filesystems typically must do this when
handling journal or so.

In general, requests order can be rearranged (anywhere in block layer, not only in dmcrypt).
But this is correct behaviour.

Milan

Re: [dm-crypt] out of order encryption

[Vasile Catalin-B50542]

[-- Attachment #1: Type: text/plain, Size: 1750 bytes --]

I was referring to the submitted requests.
Does the underlying encryption layer (CryptoAPI) have to ensure the
complete callbacks are called in the order the requests were submitted?
Or does dm-crypt figure out where to read/write after request is done
no matter in which order the crypto requests finished?

On 06.08.2015 12:15, Milan Broz wrote:
> On 08/06/2015 09:00 AM, Vasile Catalin-B50542 wrote:
>> Would dm-crypt execute correctly if sector encryption ended asynchronously?
>> For example:
>> If sector 1, 2, 3 are sent to be done asynchronously to the same
>> algorithm instance,
>> and the jobs end in the following order: 2, 1, 3; does the dm-crypt
>> module know to
>> write the data in the proper place when the encryption "done" callback
>> is called?
> Not sure if I understand your question - encryption in dmcrypt works on sector
> level, sectors are encrypted independetly, so order cannot influence result
> of encryption of individual sectors.
>
> Or if the question is about order of submitted requests:
> If you need to ensure order of processing, you have to issue "flush" operation
> before submitting next data content. Filesystems typically must do this when
> handling journal or so.
>
> In general, requests order can be rearranged (anywhere in block layer, not only in dmcrypt).
> But this is correct behaviour.
>
> Milan

-- 

CatalinVasile

Intern, DN-Software

FreescaleSemiconductor, Inc.

www.freescale.com

phone: 073-021-1938

e-mail: catalin.vasile@freescale.com <mailto:your.name@freescale.com>

*Freescale_Logo-nosemi_Lh_4c***

This e-mail, and any associated attachments have been classified as:

[ ] Public

[ ] Freescale Semiconductor Internal Use Only

[ ] Freescale Semiconductor Confidential Proprietary


[-- Attachment #2.1: Type: text/html, Size: 29900 bytes --]

[-- Attachment #2.2: image002.gif --]
[-- Type: image/gif, Size: 1979 bytes --]

Re: [dm-crypt] out of order encryption

[Vasile Catalin-B50542]

I was referring to the submitted requests.
Does the underlying encryption layer (CryptoAPI) have to ensure the
complete callbacks are called in the order the requests were submitted?
Or does dm-crypt figure out where to read/write after request is done
no matter in which order the crypto requests finished?

On 06.08.2015 12:15, Milan Broz wrote:
> On 08/06/2015 09:00 AM, Vasile Catalin-B50542 wrote:
>> Would dm-crypt execute correctly if sector encryption ended asynchronously?
>> For example:
>> If sector 1, 2, 3 are sent to be done asynchronously to the same
>> algorithm instance,
>> and the jobs end in the following order: 2, 1, 3; does the dm-crypt
>> module know to
>> write the data in the proper place when the encryption "done" callback
>> is called?
> Not sure if I understand your question - encryption in dmcrypt works on sector
> level, sectors are encrypted independetly, so order cannot influence result
> of encryption of individual sectors.
>
> Or if the question is about order of submitted requests:
> If you need to ensure order of processing, you have to issue "flush" operation
> before submitting next data content. Filesystems typically must do this when
> handling journal or so.
>
> In general, requests order can be rearranged (anywhere in block layer, not only in dmcrypt).
> But this is correct behaviour.
>
> Milan

Re: [dm-crypt] out of order encryption

[Milan Broz]

On 08/06/2015 12:38 PM, Vasile Catalin-B50542 wrote:
> I was referring to the submitted requests.
> Does the underlying encryption layer (CryptoAPI) have to ensure the
> complete callbacks are called in the order the requests were submitted?

No. And it really does not matter, requests are independent.

> Or does dm-crypt figure out where to read/write after request is done
> no matter in which order the crypto requests finished?

In recent kernels write request are in dmcrypt partially sorted for performance reasons
http://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/commit/drivers/md/dm-crypt.c?id=b3c5fd3052492f1b8d060799d4f18be5a5438add

Otherwise there is no other logic related.

m.


> 
> On 06.08.2015 12:15, Milan Broz wrote:
>> On 08/06/2015 09:00 AM, Vasile Catalin-B50542 wrote:
>>> Would dm-crypt execute correctly if sector encryption ended asynchronously?
>>> For example:
>>> If sector 1, 2, 3 are sent to be done asynchronously to the same
>>> algorithm instance,
>>> and the jobs end in the following order: 2, 1, 3; does the dm-crypt
>>> module know to
>>> write the data in the proper place when the encryption "done" callback
>>> is called?
>> Not sure if I understand your question - encryption in dmcrypt works on sector
>> level, sectors are encrypted independetly, so order cannot influence result
>> of encryption of individual sectors.
>>
>> Or if the question is about order of submitted requests:
>> If you need to ensure order of processing, you have to issue "flush" operation
>> before submitting next data content. Filesystems typically must do this when
>> handling journal or so.
>>
>> In general, requests order can be rearranged (anywhere in block layer, not only in dmcrypt).
>> But this is correct behaviour.
>>
>> Milan
> 
> _______________________________________________
> dm-crypt mailing list
> dm-crypt@saout.de
> http://www.saout.de/mailman/listinfo/dm-crypt
> 

Re: [dm-crypt] out of order encryption

[Lars Winterfeld]

[-- Attachment #1: Type: text/plain, Size: 273 bytes --]

Am 06.08.2015 um 12:38 schrieb Vasile Catalin-B50542:
> Does the underlying encryption layer (CryptoAPI) have to ensure the
> complete callbacks are called in the order the requests were submitted?

Are you thinking about parallelization or asynchronous processing?


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

Re: [dm-crypt] out of order encryption

[Vasile Catalin-B50542]

[-- Attachment #1: Type: text/plain, Size: 763 bytes --]

I am sending crypto requests to a hardware component which has
an option to keeping them in order or not. I was trying to use the out of
order option because this would let requests run in parallel.
If I tried running them with the out of order option bun then I get:
No key available with this passphrase.
When trying to execute the following commands:
cryptsetup -c aes-xts-plain -y -v luksFormat /dev/sda1
cryptsetup open /dev/sda1 test_xts1

On 06.08.2015 14:19, Lars Winterfeld wrote:
> Am 06.08.2015 um 12:38 schrieb Vasile Catalin-B50542:
>> Does the underlying encryption layer (CryptoAPI) have to ensure the
>> complete callbacks are called in the order the requests were submitted?
> Are you thinking about parallelization or asynchronous processing?
>

[-- Attachment #2: Type: text/html, Size: 1448 bytes --]

Re: [dm-crypt] out of order encryption

[Lars Winterfeld]

[-- Attachment #1: Type: text/plain, Size: 1403 bytes --]

As far as I understand, your commands are used to set up the crypto
mapper. They should be executed previous to any read/write operations. I
think Milan was talking about read/write operations that are executed
after the crypto mapper has been set up successfully.

Btw... I guess that if two write operations result in two different
changes in the same sector, there are no write conflicts, because for
each write, the current sector contents is decrypted first, changes are
applies, encrypted and written back. Is that correct?


Am 06.08.2015 um 13:32 schrieb Vasile Catalin-B50542:
> I am sending crypto requests to a hardware component which has
> an option to keeping them in order or not. I was trying to use the out of
> order option because this would let requests run in parallel.
> If I tried running them with the out of order option bun then I get:
> No key available with this passphrase.
> When trying to execute the following commands:
> cryptsetup -c aes-xts-plain -y -v luksFormat /dev/sda1
> cryptsetup open /dev/sda1 test_xts1
> 
> On 06.08.2015 14:19, Lars Winterfeld wrote:
>> Am 06.08.2015 um 12:38 schrieb Vasile Catalin-B50542:
>>> Does the underlying encryption layer (CryptoAPI) have to ensure the
>>> complete callbacks are called in the order the requests were submitted?
>> Are you thinking about parallelization or asynchronous processing?
>>


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

Re: [dm-crypt] out of order encryption

[Milan Broz]

On 08/06/2015 01:32 PM, Vasile Catalin-B50542 wrote:
> I am sending crypto requests to a hardware component which has
> an option to keeping them in order or not. I was trying to use the out of
> order option because this would let requests run in parallel.
> If I tried running them with the out of order option bun then I get:
> No key available with this passphrase.
> When trying to execute the following commands:
> cryptsetup -c aes-xts-plain -y -v luksFormat /dev/sda1
> cryptsetup open /dev/sda1 test_xts1


Sigh. Please when you asking about some explicit problem, please
always mention it from the beginning...

So this is probably unrelated to dmcrypt. Userspace cryptsetup could use
kernel crypto API wrapper directly (add --debug and you will see,
there should be log message that it uses kernel crypto userspace interface).

This seems like a bug in the kernel crypto API provider, what exactly it is?
What architecture?

We have seen the same problem with Raspbery PI2 & NEON crypto driver
(it was a bug in kernel code), I guess it is the same here.

Milan

> 
> On 06.08.2015 14:19, Lars Winterfeld wrote:
>> Am 06.08.2015 um 12:38 schrieb Vasile Catalin-B50542:
>>> Does the underlying encryption layer (CryptoAPI) have to ensure the
>>> complete callbacks are called in the order the requests were submitted?
>> Are you thinking about parallelization or asynchronous processing?
>>
> 
> 
> _______________________________________________
> dm-crypt mailing list
> dm-crypt@saout.de
> http://www.saout.de/mailman/listinfo/dm-crypt
> 

Re: [dm-crypt] out of order encryption

[Vasile Catalin-B50542]

[-- Attachment #1: Type: text/plain, Size: 2164 bytes --]

I am using a Freescale P3041DS platform with CAAM 
(drivers/crypto/CAAM/*) hardware accelerator.
The platforms is a PowerPC processor.

On 06.08.2015 14:48, Milan Broz wrote:
> On 08/06/2015 01:32 PM, Vasile Catalin-B50542 wrote:
>> I am sending crypto requests to a hardware component which has
>> an option to keeping them in order or not. I was trying to use the out of
>> order option because this would let requests run in parallel.
>> If I tried running them with the out of order option bun then I get:
>> No key available with this passphrase.
>> When trying to execute the following commands:
>> cryptsetup -c aes-xts-plain -y -v luksFormat /dev/sda1
>> cryptsetup open /dev/sda1 test_xts1
>
> Sigh. Please when you asking about some explicit problem, please
> always mention it from the beginning...
>
> So this is probably unrelated to dmcrypt. Userspace cryptsetup could use
> kernel crypto API wrapper directly (add --debug and you will see,
> there should be log message that it uses kernel crypto userspace interface).
>
> This seems like a bug in the kernel crypto API provider, what exactly it is?
> What architecture?
>
> We have seen the same problem with Raspbery PI2 & NEON crypto driver
> (it was a bug in kernel code), I guess it is the same here.
>
> Milan
>
>> On 06.08.2015 14:19, Lars Winterfeld wrote:
>>> Am 06.08.2015 um 12:38 schrieb Vasile Catalin-B50542:
>>>> Does the underlying encryption layer (CryptoAPI) have to ensure the
>>>> complete callbacks are called in the order the requests were submitted?
>>> Are you thinking about parallelization or asynchronous processing?
>>>
>>
>> _______________________________________________
>> dm-crypt mailing list
>> dm-crypt@saout.de
>> http://www.saout.de/mailman/listinfo/dm-crypt
>>

-- 

CatalinVasile

Intern, DN-Software

FreescaleSemiconductor, Inc.

www.freescale.com

phone: 073-021-1938

e-mail: catalin.vasile@freescale.com <mailto:your.name@freescale.com>

*Freescale_Logo-nosemi_Lh_4c***

This e-mail, and any associated attachments have been classified as:

[ ] Public

[ ] Freescale Semiconductor Internal Use Only

[ ] Freescale Semiconductor Confidential Proprietary


[-- Attachment #2.1: Type: text/html, Size: 30720 bytes --]

[-- Attachment #2.2: image002.gif --]
[-- Type: image/gif, Size: 1979 bytes --]

Re: [dm-crypt] out of order encryption

[Milan Broz]

On 08/06/2015 02:10 PM, Vasile Catalin-B50542 wrote:
> I am using a Freescale P3041DS platform with CAAM (drivers/crypto/CAAM/*) hardware accelerator.
> The platforms is a PowerPC processor.

Well, I think there is some problem with the crypto driver in kernel.

And because I see that this driver is currently in active development, I would suggest
to contact the developer and ask for an advice, see
http://thread.gmane.org/gmane.linux.kernel.cryptoapi/16176

Unfortunately we do not have this hw available so I cannot provide more help here.

You can try to compile cryptsetup with --disable-kernel_crypto option - maybe it is a workaround.
(But there can be a data corruption later...)

Anyway, if it works, the I would guess there is a bug in userspace interface code.

Maybe it can help how we debugged a similar issue here
http://thread.gmane.org/gmane.linux.kernel.device-mapper.dm-crypt/7966/focus=13358

Milan

> 
> On 06.08.2015 14:48, Milan Broz wrote:
>> On 08/06/2015 01:32 PM, Vasile Catalin-B50542 wrote:
>>> I am sending crypto requests to a hardware component which has
>>> an option to keeping them in order or not. I was trying to use the out of
>>> order option because this would let requests run in parallel.
>>> If I tried running them with the out of order option bun then I get:
>>> No key available with this passphrase.
>>> When trying to execute the following commands:
>>> cryptsetup -c aes-xts-plain -y -v luksFormat /dev/sda1
>>> cryptsetup open /dev/sda1 test_xts1
>>
>> Sigh. Please when you asking about some explicit problem, please
>> always mention it from the beginning...
>>
>> So this is probably unrelated to dmcrypt. Userspace cryptsetup could use
>> kernel crypto API wrapper directly (add --debug and you will see,
>> there should be log message that it uses kernel crypto userspace interface).
>>
>> This seems like a bug in the kernel crypto API provider, what exactly it is?
>> What architecture?
>>
>> We have seen the same problem with Raspbery PI2 & NEON crypto driver
>> (it was a bug in kernel code), I guess it is the same here.
>>
>> Milan
>>
>>> On 06.08.2015 14:19, Lars Winterfeld wrote:
>>>> Am 06.08.2015 um 12:38 schrieb Vasile Catalin-B50542:
>>>>> Does the underlying encryption layer (CryptoAPI) have to ensure the
>>>>> complete callbacks are called in the order the requests were submitted?
>>>> Are you thinking about parallelization or asynchronous processing?
>>>>
>>>
>>> _______________________________________________
>>> dm-crypt mailing list
>>> dm-crypt@saout.de
>>> http://www.saout.de/mailman/listinfo/dm-crypt
>>>
> 
> -- 
> 
> CatalinVasile
> 
> Intern, DN-Software
> 
> FreescaleSemiconductor, Inc.
> 
> www.freescale.com
> 
> phone: 073-021-1938
> 
> e-mail: catalin.vasile@freescale.com <mailto:your.name@freescale.com>
> 
> *Freescale_Logo-nosemi_Lh_4c***
> 
> This e-mail, and any associated attachments have been classified as:
> 
> [ ] Public
> 
> [ ] Freescale Semiconductor Internal Use Only
> 
> [ ] Freescale Semiconductor Confidential Proprietary
> 
> 
> 
> _______________________________________________
> dm-crypt mailing list
> dm-crypt@saout.de
> http://www.saout.de/mailman/listinfo/dm-crypt
>