[dm-crypt] Initializing dm-crypt on Android results in "device lookup failed"

[dm-crypt] Initializing dm-crypt on Android results in "device lookup failed"

[Roman Schlegel]

Hello,

I am currently trying to track down an issue with dm-crypt on an Android
device (more specifically, I am building a CM 12.1 ROM), where
initializing device encryption fails with an error.

I have asked this question in a number of other places as well
(stackoverflow and xda), but have not gotten any responses. I therefore
decided to ask the question on a list more directly connected to
dm-crypt. It is if course entirely possible that the error lies
somewhere in the device-mapper, but in that case my hope is still that
somebody on this list is familiar enough with the device-mapper and its
interaction with dm-crypt to point me to a potential solution.

So here is the problem: when starting the device encryption, the
encryption itself fails with the following error:

01-01 03:20:02.631 I/Cryptfs (  254): Using scrypt for cryptfs KDF

01-01 03:20:03.592 I/Cryptfs (  254): Enabling support for
allow_discards in dmcrypt.

01-01 03:20:03.592 I/Cryptfs (  254): load_crypto_mapping_table:
target_type = crypt

01-01 03:20:03.592 I/Cryptfs (  254): load_crypto_mapping_table:
real_blk_name = /dev/block/platform/mtk-msdc.0/by-name/userdata,
extra_params = 1 allow_discards

01-01 03:20:03.592 I/Cryptfs (  254): load_crypto_mapping_table:
crypt_params = aes-cbc-essiv:sha256 0A66F89B0D3DFC0B05D9BD23B3453A70 0
/dev/block/platform/mtk-msdc.0/by-name/userdata 0 1 allow_discards 0

01-01 03:20:08.619 E/Cryptfs (  254): Cannot load dm-crypt mapping table.


at the same time, the kernel log prints the following messages:

<3>[  138.163773] (5)[327:vold]device-mapper: table: 253:0: crypt:
Device lookup failed
<4>[  138.163781] (5)[327:vold]device-mapper: ioctl: error adding target
to table

Unfortunately, the device I am testing this with has not had the kernel
sources released, so I only have a prebuilt kernel (3.10.61) that I
cannot recompile. However, another ROM using the same kernel (IIRC) can
successfully encrypt the device, so I don't think it is anything
inherent in the kernel itself. But I am at a loss what else could be the
problem.

I'd be happy to provide any additional information as needed.

Any help or pointers in solving this issue would be greatly appreciated.
Thanks!

Best regards,

Roman

Re: [dm-crypt] Initializing dm-crypt on Android results in "device lookup failed"

[Ondrej Kozina]

Hello Roman,

I'm no Android developer, so just according to what I can see in your logs:

On 05/22/2016 11:31 AM, Roman Schlegel wrote:
>
> 01-01 03:20:03.592 I/Cryptfs (  254): load_crypto_mapping_table:
> crypt_params = aes-cbc-essiv:sha256 0A66F89B0D3DFC0B05D9BD23B3453A70 0

Don't post your volume key publicly! If it was just a test device it's 
not so big deal but it's generally good rule to follow so that you don't 
post your VK for real device accidentally.

</mentoring_mode>

> /dev/block/platform/mtk-msdc.0/by-name/userdata 0 1 allow_discards 0
>
> 01-01 03:20:08.619 E/Cryptfs (  254): Cannot load dm-crypt mapping table.
>
>
> at the same time, the kernel log prints the following messages:
>
> <3>[  138.163773] (5)[327:vold]device-mapper: table: 253:0: crypt:
> Device lookup failed

The kernel driver complaints that it can't access a block device 
referenced by path /dev/block/platform/mtk-msdc.0/by-name/userdata. Is 
it block device? If it's supposed to be a symlink does a referenced 
block device exist?

The thing is kernel accepts only limited set of paths as device 
identifier in DM table. Usually major:minor numbers, kernel path, 
part_uuid. The translation from arbitrary path '
/dev/block/platform/mtk-msdc.0/by-name/userdata' to something DM driver 
is able to parse should be done in userspace. Usually, it's libdevmapper 
library what does that. Given that kernel complaints, it seems userspace 
haven't done the translation properly or there was a race maybe. Could 
you pass major:minor for /dev/block/platform/mtk-msdc.0/by-name/userdata 
directly in userspace utility that's supposed to establish device 
encryption?

Regards
Ondrej

Re: [dm-crypt] Initializing dm-crypt on Android results in "device lookup failed"

[Milan Broz]

On 05/22/2016 11:31 AM, Roman Schlegel wrote:
> Hello,
> 
> I am currently trying to track down an issue with dm-crypt on an Android
> device (more specifically, I am building a CM 12.1 ROM), where
> initializing device encryption fails with an error.

...

> 01-01 03:20:03.592 I/Cryptfs (  254): load_crypto_mapping_table:
> crypt_params = aes-cbc-essiv:sha256 0A66F89B0D3DFC0B05D9BD23B3453A70 0
> /dev/block/platform/mtk-msdc.0/by-name/userdata 0 1 allow_discards 0

I really hope Android does not log volume key such a way to system log...

> 
> 01-01 03:20:08.619 E/Cryptfs (  254): Cannot load dm-crypt mapping table.
> 
> 
> at the same time, the kernel log prints the following messages:
> 
> <3>[  138.163773] (5)[327:vold]device-mapper: table: 253:0: crypt:
> Device lookup failed

This message seems to indicate that problem is with device specification:
/dev/block/platform/mtk-msdc.0/by-name/userdata

Can you check, that such device node exists before calling this activation?

You can try to use "major:minor" pair instead (of the device above),
but it can sause problem if it is dynamic (it shouldn't be in this
environment though).

Milan
p.s.

> Unfortunately, the device I am testing this with has not had the kernel
> sources released, so I only have a prebuilt kernel (3.10.61) that I
> cannot recompile. However, another ROM using the same kernel (IIRC) can
> successfully encrypt the device, so I don't think it is anything
> inherent in the kernel itself. But I am at a loss what else could be the
> problem.

The situation in Android world is really not good. Some major vendors even
reimplemented dm-crypt with hardcoded algorithms to speed up on their crappy hw.
Obviously without contact mainline kernel developers, and breaking all other
interfaces as a side effect etc.
So do not be surprised that we will ignore any support questions if someone
is doing such things, nobody really know what source code was used to compile
the kernel.

Re: [dm-crypt] Initializing dm-crypt on Android results in "device lookup failed"

[Roman Schlegel]

Hi Ondrej,

Thanks a lot for your reply.

On 23.05.2016 10:17, Ondrej Kozina wrote:
> Hello Roman,
> 
> I'm no Android developer, so just according to what I can see in your logs:
> 
> On 05/22/2016 11:31 AM, Roman Schlegel wrote:
>>
>> 01-01 03:20:03.592 I/Cryptfs (  254): load_crypto_mapping_table:
>> crypt_params = aes-cbc-essiv:sha256 0A66F89B0D3DFC0B05D9BD23B3453A70 0
> 
> Don't post your volume key publicly! If it was just a test device it's
> not so big deal but it's generally good rule to follow so that you don't
> post your VK for real device accidentally.
> 
> </mentoring_mode>
> 

I was aware that the log contained the volume key (it is a test device),
but you're of course right that I should not get in the habit of posting
it :-).

>> /dev/block/platform/mtk-msdc.0/by-name/userdata 0 1 allow_discards 0
>>
>> 01-01 03:20:08.619 E/Cryptfs (  254): Cannot load dm-crypt mapping table.
>>
>>
>> at the same time, the kernel log prints the following messages:
>>
>> <3>[  138.163773] (5)[327:vold]device-mapper: table: 253:0: crypt:
>> Device lookup failed
> 
> The kernel driver complaints that it can't access a block device
> referenced by path /dev/block/platform/mtk-msdc.0/by-name/userdata. Is
> it block device? If it's supposed to be a symlink does a referenced
> block device exist?
> 
> The thing is kernel accepts only limited set of paths as device
> identifier in DM table. Usually major:minor numbers, kernel path,
> part_uuid. The translation from arbitrary path '
> /dev/block/platform/mtk-msdc.0/by-name/userdata' to something DM driver
> is able to parse should be done in userspace. Usually, it's libdevmapper
> library what does that. Given that kernel complaints, it seems userspace
> haven't done the translation properly or there was a race maybe. Could
> you pass major:minor for /dev/block/platform/mtk-msdc.0/by-name/userdata
> directly in userspace utility that's supposed to establish device
> encryption?
> 

The device /dev/block/platform/mtk-msdc.0/by-name/userdata is actually a
symbolic link to another block device, but it seems to exist at the time
of encryption.

What exactly do you mean with that there might be a race?

Thanks and best regards,

Roman

Re: [dm-crypt] Initializing dm-crypt on Android results in "device lookup failed"

[Roman Schlegel]

Hi Milan,

Thanks a lot for your reply.

On 23.05.2016 11:04, Milan Broz wrote:
> On 05/22/2016 11:31 AM, Roman Schlegel wrote:
>> Hello,
>>
>> I am currently trying to track down an issue with dm-crypt on an Android
>> device (more specifically, I am building a CM 12.1 ROM), where
>> initializing device encryption fails with an error.
> 
> ...
> 
>> 01-01 03:20:03.592 I/Cryptfs (  254): load_crypto_mapping_table:
>> crypt_params = aes-cbc-essiv:sha256 0A66F89B0D3DFC0B05D9BD23B3453A70 0
>> /dev/block/platform/mtk-msdc.0/by-name/userdata 0 1 allow_discards 0
> 
> I really hope Android does not log volume key such a way to system log...
> 

No, it doesn't :-). That's the testing output I added to vold to check
that it wasn't a problem with the parameters submitted to dm-crypt.

>> 01-01 03:20:08.619 E/Cryptfs (  254): Cannot load dm-crypt mapping table.
>>
>>
>> at the same time, the kernel log prints the following messages:
>>
>> <3>[  138.163773] (5)[327:vold]device-mapper: table: 253:0: crypt:
>> Device lookup failed
> 
> This message seems to indicate that problem is with device specification:
> /dev/block/platform/mtk-msdc.0/by-name/userdata
> 
> Can you check, that such device node exists before calling this activation?
> 
> You can try to use "major:minor" pair instead (of the device above),
> but it can sause problem if it is dynamic (it shouldn't be in this
> environment though).

I checked, and the /dev/block/platform/mtk-msdc.0/by-name/userdata is a
symbolic link to /dev/block/mmcblk0p17, which seems to be a real block
device. And both exist at the time when the encryption process is
started (and the symbolic link is valid).
As a first step I tried to submit the actual block device (i.e.,
/dev/block/mmcblk0p17) as a parameter, but it still fails with the exact
same error.
I'll now try to pass the major:minor pair to see if that works.

For what it is worth, when trying to narrow down the problem in the last
few days, I think I could track it down to a function call in dm-crypt.c
in the kernel, namely in the function crypt_ctr():

...
	if (dm_get_device(ti, argv[3], dm_table_get_mode(ti->table), &cc->dev)) {
		ti->error = "Device lookup failed";
		goto bad;
	}
...

Within the called function (dm_get_device() in dm-table.c), I can't
really tell where the error happens, I think it could be any of the
following three lines:

struct block_device *bdev = lookup_bdev(path);

if ((r = open_dev(dd, dev, t->md))) { ... }

r = upgrade_mode(dd, mode, t->md);

But where exactly I cannot say just from the source code, and an error
in any of these produces the same "Device lookup failed" in userspace.

>> Unfortunately, the device I am testing this with has not had the kernel
>> sources released, so I only have a prebuilt kernel (3.10.61) that I
>> cannot recompile. However, another ROM using the same kernel (IIRC) can
>> successfully encrypt the device, so I don't think it is anything
>> inherent in the kernel itself. But I am at a loss what else could be the
>> problem.
> 
> The situation in Android world is really not good. Some major vendors even
> reimplemented dm-crypt with hardcoded algorithms to speed up on their crappy hw.
> Obviously without contact mainline kernel developers, and breaking all other
> interfaces as a side effect etc.
> So do not be surprised that we will ignore any support questions if someone
> is doing such things, nobody really know what source code was used to compile
> the kernel.

I completely understand, which is why I am very thankful for all the
replies. They have released the source code for the same model of phone,
but with a different chipset (i.e., they have released the source for
the model with the Qualcomm Snapdragon CPU, but not the source for the
model with the MTK CPU, which is the one I have).

Thanks a lot again and best regards,

Roman

Re: [dm-crypt] Initializing dm-crypt on Android results in "device lookup failed"

[Roman Schlegel]

Hi,

On 23.05.2016 21:17, Roman Schlegel wrote:
> Hi Milan,
> 
> Thanks a lot for your reply.
> 
> On 23.05.2016 11:04, Milan Broz wrote:
>> On 05/22/2016 11:31 AM, Roman Schlegel wrote:
>>> Hello,
>>>
>>> I am currently trying to track down an issue with dm-crypt on an Android
>>> device (more specifically, I am building a CM 12.1 ROM), where
>>> initializing device encryption fails with an error.
>>
>> ...
>>
>>> 01-01 03:20:03.592 I/Cryptfs (  254): load_crypto_mapping_table:
>>> crypt_params = aes-cbc-essiv:sha256 0A66F89B0D3DFC0B05D9BD23B3453A70 0
>>> /dev/block/platform/mtk-msdc.0/by-name/userdata 0 1 allow_discards 0
>>
>> I really hope Android does not log volume key such a way to system log...
>>
> 
> No, it doesn't :-). That's the testing output I added to vold to check
> that it wasn't a problem with the parameters submitted to dm-crypt.
> 
>>> 01-01 03:20:08.619 E/Cryptfs (  254): Cannot load dm-crypt mapping table.
>>>
>>>
>>> at the same time, the kernel log prints the following messages:
>>>
>>> <3>[  138.163773] (5)[327:vold]device-mapper: table: 253:0: crypt:
>>> Device lookup failed
>>
>> This message seems to indicate that problem is with device specification:
>> /dev/block/platform/mtk-msdc.0/by-name/userdata
>>
>> Can you check, that such device node exists before calling this activation?
>>
>> You can try to use "major:minor" pair instead (of the device above),
>> but it can sause problem if it is dynamic (it shouldn't be in this
>> environment though).
> 

I just tried passing the major:minor of the device, instead of a device
path, but I am still getting the same error, "Device lookup failed".

Any further hints on what could be the issue would be very much
appreciated...

Thanks and best regards,

Roman