Bug: luksClose search LUKS header but it is detached

Bug: luksClose search LUKS header but it is detached

[doffloster]

Hi all,

I noticed that cryptsetup LUKS extensions is searching for a LUKS
header in the device.
This is expected when the LUKS header should be in the device.
But I used a detached LUKS header, so I wasn't expecting to see
cryptsetup searching for the LUKS header in the device.

You can see that in the log file "log.txt" - in the URL:
https://drive.google.com/file/d/1DcrpklgE75oE5znAxHAlp9CPikkYbVq1/view?usp=sharing

You may notice that I used a "--header" flag, but I also tried without
it - still, cryptsetup behaved the same.

The incriminating lines from the attached log are as follows:
"
# Trying to read primary LUKS2 header at offset 0x0.
# Opening locked device /dev/sdb1
# Verifying locked device handle (bdev)
# Trying to read secondary LUKS2 header at offset 0x4000.
# Reusing open ro fd on device /dev/sdb1
# Trying to read secondary LUKS2 header at offset 0x8000.
# Reusing open ro fd on device /dev/sdb1
# Trying to read secondary LUKS2 header at offset 0x10000.
# Reusing open ro fd on device /dev/sdb1
# Trying to read secondary LUKS2 header at offset 0x20000.
# Reusing open ro fd on device /dev/sdb1
# Trying to read secondary LUKS2 header at offset 0x40000.
# Reusing open ro fd on device /dev/sdb1
# Trying to read secondary LUKS2 header at offset 0x80000.
# Reusing open ro fd on device /dev/sdb1
# Trying to read secondary LUKS2 header at offset 0x100000.
# Reusing open ro fd on device /dev/sdb1
# Trying to read secondary LUKS2 header at offset 0x200000.
# Reusing open ro fd on device /dev/sdb1
# Trying to read secondary LUKS2 header at offset 0x400000.
# Reusing open ro fd on device /dev/sdb1
# LUKS2 header read failed (-22).
"

Do you consider it to be a bug?

Best Regards,
David.

Re: Bug: luksClose search LUKS header but it is detached

[Milan Broz]

On 09/08/2022 23:53, doffloster@gmail.com wrote:
> Hi all,
> 
> I noticed that cryptsetup LUKS extensions is searching for a LUKS
> header in the device.
> This is expected when the LUKS header should be in the device.
> But I used a detached LUKS header, so I wasn't expecting to see
> cryptsetup searching for the LUKS header in the device.
> 
> You can see that in the log file "log.txt" - in the URL:
> https://drive.google.com/file/d/1DcrpklgE75oE5znAxHAlp9CPikkYbVq1/view?usp=sharing
> 
> You may notice that I used a "--header" flag, but I also tried without
> it - still, cryptsetup behaved the same.
> 
> The incriminating lines from the attached log are as follows:

...
> Do you consider it to be a bug?

Hi,

No, this is expected. But for close (deactivation) it proceeds even
if it does not find the detached header.

The reason is that even close need some information from the header
(for example UUID).

Is there any functional problem with this or it just looks strange?

If it is problem, you can always use low-level access through
"dmsetup remove <device>" here, but it can keeps some underlying device
active (for LUKS2 this is used only for integrity protected devices).

Milan

Re: Bug: luksClose search LUKS header but it is detached

[doffloster]

Hi Milan,

I appreciate your quick response.

Here is my reply:

> No, this is expected. But for close (deactivation) it proceeds even
> if it does not find the detached header.

Why would cryptsetup seek the detached header in the device?
Wouldn't it be better for cryptsetup to use the '--header' flag?

> The reason is that even close need some information from the header
> (for example UUID).

Doesn't the linux kernel store this kind of information?

> Is there any functional problem with this or it just looks strange?

There isn't any issue.
I just noticed an unexpected behavior and thought that I should report about it.

Best regards,
David.


On Wed, Aug 10, 2022 at 9:58 AM Milan Broz <gmazyland@gmail.com> wrote:
>
> On 09/08/2022 23:53, doffloster@gmail.com wrote:
> > Hi all,
> >
> > I noticed that cryptsetup LUKS extensions is searching for a LUKS
> > header in the device.
> > This is expected when the LUKS header should be in the device.
> > But I used a detached LUKS header, so I wasn't expecting to see
> > cryptsetup searching for the LUKS header in the device.
> >
> > You can see that in the log file "log.txt" - in the URL:
> > https://drive.google.com/file/d/1DcrpklgE75oE5znAxHAlp9CPikkYbVq1/view?usp=sharing
> >
> > You may notice that I used a "--header" flag, but I also tried without
> > it - still, cryptsetup behaved the same.
> >
> > The incriminating lines from the attached log are as follows:
>
> ...
> > Do you consider it to be a bug?
>
> Hi,
>
> No, this is expected. But for close (deactivation) it proceeds even
> if it does not find the detached header.
>
> The reason is that even close need some information from the header
> (for example UUID).
>
> Is there any functional problem with this or it just looks strange?
>
> If it is problem, you can always use low-level access through
> "dmsetup remove <device>" here, but it can keeps some underlying device
> active (for LUKS2 this is used only for integrity protected devices).
>
> Milan
>
>

Re: Bug: luksClose search LUKS header but it is detached

[Milan Broz]

On 10/08/2022 14:51, doffloster@gmail.com wrote:
> Hi Milan,
> 
> I appreciate your quick response.
> 
> Here is my reply:
> 
>> No, this is expected. But for close (deactivation) it proceeds even
>> if it does not find the detached header.
> 
> Why would cryptsetup seek the detached header in the device?
> Wouldn't it be better for cryptsetup to use the '--header' flag?

Kernel has no information that detached header was used, so userspace
always tries to initialize the context using header.

Close was, for a long time, an exception, the device was just teared down
without checking context.
But we already have use cases where we need full initialized context
even on close (check that header has correct UUID referenced in kernel etc).

That said, it should not scan for header if there is no space for
it (IOW data offset is 0 - there is no reserved space on data device
for header; only detached header exists; data device is fully used).
If this is happening, it is a bug. (I'll check it.)
  
>> The reason is that even close need some information from the header
>> (for example UUID).
> 
> Doesn't the linux kernel store this kind of information?

Yes, but it need to be compared against the header, not the kernel.
(Imagine that you need to check password before tearing down the device
- not the current situation, but we already have a need for it in the future).
  
>> Is there any functional problem with this or it just looks strange?
> 
> There isn't any issue.
> I just noticed an unexpected behavior and thought that I should report about it.

We can easily add option that will not check the header on close, but
if it is no real problem, I would better keep it as all actions basically
works the same.

Thanks,
Milan


> 
> Best regards,
> David.
> 
> 
> On Wed, Aug 10, 2022 at 9:58 AM Milan Broz <gmazyland@gmail.com> wrote:
>>
>> On 09/08/2022 23:53, doffloster@gmail.com wrote:
>>> Hi all,
>>>
>>> I noticed that cryptsetup LUKS extensions is searching for a LUKS
>>> header in the device.
>>> This is expected when the LUKS header should be in the device.
>>> But I used a detached LUKS header, so I wasn't expecting to see
>>> cryptsetup searching for the LUKS header in the device.
>>>
>>> You can see that in the log file "log.txt" - in the URL:
>>> https://drive.google.com/file/d/1DcrpklgE75oE5znAxHAlp9CPikkYbVq1/view?usp=sharing
>>>
>>> You may notice that I used a "--header" flag, but I also tried without
>>> it - still, cryptsetup behaved the same.
>>>
>>> The incriminating lines from the attached log are as follows:
>>
>> ...
>>> Do you consider it to be a bug?
>>
>> Hi,
>>
>> No, this is expected. But for close (deactivation) it proceeds even
>> if it does not find the detached header.
>>
>> The reason is that even close need some information from the header
>> (for example UUID).
>>
>> Is there any functional problem with this or it just looks strange?
>>
>> If it is problem, you can always use low-level access through
>> "dmsetup remove <device>" here, but it can keeps some underlying device
>> active (for LUKS2 this is used only for integrity protected devices).
>>
>> Milan
>>
>>

Re: Bug: luksClose search LUKS header but it is detached

[doffloster]

Hi Milan,

> Kernel has no information that detached header was used, so userspace
> always tries to initialize the context using header.

Is it possible to store all the LUKS header information in the linux
kernel - after that the "open" operation completed successfully?
That LUKS header would be associated with the opened device (for
example, the kernel would know that "/dev/mapper/data_crypt" has a
LUKS header stored in the kernel.)

> That said, it should not scan for header if there is no space for
> it (IOW data offset is 0 - there is no reserved space on data device
> for header; only detached header exists; data device is fully used).
> If this is happening, it is a bug. (I'll check it.)

From that paragraph, I understand the following:
cryptsetup behavior during luksClose (as I've shown in my first email)
shouldn't happen, because that the LUKS header is detached, but we've
seen cryptsetup scanning for the header in offset zero which contains
data.
As can be seen from the log file that I sent in the first email:
> # Trying to read primary LUKS2 header at offset 0x0.

Thanks,
David.

On Thu, Aug 11, 2022 at 9:42 AM Milan Broz <gmazyland@gmail.com> wrote:
>
> On 10/08/2022 14:51, doffloster@gmail.com wrote:
> > Hi Milan,
> >
> > I appreciate your quick response.
> >
> > Here is my reply:
> >
> >> No, this is expected. But for close (deactivation) it proceeds even
> >> if it does not find the detached header.
> >
> > Why would cryptsetup seek the detached header in the device?
> > Wouldn't it be better for cryptsetup to use the '--header' flag?
>
> Kernel has no information that detached header was used, so userspace
> always tries to initialize the context using header.
>
> Close was, for a long time, an exception, the device was just teared down
> without checking context.
> But we already have use cases where we need full initialized context
> even on close (check that header has correct UUID referenced in kernel etc).
>
> That said, it should not scan for header if there is no space for
> it (IOW data offset is 0 - there is no reserved space on data device
> for header; only detached header exists; data device is fully used).
> If this is happening, it is a bug. (I'll check it.)
>
> >> The reason is that even close need some information from the header
> >> (for example UUID).
> >
> > Doesn't the linux kernel store this kind of information?
>
> Yes, but it need to be compared against the header, not the kernel.
> (Imagine that you need to check password before tearing down the device
> - not the current situation, but we already have a need for it in the future).
>
> >> Is there any functional problem with this or it just looks strange?
> >
> > There isn't any issue.
> > I just noticed an unexpected behavior and thought that I should report about it.
>
> We can easily add option that will not check the header on close, but
> if it is no real problem, I would better keep it as all actions basically
> works the same.
>
> Thanks,
> Milan
>
>
> >
> > Best regards,
> > David.
> >
> >
> > On Wed, Aug 10, 2022 at 9:58 AM Milan Broz <gmazyland@gmail.com> wrote:
> >>
> >> On 09/08/2022 23:53, doffloster@gmail.com wrote:
> >>> Hi all,
> >>>
> >>> I noticed that cryptsetup LUKS extensions is searching for a LUKS
> >>> header in the device.
> >>> This is expected when the LUKS header should be in the device.
> >>> But I used a detached LUKS header, so I wasn't expecting to see
> >>> cryptsetup searching for the LUKS header in the device.
> >>>
> >>> You can see that in the log file "log.txt" - in the URL:
> >>> https://drive.google.com/file/d/1DcrpklgE75oE5znAxHAlp9CPikkYbVq1/view?usp=sharing
> >>>
> >>> You may notice that I used a "--header" flag, but I also tried without
> >>> it - still, cryptsetup behaved the same.
> >>>
> >>> The incriminating lines from the attached log are as follows:
> >>
> >> ...
> >>> Do you consider it to be a bug?
> >>
> >> Hi,
> >>
> >> No, this is expected. But for close (deactivation) it proceeds even
> >> if it does not find the detached header.
> >>
> >> The reason is that even close need some information from the header
> >> (for example UUID).
> >>
> >> Is there any functional problem with this or it just looks strange?
> >>
> >> If it is problem, you can always use low-level access through
> >> "dmsetup remove <device>" here, but it can keeps some underlying device
> >> active (for LUKS2 this is used only for integrity protected devices).
> >>
> >> Milan
> >>
> >>