Re: Block based OTA update needs mtdblock

Re: Block based OTA update needs mtdblock

[Pintu Agarwal]

Hi,

On Mon, 4 Nov 2024 at 21:31, Pintu Agarwal <pintu.ping@gmail.com> wrote:
>
> Hi All,
>
> For one of our automotive products we have the following configuration:
> QC chipset, arm64, Kernel-5.15, NAND Flash 1GB, A/B system, UBI
> volumes (squashfs, ubifs), DM-verity for rootfs (squashfs), simple
> busybox platform.
>
> For OTA updates we have a strong dependency with MTD_BLOCK.
>
> Till now, we were using ubiblock for mounting squashfs volumes and
> completely got rid of mtd_block by configuring it as a loadable
> module.
> But, we also need to support OTA updates (Full, Incremental) on A/B
> volumes using the same Android OTA framework.
> https://source.android.com/docs/core/ota/nonab/block
>
> OTA update will be applied to the B (inactive) partition.
> OTA updates prefer block based update over file based especially for
> dm-verity enabled devices.
>
> Now, the problem is, on MTD we only have 2 options for block based
> updates; ubi_block or mtd_block.
> We cannot use ubiblock for OTA updates as it is read only.
> For full update volume, we can use "ubiupdatevol" interface to
> completely replace the volume content, but for partial or incremental
> update we need to update only specific blocks and not entire
> partitions.
> Thus, we have to use the MTD_BLOCK (/dev/mtdblock) interface to
> support block based OTA updates on UBI volumes.
> Thus, during ota updates (only) we need to install the mtdblock
> module, perform the update and then uninstall the module.
>
> That means, we cannot completely get rid of MTD_BLOCK from our product
> especially for OTA use cases.
>
> Is this the only way, or do we have any other option to support OTA
> updates over UBI volumes ?
>
Restarting this thread again...
Any further comment on this ?

Did anybody used block based OTA update NAND A/B system without using
mtd_block ?
Since ubiblock is read-only, it seems there is no other way to perform
OTA update, if mtd_block is disabled.
Or, we need to make ubiblock also as read/write.


Thanks,
Pintu

Re: Block based OTA update needs mtdblock

[Miquel Raynal]

Hello,

On 20/11/2024 at 12:52:57 +0530, Pintu Agarwal <pintu.ping@gmail.com> wrote:

> Hi,
>
> On Mon, 4 Nov 2024 at 21:31, Pintu Agarwal <pintu.ping@gmail.com> wrote:
>>
>> Hi All,
>>
>> For one of our automotive products we have the following configuration:
>> QC chipset, arm64, Kernel-5.15, NAND Flash 1GB, A/B system, UBI
>> volumes (squashfs, ubifs), DM-verity for rootfs (squashfs), simple
>> busybox platform.
>>
>> For OTA updates we have a strong dependency with MTD_BLOCK.
>>
>> Till now, we were using ubiblock for mounting squashfs volumes and
>> completely got rid of mtd_block by configuring it as a loadable
>> module.
>> But, we also need to support OTA updates (Full, Incremental) on A/B
>> volumes using the same Android OTA framework.
>> https://source.android.com/docs/core/ota/nonab/block
>>
>> OTA update will be applied to the B (inactive) partition.
>> OTA updates prefer block based update over file based especially for
>> dm-verity enabled devices.
>>
>> Now, the problem is, on MTD we only have 2 options for block based
>> updates; ubi_block or mtd_block.
>> We cannot use ubiblock for OTA updates as it is read only.
>> For full update volume, we can use "ubiupdatevol" interface to
>> completely replace the volume content, but for partial or incremental
>> update we need to update only specific blocks and not entire
>> partitions.
>> Thus, we have to use the MTD_BLOCK (/dev/mtdblock) interface to
>> support block based OTA updates on UBI volumes.
>> Thus, during ota updates (only) we need to install the mtdblock
>> module, perform the update and then uninstall the module.
>>
>> That means, we cannot completely get rid of MTD_BLOCK from our product
>> especially for OTA use cases.
>>
>> Is this the only way, or do we have any other option to support OTA
>> updates over UBI volumes ?
>>
> Restarting this thread again...
> Any further comment on this ?
>
> Did anybody used block based OTA update NAND A/B system without using
> mtd_block ?

Not on my side, it is actually a good question. Richard, any ideas?

> Since ubiblock is read-only, it seems there is no other way to perform
> OTA update, if mtd_block is disabled.
> Or, we need to make ubiblock also as read/write.

Doesn't sound possible/a good idea.

I'm sorry for this unhelpful answer :-)

Miquèl

Re: Block based OTA update needs mtdblock

[Richard Weinberger]

----- Ursprüngliche Mail -----
> Von: "Miquel Raynal" <miquel.raynal@bootlin.com>
> An: "Pintu Agarwal" <pintu.ping@gmail.com>
> CC: "richard" <richard@nod.at>, "Vignesh Raghavendra" <vigneshr@ti.com>, "linux-mtd" <linux-mtd@lists.infradead.org>,
> "chengzhihao1" <chengzhihao1@huawei.com>, "linux-kernel" <linux-kernel@vger.kernel.org>
> Gesendet: Dienstag, 3. Dezember 2024 15:17:06
> Betreff: Re: Block based OTA update needs mtdblock

> Hello,
> 
> On 20/11/2024 at 12:52:57 +0530, Pintu Agarwal <pintu.ping@gmail.com> wrote:
> 
>> Hi,
>>
>> On Mon, 4 Nov 2024 at 21:31, Pintu Agarwal <pintu.ping@gmail.com> wrote:
>>>
>>> Hi All,
>>>
>>> For one of our automotive products we have the following configuration:
>>> QC chipset, arm64, Kernel-5.15, NAND Flash 1GB, A/B system, UBI
>>> volumes (squashfs, ubifs), DM-verity for rootfs (squashfs), simple
>>> busybox platform.
>>>
>>> For OTA updates we have a strong dependency with MTD_BLOCK.
>>>
>>> Till now, we were using ubiblock for mounting squashfs volumes and
>>> completely got rid of mtd_block by configuring it as a loadable
>>> module.
>>> But, we also need to support OTA updates (Full, Incremental) on A/B
>>> volumes using the same Android OTA framework.
>>> https://source.android.com/docs/core/ota/nonab/block
>>>
>>> OTA update will be applied to the B (inactive) partition.
>>> OTA updates prefer block based update over file based especially for
>>> dm-verity enabled devices.
>>>
>>> Now, the problem is, on MTD we only have 2 options for block based
>>> updates; ubi_block or mtd_block.
>>> We cannot use ubiblock for OTA updates as it is read only.
>>> For full update volume, we can use "ubiupdatevol" interface to
>>> completely replace the volume content, but for partial or incremental
>>> update we need to update only specific blocks and not entire
>>> partitions.
>>> Thus, we have to use the MTD_BLOCK (/dev/mtdblock) interface to
>>> support block based OTA updates on UBI volumes.
>>> Thus, during ota updates (only) we need to install the mtdblock
>>> module, perform the update and then uninstall the module.
>>>
>>> That means, we cannot completely get rid of MTD_BLOCK from our product
>>> especially for OTA use cases.
>>>
>>> Is this the only way, or do we have any other option to support OTA
>>> updates over UBI volumes ?
>>>
>> Restarting this thread again...
>> Any further comment on this ?
>>
>> Did anybody used block based OTA update NAND A/B system without using
>> mtd_block ?
> 
> Not on my side, it is actually a good question. Richard, any ideas?

What about using ubiupdatevol?

Thanks,
//richard

Re: Block based OTA update needs mtdblock

[Pintu Agarwal]

Hello Richard,

On Tue, 3 Dec 2024 at 21:38, Richard Weinberger <richard@nod.at> wrote:
>
> ----- Ursprüngliche Mail -----
> > Von: "Miquel Raynal" <miquel.raynal@bootlin.com>
> > An: "Pintu Agarwal" <pintu.ping@gmail.com>
> > CC: "richard" <richard@nod.at>, "Vignesh Raghavendra" <vigneshr@ti.com>, "linux-mtd" <linux-mtd@lists.infradead.org>,
> > "chengzhihao1" <chengzhihao1@huawei.com>, "linux-kernel" <linux-kernel@vger.kernel.org>
> > Gesendet: Dienstag, 3. Dezember 2024 15:17:06
> > Betreff: Re: Block based OTA update needs mtdblock
>
> > Hello,
> >
> > On 20/11/2024 at 12:52:57 +0530, Pintu Agarwal <pintu.ping@gmail.com> wrote:
> >
> >> Hi,
> >>
> >> On Mon, 4 Nov 2024 at 21:31, Pintu Agarwal <pintu.ping@gmail.com> wrote:
> >>>
> >>> Hi All,
> >>>
> >>> For one of our automotive products we have the following configuration:
> >>> QC chipset, arm64, Kernel-5.15, NAND Flash 1GB, A/B system, UBI
> >>> volumes (squashfs, ubifs), DM-verity for rootfs (squashfs), simple
> >>> busybox platform.
> >>>
> >>> For OTA updates we have a strong dependency with MTD_BLOCK.
> >>>
> >>> Till now, we were using ubiblock for mounting squashfs volumes and
> >>> completely got rid of mtd_block by configuring it as a loadable
> >>> module.
> >>> But, we also need to support OTA updates (Full, Incremental) on A/B
> >>> volumes using the same Android OTA framework.
> >>> https://source.android.com/docs/core/ota/nonab/block
> >>>
> >>> OTA update will be applied to the B (inactive) partition.
> >>> OTA updates prefer block based update over file based especially for
> >>> dm-verity enabled devices.
> >>>
> >>> Now, the problem is, on MTD we only have 2 options for block based
> >>> updates; ubi_block or mtd_block.
> >>> We cannot use ubiblock for OTA updates as it is read only.
> >>> For full update volume, we can use "ubiupdatevol" interface to
> >>> completely replace the volume content, but for partial or incremental
> >>> update we need to update only specific blocks and not entire
> >>> partitions.
> >>> Thus, we have to use the MTD_BLOCK (/dev/mtdblock) interface to
> >>> support block based OTA updates on UBI volumes.
> >>> Thus, during ota updates (only) we need to install the mtdblock
> >>> module, perform the update and then uninstall the module.
> >>>
> >>> That means, we cannot completely get rid of MTD_BLOCK from our product
> >>> especially for OTA use cases.
> >>>
> >>> Is this the only way, or do we have any other option to support OTA
> >>> updates over UBI volumes ?
> >>>
> >> Restarting this thread again...
> >> Any further comment on this ?
> >>
> >> Did anybody used block based OTA update NAND A/B system without using
> >> mtd_block ?
> >
> > Not on my side, it is actually a good question. Richard, any ideas?
>
> What about using ubiupdatevol?
>
As I explained above, ubiupdatevol actually performs a full volume
update, right ?
I think the partial/incremental update is not possible with this. Once
the update is triggered it should finish fully.
But for OTA updates (Delta/Incremental), only the delta changes can
also be applied.
Please refer to this [1] about updates mechanism.
[1] https://medium.com/@yigitpirildak/android-ota-update-mechanism-ecc8f6f69f71
[2] https://source.android.com/docs/core/ota/tools#:~:text=An%20incremental%20update%20is%20an,need%20to%20include%20unchanged%20files.

On our NAND target we support OTA updates, including incremental ota updates.
But without mtdblock the incremental OTA update (on A/B system) will not work.
This is the main issue.
For full updates on ubi volumes we are already using the
"ubiupdatevol" mechanism.


Thanks,
Pintu

Re: Block based OTA update needs mtdblock

[Richard Weinberger]

----- Ursprüngliche Mail -----
> Von: "Pintu Agarwal" <pintu.ping@gmail.com>
>> >> Did anybody used block based OTA update NAND A/B system without using
>> >> mtd_block ?
>> >
>> > Not on my side, it is actually a good question. Richard, any ideas?
>>
>> What about using ubiupdatevol?
>>
> As I explained above, ubiupdatevol actually performs a full volume
> update, right ?
> I think the partial/incremental update is not possible with this. Once
> the update is triggered it should finish fully.
> But for OTA updates (Delta/Incremental), only the delta changes can
> also be applied.
> Please refer to this [1] about updates mechanism.
> [1] https://medium.com/@yigitpirildak/android-ota-update-mechanism-ecc8f6f69f71
> [2]
> https://source.android.com/docs/core/ota/tools#:~:text=An%20incremental%20update%20is%20an,need%20to%20include%20unchanged%20files.
> 
> On our NAND target we support OTA updates, including incremental ota updates.
> But without mtdblock the incremental OTA update (on A/B system) will not work.
> This is the main issue.
> For full updates on ubi volumes we are already using the
> "ubiupdatevol" mechanism.

Well, for dynamic volumes you can use the UBI_IOCEBCH ioctl to replace LEBs.
So, incremental updates are doable too.

Thanks,
//richard

Re: Block based OTA update needs mtdblock

[Pintu Agarwal]

Hi Richard,

On Wed, 4 Dec 2024 at 15:46, Richard Weinberger <richard@nod.at> wrote:
>
> ----- Ursprüngliche Mail -----
> > Von: "Pintu Agarwal" <pintu.ping@gmail.com>
> >> >> Did anybody used block based OTA update NAND A/B system without using
> >> >> mtd_block ?
> >> >
> >> > Not on my side, it is actually a good question. Richard, any ideas?
> >>
> >> What about using ubiupdatevol?
> >>
> > As I explained above, ubiupdatevol actually performs a full volume
> > update, right ?
> > I think the partial/incremental update is not possible with this. Once
> > the update is triggered it should finish fully.
> > But for OTA updates (Delta/Incremental), only the delta changes can
> > also be applied.
> > Please refer to this [1] about updates mechanism.
> > [1] https://medium.com/@yigitpirildak/android-ota-update-mechanism-ecc8f6f69f71
> > [2]
> > https://source.android.com/docs/core/ota/tools#:~:text=An%20incremental%20update%20is%20an,need%20to%20include%20unchanged%20files.
> >
> > On our NAND target we support OTA updates, including incremental ota updates.
> > But without mtdblock the incremental OTA update (on A/B system) will not work.
> > This is the main issue.
> > For full updates on ubi volumes we are already using the
> > "ubiupdatevol" mechanism.
>
> Well, for dynamic volumes you can use the UBI_IOCEBCH ioctl to replace LEBs.
> So, incremental updates are doable too.
>
Oh I see. This seems useful. I will try it and confirm if it works for us.
Thank you so much for this.

Even the header file (/usr/include/mtd/ubi-user.h) mentions these details.
{{{
 * Atomic logical eraseblock change
 * ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 *
 * Atomic logical eraseblock change operation is called using the %UBI_IOCEBCH
 * ioctl command of the corresponding UBI volume character device. A pointer to
 * a &struct ubi_leb_change_req object has to be passed to the ioctl. Then the
 * user is expected to write the requested amount of bytes (similarly to what
 * should be done in case of the "volume update" ioctl).
...
/* Atomic LEB change command */
#define UBI_IOCEBCH _IOW(UBI_VOL_IOC_MAGIC, 2, __s32)

* @lnum: logical eraseblock number to change
struct ubi_leb_change_req {
        __s32 lnum;
        __s32 bytes;
        __s8  dtype; /* obsolete, do not use! */
        __s8  padding[7];
} __attribute__((packed));

}}}
This requires the logical erase-block number to be known.
I am wondering how to get the LEB number from user space.
I need to explore further..

Thank you so much...

Regards,
Pintu