Block based OTA update needs mtdblock

Block based OTA update needs mtdblock

[Pintu Agarwal]

Hi All,

For one of our automotive products we have the following configuration:
QC chipset, arm64, Kernel-5.15, NAND Flash 1GB, A/B system, UBI
volumes (squashfs, ubifs), DM-verity for rootfs (squashfs), simple
busybox platform.

For OTA updates we have a strong dependency with MTD_BLOCK.

Till now, we were using ubiblock for mounting squashfs volumes and
completely got rid of mtd_block by configuring it as a loadable
module.
But, we also need to support OTA updates (Full, Incremental) on A/B
volumes using the same Android OTA framework.
https://source.android.com/docs/core/ota/nonab/block

OTA update will be applied to the B (inactive) partition.
OTA updates prefer block based update over file based especially for
dm-verity enabled devices.

Now, the problem is, on MTD we only have 2 options for block based
updates; ubi_block or mtd_block.
We cannot use ubiblock for OTA updates as it is read only.
For full update volume, we can use "ubiupdatevol" interface to
completely replace the volume content, but for partial or incremental
update we need to update only specific blocks and not entire
partitions.
Thus, we have to use the MTD_BLOCK (/dev/mtdblock) interface to
support block based OTA updates on UBI volumes.
Thus, during ota updates (only) we need to install the mtdblock
module, perform the update and then uninstall the module.

That means, we cannot completely get rid of MTD_BLOCK from our product
especially for OTA use cases.

Is this the only way, or do we have any other option to support OTA
updates over UBI volumes ?


Thank you,
Regards,
Pintu

______________________________________________________
Linux MTD discussion mailing list
http://lists.infradead.org/mailman/listinfo/linux-mtd/

Re: Block based OTA update needs mtdblock

[Zhihao Cheng]

在 2024/11/5 0:01, Pintu Agarwal 写道:
> Hi All,
> 
> For one of our automotive products we have the following configuration:
> QC chipset, arm64, Kernel-5.15, NAND Flash 1GB, A/B system, UBI
> volumes (squashfs, ubifs), DM-verity for rootfs (squashfs), simple
> busybox platform.
> 
> For OTA updates we have a strong dependency with MTD_BLOCK.
> 
> Till now, we were using ubiblock for mounting squashfs volumes and
> completely got rid of mtd_block by configuring it as a loadable
> module.
> But, we also need to support OTA updates (Full, Incremental) on A/B
> volumes using the same Android OTA framework.
> https://source.android.com/docs/core/ota/nonab/block
> 
> OTA update will be applied to the B (inactive) partition.
> OTA updates prefer block based update over file based especially for
> dm-verity enabled devices.
> 
> Now, the problem is, on MTD we only have 2 options for block based
> updates; ubi_block or mtd_block.
> We cannot use ubiblock for OTA updates as it is read only.
> For full update volume, we can use "ubiupdatevol" interface to
> completely replace the volume content, but for partial or incremental
> update we need to update only specific blocks and not entire
> partitions.
> Thus, we have to use the MTD_BLOCK (/dev/mtdblock) interface to
> support block based OTA updates on UBI volumes.
> Thus, during ota updates (only) we need to install the mtdblock
> module, perform the update and then uninstall the module.

Hi Pintu, how do you export the mtdblock device over on an UBI volume? 
By the gluebi?
> 
> That means, we cannot completely get rid of MTD_BLOCK from our product
> especially for OTA use cases.
> 
> Is this the only way, or do we have any other option to support OTA
> updates over UBI volumes ?
> 
> 
> Thank you,
> Regards,
> Pintu
> .
> 


______________________________________________________
Linux MTD discussion mailing list
http://lists.infradead.org/mailman/listinfo/linux-mtd/

Re: Block based OTA update needs mtdblock

[Pintu Agarwal]

Hi,

On Tue, 5 Nov 2024 at 07:45, Zhihao Cheng <chengzhihao1@huawei.com> wrote:
>
> 在 2024/11/5 0:01, Pintu Agarwal 写道:
> > Hi All,
> >
> > For one of our automotive products we have the following configuration:
> > QC chipset, arm64, Kernel-5.15, NAND Flash 1GB, A/B system, UBI
> > volumes (squashfs, ubifs), DM-verity for rootfs (squashfs), simple
> > busybox platform.
> >
> > For OTA updates we have a strong dependency with MTD_BLOCK.
> >
> > Till now, we were using ubiblock for mounting squashfs volumes and
> > completely got rid of mtd_block by configuring it as a loadable
> > module.
> > But, we also need to support OTA updates (Full, Incremental) on A/B
> > volumes using the same Android OTA framework.
> > https://source.android.com/docs/core/ota/nonab/block
> >
> > OTA update will be applied to the B (inactive) partition.
> > OTA updates prefer block based update over file based especially for
> > dm-verity enabled devices.
> >
> > Now, the problem is, on MTD we only have 2 options for block based
> > updates; ubi_block or mtd_block.
> > We cannot use ubiblock for OTA updates as it is read only.
> > For full update volume, we can use "ubiupdatevol" interface to
> > completely replace the volume content, but for partial or incremental
> > update we need to update only specific blocks and not entire
> > partitions.
> > Thus, we have to use the MTD_BLOCK (/dev/mtdblock) interface to
> > support block based OTA updates on UBI volumes.
> > Thus, during ota updates (only) we need to install the mtdblock
> > module, perform the update and then uninstall the module.
>
> Hi Pintu, how do you export the mtdblock device over on an UBI volume?
> By the gluebi?
> >
Yes, we have the below configs enabled.
CONFIG_MTD_BLOCK=m
CONFIG_MTD_UBI_GLUEBI=m
CONFIG_MTD_UBI_BLOCK=y

______________________________________________________
Linux MTD discussion mailing list
http://lists.infradead.org/mailman/listinfo/linux-mtd/

Re: Block based OTA update needs mtdblock

[Pintu Agarwal]

Hi,

On Mon, 4 Nov 2024 at 21:31, Pintu Agarwal <pintu.ping@gmail.com> wrote:
>
> Hi All,
>
> For one of our automotive products we have the following configuration:
> QC chipset, arm64, Kernel-5.15, NAND Flash 1GB, A/B system, UBI
> volumes (squashfs, ubifs), DM-verity for rootfs (squashfs), simple
> busybox platform.
>
> For OTA updates we have a strong dependency with MTD_BLOCK.
>
> Till now, we were using ubiblock for mounting squashfs volumes and
> completely got rid of mtd_block by configuring it as a loadable
> module.
> But, we also need to support OTA updates (Full, Incremental) on A/B
> volumes using the same Android OTA framework.
> https://source.android.com/docs/core/ota/nonab/block
>
> OTA update will be applied to the B (inactive) partition.
> OTA updates prefer block based update over file based especially for
> dm-verity enabled devices.
>
> Now, the problem is, on MTD we only have 2 options for block based
> updates; ubi_block or mtd_block.
> We cannot use ubiblock for OTA updates as it is read only.
> For full update volume, we can use "ubiupdatevol" interface to
> completely replace the volume content, but for partial or incremental
> update we need to update only specific blocks and not entire
> partitions.
> Thus, we have to use the MTD_BLOCK (/dev/mtdblock) interface to
> support block based OTA updates on UBI volumes.
> Thus, during ota updates (only) we need to install the mtdblock
> module, perform the update and then uninstall the module.
>
> That means, we cannot completely get rid of MTD_BLOCK from our product
> especially for OTA use cases.
>
> Is this the only way, or do we have any other option to support OTA
> updates over UBI volumes ?
>
Restarting this thread again...
Any further comment on this ?

Did anybody used block based OTA update NAND A/B system without using
mtd_block ?
Since ubiblock is read-only, it seems there is no other way to perform
OTA update, if mtd_block is disabled.
Or, we need to make ubiblock also as read/write.


Thanks,
Pintu

______________________________________________________
Linux MTD discussion mailing list
http://lists.infradead.org/mailman/listinfo/linux-mtd/

Re: Block based OTA update needs mtdblock

[Miquel Raynal]

Hello,

On 20/11/2024 at 12:52:57 +0530, Pintu Agarwal <pintu.ping@gmail.com> wrote:

> Hi,
>
> On Mon, 4 Nov 2024 at 21:31, Pintu Agarwal <pintu.ping@gmail.com> wrote:
>>
>> Hi All,
>>
>> For one of our automotive products we have the following configuration:
>> QC chipset, arm64, Kernel-5.15, NAND Flash 1GB, A/B system, UBI
>> volumes (squashfs, ubifs), DM-verity for rootfs (squashfs), simple
>> busybox platform.
>>
>> For OTA updates we have a strong dependency with MTD_BLOCK.
>>
>> Till now, we were using ubiblock for mounting squashfs volumes and
>> completely got rid of mtd_block by configuring it as a loadable
>> module.
>> But, we also need to support OTA updates (Full, Incremental) on A/B
>> volumes using the same Android OTA framework.
>> https://source.android.com/docs/core/ota/nonab/block
>>
>> OTA update will be applied to the B (inactive) partition.
>> OTA updates prefer block based update over file based especially for
>> dm-verity enabled devices.
>>
>> Now, the problem is, on MTD we only have 2 options for block based
>> updates; ubi_block or mtd_block.
>> We cannot use ubiblock for OTA updates as it is read only.
>> For full update volume, we can use "ubiupdatevol" interface to
>> completely replace the volume content, but for partial or incremental
>> update we need to update only specific blocks and not entire
>> partitions.
>> Thus, we have to use the MTD_BLOCK (/dev/mtdblock) interface to
>> support block based OTA updates on UBI volumes.
>> Thus, during ota updates (only) we need to install the mtdblock
>> module, perform the update and then uninstall the module.
>>
>> That means, we cannot completely get rid of MTD_BLOCK from our product
>> especially for OTA use cases.
>>
>> Is this the only way, or do we have any other option to support OTA
>> updates over UBI volumes ?
>>
> Restarting this thread again...
> Any further comment on this ?
>
> Did anybody used block based OTA update NAND A/B system without using
> mtd_block ?

Not on my side, it is actually a good question. Richard, any ideas?

> Since ubiblock is read-only, it seems there is no other way to perform
> OTA update, if mtd_block is disabled.
> Or, we need to make ubiblock also as read/write.

Doesn't sound possible/a good idea.

I'm sorry for this unhelpful answer :-)

Miquèl

______________________________________________________
Linux MTD discussion mailing list
http://lists.infradead.org/mailman/listinfo/linux-mtd/

Re: Block based OTA update needs mtdblock

[Richard Weinberger]

----- Ursprüngliche Mail -----
> Von: "Miquel Raynal" <miquel.raynal@bootlin.com>
> An: "Pintu Agarwal" <pintu.ping@gmail.com>
> CC: "richard" <richard@nod.at>, "Vignesh Raghavendra" <vigneshr@ti.com>, "linux-mtd" <linux-mtd@lists.infradead.org>,
> "chengzhihao1" <chengzhihao1@huawei.com>, "linux-kernel" <linux-kernel@vger.kernel.org>
> Gesendet: Dienstag, 3. Dezember 2024 15:17:06
> Betreff: Re: Block based OTA update needs mtdblock

> Hello,
> 
> On 20/11/2024 at 12:52:57 +0530, Pintu Agarwal <pintu.ping@gmail.com> wrote:
> 
>> Hi,
>>
>> On Mon, 4 Nov 2024 at 21:31, Pintu Agarwal <pintu.ping@gmail.com> wrote:
>>>
>>> Hi All,
>>>
>>> For one of our automotive products we have the following configuration:
>>> QC chipset, arm64, Kernel-5.15, NAND Flash 1GB, A/B system, UBI
>>> volumes (squashfs, ubifs), DM-verity for rootfs (squashfs), simple
>>> busybox platform.
>>>
>>> For OTA updates we have a strong dependency with MTD_BLOCK.
>>>
>>> Till now, we were using ubiblock for mounting squashfs volumes and
>>> completely got rid of mtd_block by configuring it as a loadable
>>> module.
>>> But, we also need to support OTA updates (Full, Incremental) on A/B
>>> volumes using the same Android OTA framework.
>>> https://source.android.com/docs/core/ota/nonab/block
>>>
>>> OTA update will be applied to the B (inactive) partition.
>>> OTA updates prefer block based update over file based especially for
>>> dm-verity enabled devices.
>>>
>>> Now, the problem is, on MTD we only have 2 options for block based
>>> updates; ubi_block or mtd_block.
>>> We cannot use ubiblock for OTA updates as it is read only.
>>> For full update volume, we can use "ubiupdatevol" interface to
>>> completely replace the volume content, but for partial or incremental
>>> update we need to update only specific blocks and not entire
>>> partitions.
>>> Thus, we have to use the MTD_BLOCK (/dev/mtdblock) interface to
>>> support block based OTA updates on UBI volumes.
>>> Thus, during ota updates (only) we need to install the mtdblock
>>> module, perform the update and then uninstall the module.
>>>
>>> That means, we cannot completely get rid of MTD_BLOCK from our product
>>> especially for OTA use cases.
>>>
>>> Is this the only way, or do we have any other option to support OTA
>>> updates over UBI volumes ?
>>>
>> Restarting this thread again...
>> Any further comment on this ?
>>
>> Did anybody used block based OTA update NAND A/B system without using
>> mtd_block ?
> 
> Not on my side, it is actually a good question. Richard, any ideas?

What about using ubiupdatevol?

Thanks,
//richard

______________________________________________________
Linux MTD discussion mailing list
http://lists.infradead.org/mailman/listinfo/linux-mtd/

Re: Block based OTA update needs mtdblock

[Pintu Agarwal]

Hello Richard,

On Tue, 3 Dec 2024 at 21:38, Richard Weinberger <richard@nod.at> wrote:
>
> ----- Ursprüngliche Mail -----
> > Von: "Miquel Raynal" <miquel.raynal@bootlin.com>
> > An: "Pintu Agarwal" <pintu.ping@gmail.com>
> > CC: "richard" <richard@nod.at>, "Vignesh Raghavendra" <vigneshr@ti.com>, "linux-mtd" <linux-mtd@lists.infradead.org>,
> > "chengzhihao1" <chengzhihao1@huawei.com>, "linux-kernel" <linux-kernel@vger.kernel.org>
> > Gesendet: Dienstag, 3. Dezember 2024 15:17:06
> > Betreff: Re: Block based OTA update needs mtdblock
>
> > Hello,
> >
> > On 20/11/2024 at 12:52:57 +0530, Pintu Agarwal <pintu.ping@gmail.com> wrote:
> >
> >> Hi,
> >>
> >> On Mon, 4 Nov 2024 at 21:31, Pintu Agarwal <pintu.ping@gmail.com> wrote:
> >>>
> >>> Hi All,
> >>>
> >>> For one of our automotive products we have the following configuration:
> >>> QC chipset, arm64, Kernel-5.15, NAND Flash 1GB, A/B system, UBI
> >>> volumes (squashfs, ubifs), DM-verity for rootfs (squashfs), simple
> >>> busybox platform.
> >>>
> >>> For OTA updates we have a strong dependency with MTD_BLOCK.
> >>>
> >>> Till now, we were using ubiblock for mounting squashfs volumes and
> >>> completely got rid of mtd_block by configuring it as a loadable
> >>> module.
> >>> But, we also need to support OTA updates (Full, Incremental) on A/B
> >>> volumes using the same Android OTA framework.
> >>> https://source.android.com/docs/core/ota/nonab/block
> >>>
> >>> OTA update will be applied to the B (inactive) partition.
> >>> OTA updates prefer block based update over file based especially for
> >>> dm-verity enabled devices.
> >>>
> >>> Now, the problem is, on MTD we only have 2 options for block based
> >>> updates; ubi_block or mtd_block.
> >>> We cannot use ubiblock for OTA updates as it is read only.
> >>> For full update volume, we can use "ubiupdatevol" interface to
> >>> completely replace the volume content, but for partial or incremental
> >>> update we need to update only specific blocks and not entire
> >>> partitions.
> >>> Thus, we have to use the MTD_BLOCK (/dev/mtdblock) interface to
> >>> support block based OTA updates on UBI volumes.
> >>> Thus, during ota updates (only) we need to install the mtdblock
> >>> module, perform the update and then uninstall the module.
> >>>
> >>> That means, we cannot completely get rid of MTD_BLOCK from our product
> >>> especially for OTA use cases.
> >>>
> >>> Is this the only way, or do we have any other option to support OTA
> >>> updates over UBI volumes ?
> >>>
> >> Restarting this thread again...
> >> Any further comment on this ?
> >>
> >> Did anybody used block based OTA update NAND A/B system without using
> >> mtd_block ?
> >
> > Not on my side, it is actually a good question. Richard, any ideas?
>
> What about using ubiupdatevol?
>
As I explained above, ubiupdatevol actually performs a full volume
update, right ?
I think the partial/incremental update is not possible with this. Once
the update is triggered it should finish fully.
But for OTA updates (Delta/Incremental), only the delta changes can
also be applied.
Please refer to this [1] about updates mechanism.
[1] https://medium.com/@yigitpirildak/android-ota-update-mechanism-ecc8f6f69f71
[2] https://source.android.com/docs/core/ota/tools#:~:text=An%20incremental%20update%20is%20an,need%20to%20include%20unchanged%20files.

On our NAND target we support OTA updates, including incremental ota updates.
But without mtdblock the incremental OTA update (on A/B system) will not work.
This is the main issue.
For full updates on ubi volumes we are already using the
"ubiupdatevol" mechanism.


Thanks,
Pintu

______________________________________________________
Linux MTD discussion mailing list
http://lists.infradead.org/mailman/listinfo/linux-mtd/

Re: Block based OTA update needs mtdblock

[Richard Weinberger]

----- Ursprüngliche Mail -----
> Von: "Pintu Agarwal" <pintu.ping@gmail.com>
>> >> Did anybody used block based OTA update NAND A/B system without using
>> >> mtd_block ?
>> >
>> > Not on my side, it is actually a good question. Richard, any ideas?
>>
>> What about using ubiupdatevol?
>>
> As I explained above, ubiupdatevol actually performs a full volume
> update, right ?
> I think the partial/incremental update is not possible with this. Once
> the update is triggered it should finish fully.
> But for OTA updates (Delta/Incremental), only the delta changes can
> also be applied.
> Please refer to this [1] about updates mechanism.
> [1] https://medium.com/@yigitpirildak/android-ota-update-mechanism-ecc8f6f69f71
> [2]
> https://source.android.com/docs/core/ota/tools#:~:text=An%20incremental%20update%20is%20an,need%20to%20include%20unchanged%20files.
> 
> On our NAND target we support OTA updates, including incremental ota updates.
> But without mtdblock the incremental OTA update (on A/B system) will not work.
> This is the main issue.
> For full updates on ubi volumes we are already using the
> "ubiupdatevol" mechanism.

Well, for dynamic volumes you can use the UBI_IOCEBCH ioctl to replace LEBs.
So, incremental updates are doable too.

Thanks,
//richard

______________________________________________________
Linux MTD discussion mailing list
http://lists.infradead.org/mailman/listinfo/linux-mtd/

Re: Block based OTA update needs mtdblock

[Pintu Agarwal]

Hi Richard,

On Wed, 4 Dec 2024 at 15:46, Richard Weinberger <richard@nod.at> wrote:
>
> ----- Ursprüngliche Mail -----
> > Von: "Pintu Agarwal" <pintu.ping@gmail.com>
> >> >> Did anybody used block based OTA update NAND A/B system without using
> >> >> mtd_block ?
> >> >
> >> > Not on my side, it is actually a good question. Richard, any ideas?
> >>
> >> What about using ubiupdatevol?
> >>
> > As I explained above, ubiupdatevol actually performs a full volume
> > update, right ?
> > I think the partial/incremental update is not possible with this. Once
> > the update is triggered it should finish fully.
> > But for OTA updates (Delta/Incremental), only the delta changes can
> > also be applied.
> > Please refer to this [1] about updates mechanism.
> > [1] https://medium.com/@yigitpirildak/android-ota-update-mechanism-ecc8f6f69f71
> > [2]
> > https://source.android.com/docs/core/ota/tools#:~:text=An%20incremental%20update%20is%20an,need%20to%20include%20unchanged%20files.
> >
> > On our NAND target we support OTA updates, including incremental ota updates.
> > But without mtdblock the incremental OTA update (on A/B system) will not work.
> > This is the main issue.
> > For full updates on ubi volumes we are already using the
> > "ubiupdatevol" mechanism.
>
> Well, for dynamic volumes you can use the UBI_IOCEBCH ioctl to replace LEBs.
> So, incremental updates are doable too.
>
Oh I see. This seems useful. I will try it and confirm if it works for us.
Thank you so much for this.

Even the header file (/usr/include/mtd/ubi-user.h) mentions these details.
{{{
 * Atomic logical eraseblock change
 * ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 *
 * Atomic logical eraseblock change operation is called using the %UBI_IOCEBCH
 * ioctl command of the corresponding UBI volume character device. A pointer to
 * a &struct ubi_leb_change_req object has to be passed to the ioctl. Then the
 * user is expected to write the requested amount of bytes (similarly to what
 * should be done in case of the "volume update" ioctl).
...
/* Atomic LEB change command */
#define UBI_IOCEBCH _IOW(UBI_VOL_IOC_MAGIC, 2, __s32)

* @lnum: logical eraseblock number to change
struct ubi_leb_change_req {
        __s32 lnum;
        __s32 bytes;
        __s8  dtype; /* obsolete, do not use! */
        __s8  padding[7];
} __attribute__((packed));

}}}
This requires the logical erase-block number to be known.
I am wondering how to get the LEB number from user space.
I need to explore further..

Thank you so much...

Regards,
Pintu

______________________________________________________
Linux MTD discussion mailing list
http://lists.infradead.org/mailman/listinfo/linux-mtd/