[Buildroot] user question about ubi nand

[Buildroot] user question about ubi nand

[Steve Calfee]

Ubi seems very well supported in buildroot and linux. The problem is
figuring out how to use it.

I have created two nand partitions, one rootfs and one data. I would
like the rootfs to be readonly and data to be r/w. After tons of
searches I have configured the rootfs and stored the buildroot built
image in it.

My question is how do I move r/w areas from rootfs into the r/w
partition. Examples are /etc/ files /run files which start as /var/run
and finally application files like mysql databases which are in
/usr/share/. I assume the post scripts can be used, but I cannot find
examples. In post scripts how do I set permissions/user/group fields.
How do I link to a mounted partition.

http://free-electrons.com/blog/creating-flashing-ubi-ubifs-images/ has
lots of info. But say I want a custom ubi.cfg file, where do I put it
or how do i menuconfig it? Say I want 2 partitions, one ro and one rw
I assume I need a special cfg file. Ubi asks for lots of magic
numbers, how do I get them - such as erasesize, subpagesize, offsets
etc? I assume defaults are good, but how can I know. Typical Linux
stuff, however these thing are knowable, how do I find it out (via web
searches etc). How do I do set the config file in BR2_EXTERNAL?

All info I find is schizophrenic. Net searches say how to do things
like this in uboot and linux. Sometimes in the same doc paragraph. It
is never clear which instruction matches which of uboot and linux. For
buildroot the best is stuff mainly done in the build, but something
must be done on the target. What is the recommended procedure for mass
building systems?

What if the nand and rootfs is bigger than ram? How do you update it in uboot?

Steve

[Buildroot] user question about ubi nand

[Jörg Krause]

Hi Steve,

On Di, 2016-02-23 at 19:34 -0800, Steve Calfee wrote:
> Ubi seems very well supported in buildroot and linux. The problem is
> figuring out how to use it.
> 
> I have created two nand partitions, one rootfs and one data. I would
> like the rootfs to be readonly and data to be r/w. After tons of
> searches I have configured the rootfs and stored the buildroot built
> image in it.
> 
> My question is how do I move r/w areas from rootfs into the r/w
> partition. Examples are /etc/ files /run files which start as
> /var/run
> and finally application files like mysql databases which are in
> /usr/share/. I assume the post scripts can be used, but I cannot find
> examples. In post scripts how do I set permissions/user/group fields.
> How do I link to a mounted partition.
> 
> http://free-electrons.com/blog/creating-flashing-ubi-ubifs-images/
> has
> lots of info. But say I want a custom ubi.cfg file, where do I put it
> or how do i menuconfig it? Say I want 2 partitions, one ro and one rw
> I assume I need a special cfg file. Ubi asks for lots of magic
> numbers, how do I get them - such as erasesize, subpagesize, offsets
> etc? I assume defaults are good, but how can I know. Typical Linux
> stuff, however these thing are knowable, how do I find it out (via
> web
> searches etc). How do I do set the config file in BR2_EXTERNAL?
> 
> All info I find is schizophrenic. Net searches say how to do things
> like this in uboot and linux. Sometimes in the same doc paragraph. It
> is never clear which instruction matches which of uboot and linux.
> For
> buildroot the best is stuff mainly done in the build, but something
> must be done on the target. What is the recommended procedure for
> mass
> building systems?
> 
> What if the nand and rootfs is bigger than ram? How do you update it
> in uboot?

These are a lot of questions :-)

I guess you already read the MTD docs [1]. It has a lot of useful
information about NAND, UBI, and UBIFS...

I use Buildroot to build a ro rootfs, a rw overlayfs and a rw data
partition. I configured to BR to create a UBIFS rootfs
(BR2_TARGET_ROOTFS_UBIFS). The parameters like LEB size depends on your
NAND device and partition layout (see [1]).

The data UBIFS and UBI image is created in the post-image script.
Again, the parameter values for the mkfs.ubifs and ubinize tools
depends on your NAND device and partition layout.

To append the data parition to your filesystem you'll need to add an
entry to fstab, eg.
"ubi1:data	/data	ubifs	defaults	0	0"

You can use fstab to move parts of your system to another partition,
tool.

I know it's like a big puzzle to get all pieces together. It took me
some while to go to all the available documentation thoroughly, take
one step-by-step and adapt it all to my needs.

Good luck!

[1]?http://www.linux-mtd.infradead.org/doc/general.html

Best regards
J?rg Krause

[Buildroot] user question about ubi nand

[Steve Calfee]

On Wed, Feb 24, 2016 at 1:49 AM, J?rg Krause
<joerg.krause@embedded.rocks> wrote:
> Hi Steve,
>
> On Di, 2016-02-23 at 19:34 -0800, Steve Calfee wrote:
>> Ubi seems very well supported in buildroot and linux. The problem is
>> figuring out how to use it.
>>

>
> These are a lot of questions :-)
>
> I guess you already read the MTD docs [1]. It has a lot of useful
> information about NAND, UBI, and UBIFS...
>
Yes, in fact I manually have built my fs. What I am trying to do is
figure out the approved, "buildroot way". Sorry about the number of
questions.

> I use Buildroot to build a ro rootfs, a rw overlayfs and a rw data
> partition. I configured to BR to create a UBIFS rootfs
> (BR2_TARGET_ROOTFS_UBIFS). The parameters like LEB size depends on your
> NAND device and partition layout (see [1]).

So I just have to know, there is no tool that probes for that information?

>
> The data UBIFS and UBI image is created in the post-image script.
> Again, the parameter values for the mkfs.ubifs and ubinize tools
> depends on your NAND device and partition layout.
>

Yes and I found in menuconfig the multiple places where programs and
params are set. And the linux-menuconfig must also have some ubi
related things set.

> To append the data parition to your filesystem you'll need to add an
> entry to fstab, eg.
> "ubi1:data      /data   ubifs   defaults        0       0"
>
> You can use fstab to move parts of your system to another partition,
> tool.
>

Thanks, that is a valuable tip. How do people ordinarily alter fstab?
I can put the entire file in the skeleton overlay or I could do a
patch in a post script. Which is better?

http://free-electrons.com/blog/creating-flashing-ubi-ubifs-images/ has
good information, but not really how to integrate the techniques into
buildroot. Specifically how to use their suggested

[rootfs_volume]
mode=ubi
image=rootfs.ubifs
vol_id=1
vol_type=static
vol_name=rootfs
vol_alignment=1

[rwdata_volume]
mode=ubi
image=data.ubifs
vol_id=2
vol_type=dynamic
vol_name=data
vol_alignment=1
vol_flags=autoresize

Which is different from the ubinize.cfg built into buildroot
[ubifs]
mode=ubi
vol_id=0
vol_type=dynamic
vol_name=rootfs
vol_alignment=1
vol_flags=autoresize
image=BR2_ROOTFS_UBIFS_PATH

So from a buildroot BR2_EXTERNAL mechanism perspective what is a
possible/best way to customize the cfg file?

Again too many questions. I know others will google this in the
future, so I and others will appreciate any info you have.

Thanks, Steve

[Buildroot] user question about ubi nand

[Arnout Vandecappelle]

On 02/24/16 04:34, Steve Calfee wrote:
> Ubi seems very well supported in buildroot and linux. The problem is
> figuring out how to use it.
> 
> I have created two nand partitions, one rootfs and one data. 

 That's already a bad start, because this way you reduce the wear levelling
opportunities. The best way to use UBI is to put everything in a single
partition, and create several UBI volumes within that partition. The canonical
flash layout is:

4   EB SPL (=pre-bootloader)
16  EB bootloader
8   EB bootloader environment (only relevant for U-Boot I guess)
the rest UBI
  2 volumes kernel
  2 volumes DTB
  2 volumes rootfs
  1 volume  rwdata
 (1 volume  per-device ro data is also often needed)

kernel, DTB and rootfs have two copies for upgrade, i.e. you write to a new
volume while keeping the currently running version alive.


> I would
> like the rootfs to be readonly and data to be r/w. After tons of
> searches I have configured the rootfs and stored the buildroot built
> image in it.
> 
> My question is how do I move r/w areas from rootfs into the r/w
> partition. Examples are /etc/ files /run files which start as /var/run
> and finally application files like mysql databases which are in
> /usr/share/.

 For this, there is no standard way because there are several ways to do it.

- Use some kind of overlayfs so that your readonly rootfs acts as if it's wriitable.

- Move the files that should be writeable to the rw partition and create
symlinks for them.

- Move the files that should be writeable to tmpfs and generate them at boot time.

- Configure packages to search for the writable files in the rw partition
instead of their usual location.


> I assume the post scripts can be used, but I cannot find
> examples. In post scripts how do I set permissions/user/group fields.

 For ownership and permissions, there is BR2_ROOTFS_DEVICE_TABLE. post-build
script doesn't help because you have to do that as root, and the post-build
script is executed as an ordinary user.

> How do I link to a mounted partition.
> 
> http://free-electrons.com/blog/creating-flashing-ubi-ubifs-images/ has
> lots of info. But say I want a custom ubi.cfg file, where do I put it
> or how do i menuconfig it? 

BR2_TARGET_ROOTFS_UBI_USE_CUSTOM_CONFIG

> Say I want 2 partitions, one ro and one rw
> I assume I need a special cfg file. 

 Buildroot will not help you to create the rw partition. In many systems, it is
created at boot time if it doesn't exist yet. That's also a simple way to do a
factory reset: just remove the rw volume. But you can also create it at build
time (using mkubifs) in the post-build script, and specify it in the custom ubi.cfg.


> Ubi asks for lots of magic
> numbers, how do I get them - such as erasesize, subpagesize, offsets

 On the device, you can use mtdinfo to get these. But unless you're using an
off-the-shelf board, you already had to specify these values in the device tree.

> etc? I assume defaults are good, but how can I know. Typical Linux
> stuff, however these thing are knowable, how do I find it out (via web
> searches etc). 


> How do I do set the config file in BR2_EXTERNAL?

 I don't see how this is related to ubi... Actually I don't understand the
question even.

> 
> All info I find is schizophrenic. Net searches say how to do things
> like this in uboot and linux. Sometimes in the same doc paragraph. It
> is never clear which instruction matches which of uboot and linux. For
> buildroot the best is stuff mainly done in the build, but something
> must be done on the target. What is the recommended procedure for mass
> building systems?

 You almost always need to do something at first boot, some kind of provisioning
e.g. to set MAC addresses. What I often see is that you build a specific
production image, flash that one in the factory (or put it on an SD card and
boot from that). This production image does some hardware tests, does
provisioning, and downloads the up-to-date application image to write to flash.
So it has a completely different configuration than the application image.

> 
> What if the nand and rootfs is bigger than ram? How do you update it in uboot?

 You don't want to do updates from the bootloader. You want to use swupdate,
though I'm not sure if it supports streaming upgrade. You also want to use a
recent kernel that support atomic volume rename, then you don't need any
involvement of the bootloader to do upgrades.

 Oh, and you probably want to hire someone like to figure all this out for you :-)


 Regards,
 Arnout

> 
> Steve
> _______________________________________________
> buildroot mailing list
> buildroot at busybox.net
> http://lists.busybox.net/mailman/listinfo/buildroot
> 


-- 
Arnout Vandecappelle                          arnout at mind be
Senior Embedded Software Architect            +32-16-286500
Essensium/Mind                                http://www.mind.be
G.Geenslaan 9, 3001 Leuven, Belgium           BE 872 984 063 RPR Leuven
LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle
GPG fingerprint:  7493 020B C7E3 8618 8DEC 222C 82EB F404 F9AC 0DDF

[Buildroot] user question about ubi nand

[Steve Calfee]

Hi Arnout,

On Wed, Feb 24, 2016 at 3:00 PM, Arnout Vandecappelle <arnout@mind.be> wrote:
> On 02/24/16 04:34, Steve Calfee wrote:
>> Ubi seems very well supported in buildroot and linux. The problem is
>> figuring out how to use it.
>>
>> I have created two nand partitions, one rootfs and one data.
>
>  That's already a bad start, because this way you reduce the wear levelling
> opportunities. The best way to use UBI is to put everything in a single
> partition, and create several UBI volumes within that partition.

Hmm, part the the reason to partition flash is so failures in one will
not mess up other partitions. I agree that r/w parts should be in one
ubi partition. I guess everything in one might work, if ubi is very
careful with updates.

 The canonical
> flash layout is:
>
> 4   EB SPL (=pre-bootloader)
> 16  EB bootloader
> 8   EB bootloader environment (only relevant for U-Boot I guess)
> the rest UBI
>   2 volumes kernel
>   2 volumes DTB
>   2 volumes rootfs
>   1 volume  rwdata
>  (1 volume  per-device ro data is also often needed)

Yes I have seen layouts like that. This is for a personal project, and
I don't have enough flash. If I have a problem I can always go back to
uboot and nfs boot the system for updates.

>
> kernel, DTB and rootfs have two copies for upgrade, i.e. you write to a new
> volume while keeping the currently running version alive.
>
>
>> I would
>> like the rootfs to be readonly and data to be r/w. After tons of
>> searches I have configured the rootfs and stored the buildroot built
>> image in it.
>>
>> My question is how do I move r/w areas from rootfs into the r/w
>> partition. Examples are /etc/ files /run files which start as /var/run
>> and finally application files like mysql databases which are in
>> /usr/share/.
>
>  For this, there is no standard way because there are several ways to do it.
>
> - Use some kind of overlayfs so that your readonly rootfs acts as if it's wriitable.
>
> - Move the files that should be writeable to the rw partition and create
> symlinks for them.
>
> - Move the files that should be writeable to tmpfs and generate them at boot time.
>
> - Configure packages to search for the writable files in the rw partition
> instead of their usual location.
>

The fstab idea and making a rw partition empty by default - inited by
programs that need it sounds good to me.

>
>> I assume the post scripts can be used, but I cannot find
>> examples. In post scripts how do I set permissions/user/group fields.
>
>  For ownership and permissions, there is BR2_ROOTFS_DEVICE_TABLE. post-build
> script doesn't help because you have to do that as root, and the post-build
> script is executed as an ordinary user.
>

Thanks, I will look into that, These options built into buildroot are
really great, but sometimes their existence and how to use them are
not clear to me.

>> How do I link to a mounted partition.
>>
>> http://free-electrons.com/blog/creating-flashing-ubi-ubifs-images/ has
>> lots of info. But say I want a custom ubi.cfg file, where do I put it
>> or how do i menuconfig it?
>
> BR2_TARGET_ROOTFS_UBI_USE_CUSTOM_CONFIG
>

Thanks, that config was hidden behind another one, and a little tricky
to find, but once I knew what I wanted it worked great.

>> Say I want 2 partitions, one ro and one rw
>> I assume I need a special cfg file.
>
>  Buildroot will not help you to create the rw partition. In many systems, it is
> created at boot time if it doesn't exist yet. That's also a simple way to do a
> factory reset: just remove the rw volume. But you can also create it at build
> time (using mkubifs) in the post-build script, and specify it in the custom ubi.cfg.
>
>
>> Ubi asks for lots of magic
>> numbers, how do I get them - such as erasesize, subpagesize, offsets
>
>  On the device, you can use mtdinfo to get these. But unless you're using an
> off-the-shelf board, you already had to specify these values in the device tree.
>
>> etc? I assume defaults are good, but how can I know. Typical Linux
>> stuff, however these thing are knowable, how do I find it out (via web
>> searches etc).
>

Yes it must be built into my device tree or something. I just wondered
how to find this info.

>
>> How do I do set the config file in BR2_EXTERNAL?
>
>  I don't see how this is related to ubi... Actually I don't understand the
> question even.
>

I was asking about the ubi configuration, I set the ubi custom config
to $(BR2_EXTERNAL)/myboard/myubi.cfg


>>
>> All info I find is schizophrenic. Net searches say how to do things
>> like this in uboot and linux. Sometimes in the same doc paragraph. It
>> is never clear which instruction matches which of uboot and linux. For
>> buildroot the best is stuff mainly done in the build, but something
>> must be done on the target. What is the recommended procedure for mass
>> building systems?
>
>  You almost always need to do something at first boot, some kind of provisioning
> e.g. to set MAC addresses. What I often see is that you build a specific
> production image, flash that one in the factory (or put it on an SD card and
> boot from that). This production image does some hardware tests, does
> provisioning, and downloads the up-to-date application image to write to flash.
> So it has a completely different configuration than the application image.
>
>>
>> What if the nand and rootfs is bigger than ram? How do you update it in uboot?
>
>  You don't want to do updates from the bootloader. You want to use swupdate,
> though I'm not sure if it supports streaming upgrade. You also want to use a
> recent kernel that support atomic volume rename, then you don't need any
> involvement of the bootloader to do upgrades.
>

Good tips, but my production volume will probably be 2 small systems.

>  Oh, and you probably want to hire someone like to figure all this out for you :-)
>
That's no fun. I am doing this on a personal project. I have worked on
NOR flash systems, so this time I am trying to work with nand and the
ubi layer. It's very educational to actually bring up a new system.

Thanks for the tips, Steve

[Buildroot] user question about ubi nand

[Arnout Vandecappelle]

On 02/25/16 22:14, Steve Calfee wrote:
> Hi Arnout,
> 
> On Wed, Feb 24, 2016 at 3:00 PM, Arnout Vandecappelle <arnout@mind.be> wrote:
>> On 02/24/16 04:34, Steve Calfee wrote:
>>> Ubi seems very well supported in buildroot and linux. The problem is
>>> figuring out how to use it.
>>>
>>> I have created two nand partitions, one rootfs and one data.
>>
>>  That's already a bad start, because this way you reduce the wear levelling
>> opportunities. The best way to use UBI is to put everything in a single
>> partition, and create several UBI volumes within that partition.
> 
> Hmm, part the the reason to partition flash is so failures in one will
> not mess up other partitions. I agree that r/w parts should be in one
> ubi partition. I guess everything in one might work, if ubi is very
> careful with updates.

 Well, it depends on where you put your trust, and what lifetime you want from
your product. If your device is supposed to last for many years, then you are
guaranteed to get a lot of bad blocks, which are unpredictably spread over the
partitions. If you have many devices, some of them will have all the bad blocks
concentrated in a single partition. So each partition has to have enough spare
space to take all the bad blocks. This is going to be particularly painful for
the small partitions. I've had projects where the size of the kernel partitions
had to be doubled to take into account bad blocks.

 UBI gets extremely rigorous testing, including power cut testing. So I would
trust UBI more than the datasheet of an el-cheapo flash device :-)


> 
>  The canonical
>> flash layout is:
>>
>> 4   EB SPL (=pre-bootloader)
>> 16  EB bootloader
>> 8   EB bootloader environment (only relevant for U-Boot I guess)
>> the rest UBI
>>   2 volumes kernel
>>   2 volumes DTB
>>   2 volumes rootfs
>>   1 volume  rwdata
>>  (1 volume  per-device ro data is also often needed)
> 
> Yes I have seen layouts like that. This is for a personal project, and
> I don't have enough flash. If I have a problem I can always go back to
> uboot and nfs boot the system for updates.

 Well, in that case the reliability is not an issue either :-)

[snip]
>>> I assume the post scripts can be used, but I cannot find
>>> examples. In post scripts how do I set permissions/user/group fields.
>>
>>  For ownership and permissions, there is BR2_ROOTFS_DEVICE_TABLE. post-build
>> script doesn't help because you have to do that as root, and the post-build
>> script is executed as an ordinary user.
>>
> 
> Thanks, I will look into that, These options built into buildroot are
> really great, but sometimes their existence and how to use them are
> not clear to me.

 If you have ideas of how it could be documented better, that would be great. It
is documented in chapter 9.5.1 but that's perhaps a bit hidden away.


> 
>>> How do I link to a mounted partition.
>>>
>>> http://free-electrons.com/blog/creating-flashing-ubi-ubifs-images/ has
>>> lots of info. But say I want a custom ubi.cfg file, where do I put it
>>> or how do i menuconfig it?
>>
>> BR2_TARGET_ROOTFS_UBI_USE_CUSTOM_CONFIG
>>
> 
> Thanks, that config was hidden behind another one, and a little tricky
> to find, but once I knew what I wanted it worked great.

 Ah yes, it makes sense from the perspective of the buildroot logic (buildroot
primarily builds a filesystem, and for ubifs you can optionally embed that into
a UBI image, and only in that case there is a config file possible). But from a
user it is not completel logical.

 We should perhaps restructure the filesystem creation a little into a part
about the actual filesystem and a part about making images out of the
filesystems. Because we now already have a few different ways of making images
that are not just filesystems (isohybrid, ubi). And in the defconfigs we use
genimage instead...

[snip]
>>> What if the nand and rootfs is bigger than ram? How do you update it in uboot?
>>
>>  You don't want to do updates from the bootloader. You want to use swupdate,
>> though I'm not sure if it supports streaming upgrade. You also want to use a
>> recent kernel that support atomic volume rename, then you don't need any
>> involvement of the bootloader to do upgrades.
>>
> 
> Good tips, but my production volume will probably be 2 small systems.

 Even so it's probably easiest to do nfsboot and flash from there. And even so,
using swupgrade is probably easier than writing your own flash script.


> 
>>  Oh, and you probably want to hire someone like to figure all this out for you :-)
>>
> That's no fun. I am doing this on a personal project. I have worked on
> NOR flash systems, so this time I am trying to work with nand and the
> ubi layer. It's very educational to actually bring up a new system.

 It is! But then you should consider it a challenge that it is difficult to find
information :-P


 Regards,
 Arnout

> 
> Thanks for the tips, Steve
> 


-- 
Arnout Vandecappelle                          arnout at mind be
Senior Embedded Software Architect            +32-16-286500
Essensium/Mind                                http://www.mind.be
G.Geenslaan 9, 3001 Leuven, Belgium           BE 872 984 063 RPR Leuven
LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle
GPG fingerprint:  7493 020B C7E3 8618 8DEC 222C 82EB F404 F9AC 0DDF

[Buildroot] user question about ubi nand

[Peter Korsgaard]

>>>>> "Arnout" == Arnout Vandecappelle <arnout@mind.be> writes:

Hi,

 >> Thanks, that config was hidden behind another one, and a little tricky
 >> to find, but once I knew what I wanted it worked great.

 >  Ah yes, it makes sense from the perspective of the buildroot logic (buildroot
 > primarily builds a filesystem, and for ubifs you can optionally embed that into
 > a UBI image, and only in that case there is a config file possible). But from a
 > user it is not completel logical.

 >  We should perhaps restructure the filesystem creation a little into a part
 > about the actual filesystem and a part about making images out of the
 > filesystems. Because we now already have a few different ways of making images
 > that are not just filesystems (isohybrid, ubi). And in the defconfigs we use
 > genimage instead...

FYI, I think it makes sense to add a genimage (post-)filesystem option,
now that we use it is in several defconfigs.

-- 
Bye, Peter Korsgaard