Create uncompressed rootfs

Create uncompressed rootfs

[Roberto]

[-- Attachment #1: Type: text/plain, Size: 860 bytes --]

Hi,

 

I apologise if this is not the right channel for asking the following
question. If so, please advise me on a more appropriate support channel.

 

I am looking for the quickest workflow for debugging a recipe. I basically
need to modify a recipe, deploy the entire distribution to the target board
and test it.

 

have come to the conclusion that the quickest way is to boot the target
board through TFTP for the kernel and then mount the rootfs via NFS.

 

In my view, the only issue of this workflow is that when I bitbake the image
it creates a compressed rootfs that then I have to uncompress in order to
provide the rootfs to the target board via NFS.

 

Is there any way to avoid bitbake compressing the rootfs but rather having
it uncompressed in some folders under tmp/deploy ?

 

Regards,

 

Roberto


[-- Attachment #2: Type: text/html, Size: 2911 bytes --]

Re: Create uncompressed rootfs

[Marco Cavallini]

You could mount the .ext3 image on a partition using -o loop
or find/customize a different solution using IMAGE_TYPES
(i.e. when IMAGE_FSTYPES contains "live")
http://www.yoctoproject.org/docs/1.8/ref-manual/ref-manual.html
--
Marco Cavallini | KOAN sas | Bergamo - Italia
embedded and real-time software engineering
http://www.KoanSoftware.com

2015-11-02 6:20 GMT+01:00 Roberto <ramatekd@gmail.com>:
> Hi,
>
>
>
> I apologise if this is not the right channel for asking the following
> question. If so, please advise me on a more appropriate support channel.
>
>
>
> I am looking for the quickest workflow for debugging a recipe. I basically
> need to modify a recipe, deploy the entire distribution to the target board
> and test it.
>
>
>
> have come to the conclusion that the quickest way is to boot the target
> board through TFTP for the kernel and then mount the rootfs via NFS.
>
>
>
> In my view, the only issue of this workflow is that when I bitbake the image
> it creates a compressed rootfs that then I have to uncompress in order to
> provide the rootfs to the target board via NFS.
>
>
>
> Is there any way to avoid bitbake compressing the rootfs but rather having
> it uncompressed in some folders under tmp/deploy ?
>
>
>
> Regards,
>
>
>
> Roberto
>
>
> --
> _______________________________________________
> yocto mailing list
> yocto@yoctoproject.org
> https://lists.yoctoproject.org/listinfo/yocto
>

Re: Create uncompressed rootfs

[Andre McCurdy]

On Sun, Nov 1, 2015 at 9:20 PM, Roberto <ramatekd@gmail.com> wrote:
> Hi,
>

Hi Roberto,

>
> I apologise if this is not the right channel for asking the following
> question. If so, please advise me on a more appropriate support channel.
>
>
>
> I am looking for the quickest workflow for debugging a recipe. I basically
> need to modify a recipe, deploy the entire distribution to the target board
> and test it.
>
>
>
> have come to the conclusion that the quickest way is to boot the target
> board through TFTP for the kernel and then mount the rootfs via NFS.
>
>
>
> In my view, the only issue of this workflow is that when I bitbake the image
> it creates a compressed rootfs that then I have to uncompress in order to
> provide the rootfs to the target board via NFS.
>
>
>
> Is there any way to avoid bitbake compressing the rootfs but rather having
> it uncompressed in some folders under tmp/deploy ?

See the "IMAGE_TYPES" variable for a list of rootfs types which are
supported. There's support for creating an uncompressed .tar file, but
I don't see any support for creating a rootfs directory under
tmp/deploy.

Depending on your work flow there are a few different solutions
though. You could extend
openembedded-core/meta/classes/image_types.bbclass to do what you want
(e.g. define a new image type or hack "IMAGE_CMD_tar" so that it also
untars rootfs.tar right after creating it). Or you could ignore OE's
image creation and manually copy or rsync your image's rootfs ( under
tmp/work/<MACHINE>.../<IMAGE>/.../rootfs ) to the directory exported
by your NFS server. Note that you probably shouldn't export
tmp/work/.../rootfs directly since when the target boots it will
create files owned by root, which will cause problems later if you try
to rebuild or modify the rootfs.

>
> Regards,
>
>
>
> Roberto
>
>
> --
> _______________________________________________
> yocto mailing list
> yocto@yoctoproject.org
> https://lists.yoctoproject.org/listinfo/yocto
>

Re: Create uncompressed rootfs

[Anders Darander]

* Andre McCurdy <armccurdy@gmail.com> [151102 20:35]:

> See the "IMAGE_TYPES" variable for a list of rootfs types which are
> supported. There's support for creating an uncompressed .tar file, but
> I don't see any support for creating a rootfs directory under
> tmp/deploy.

No, the usual workflow here is to unpack the tar'ed rootfs at a suitable
location.

> Depending on your work flow there are a few different solutions
> though. You could extend
> openembedded-core/meta/classes/image_types.bbclass to do what you want
> (e.g. define a new image type or hack "IMAGE_CMD_tar" so that it also
> untars rootfs.tar right after creating it). 

Well, there's some issues with this approach. In order to uncompress the
tarball and be able to set owner, group, and permissions on all files,
you need to untar the rootfs with root privileges. The same is true when
it comes to creating device nodes.

Thus, I think it really do make a lot more sense to untar the rootfs in
an external step, which very well could be scripted.

> Or you could ignore OE's
> image creation and manually copy or rsync your image's rootfs ( under
> tmp/work/<MACHINE>.../<IMAGE>/.../rootfs ) to the directory exported
> by your NFS server. Note that you probably shouldn't export
> tmp/work/.../rootfs directly since when the target boots it will
> create files owned by root, which will cause problems later if you try
> to rebuild or modify the rootfs.

No, avoid doing this. The reasons are the same as why you shouldn't
unpack the tarball using a new image type; you're not allowed to set the
correct owner, group, and permissions on files, nor are you allowed to
create device nodes.

Cheers,
Anders

-- 
Anders Darander
ChargeStorm AB / eStorm AB

Re: Create uncompressed rootfs

[Andre McCurdy]

On Mon, Nov 2, 2015 at 11:05 PM, Anders Darander <anders@chargestorm.se> wrote:
> * Andre McCurdy <armccurdy@gmail.com> [151102 20:35]:
>
>> See the "IMAGE_TYPES" variable for a list of rootfs types which are
>> supported. There's support for creating an uncompressed .tar file, but
>> I don't see any support for creating a rootfs directory under
>> tmp/deploy.
>
> No, the usual workflow here is to unpack the tar'ed rootfs at a suitable
> location.

Indeed. I don't think it's what Roberto was asking for though...

>> Depending on your work flow there are a few different solutions
>> though. You could extend
>> openembedded-core/meta/classes/image_types.bbclass to do what you want
>> (e.g. define a new image type or hack "IMAGE_CMD_tar" so that it also
>> untars rootfs.tar right after creating it).
>
> Well, there's some issues with this approach. In order to uncompress the
> tarball and be able to set owner, group, and permissions on all files,
> you need to untar the rootfs with root privileges. The same is true when
> it comes to creating device nodes.

Enabling CONFIG_DEVTMPFS in the kernel is pretty standard, so for most
people there are no device nodes in the rootfs tarfile and /dev is an
empty directory.

Extracting rootfs tarfiles as an unprivileged user has always worked
fine for me. Do you have a specific example where root privileges are
required?

> Thus, I think it really do make a lot more sense to untar the rootfs in
> an external step, which very well could be scripted.
>
>> Or you could ignore OE's
>> image creation and manually copy or rsync your image's rootfs ( under
>> tmp/work/<MACHINE>.../<IMAGE>/.../rootfs ) to the directory exported
>> by your NFS server. Note that you probably shouldn't export
>> tmp/work/.../rootfs directly since when the target boots it will
>> create files owned by root, which will cause problems later if you try
>> to rebuild or modify the rootfs.
>
> No, avoid doing this. The reasons are the same as why you shouldn't
> unpack the tarball using a new image type; you're not allowed to set the
> correct owner, group, and permissions on files, nor are you allowed to
> create device nodes.
>
> Cheers,
> Anders
>
> --
> Anders Darander
> ChargeStorm AB / eStorm AB
> --
> _______________________________________________
> yocto mailing list
> yocto@yoctoproject.org
> https://lists.yoctoproject.org/listinfo/yocto

Re: Create uncompressed rootfs

[Anders Darander]

* Andre McCurdy <armccurdy@gmail.com> [151103 11:34]:

> On Mon, Nov 2, 2015 at 11:05 PM, Anders Darander <anders@chargestorm.se> wrote:
> > * Andre McCurdy <armccurdy@gmail.com> [151102 20:35]:

> >> See the "IMAGE_TYPES" variable for a list of rootfs types which are
> >> supported. There's support for creating an uncompressed .tar file, but
> >> I don't see any support for creating a rootfs directory under
> >> tmp/deploy.

> > No, the usual workflow here is to unpack the tar'ed rootfs at a suitable
> > location.

> Indeed. I don't think it's what Roberto was asking for though...

Well, sure, that wasn't what Roberto explicitly was asking for, though
that's a safer way to do what he's after...

After all, if you never learn about best practises, you'll never know...

> >> Depending on your work flow there are a few different solutions
> >> though. You could extend
> >> openembedded-core/meta/classes/image_types.bbclass to do what you want
> >> (e.g. define a new image type or hack "IMAGE_CMD_tar" so that it also
> >> untars rootfs.tar right after creating it).

> > Well, there's some issues with this approach. In order to uncompress the
> > tarball and be able to set owner, group, and permissions on all files,
> > you need to untar the rootfs with root privileges. The same is true when
> > it comes to creating device nodes.

> Enabling CONFIG_DEVTMPFS in the kernel is pretty standard, so for most
> people there are no device nodes in the rootfs tarfile and /dev is an
> empty directory.

Sure, devtmpfs is pretty much standard, though it could nonetheless be
good to know about.

> Extracting rootfs tarfiles as an unprivileged user has always worked
> fine for me. Do you have a specific example where root privileges are
> required?

Well, everything that requires a specific user and permissions. When it
comes to permissions it's likely most if you need to setuid or setgid on
binaries / files / directories.

When you unpack as an unpriviliged user, every file will belong to you.

Thus, the recommended (and safe) way to do this, is to unpack as a
priviliged user.

Cheers,
Anders

-- 
Anders Darander
ChargeStorm AB / eStorm AB

Re: Create uncompressed rootfs

[Andre McCurdy]

On Tue, Nov 3, 2015 at 3:54 AM, Anders Darander <anders@chargestorm.se> wrote:
> * Andre McCurdy <armccurdy@gmail.com> [151103 11:34]:
>
>> On Mon, Nov 2, 2015 at 11:05 PM, Anders Darander <anders@chargestorm.se> wrote:
>> > * Andre McCurdy <armccurdy@gmail.com> [151102 20:35]:
>
>> >> See the "IMAGE_TYPES" variable for a list of rootfs types which are
>> >> supported. There's support for creating an uncompressed .tar file, but
>> >> I don't see any support for creating a rootfs directory under
>> >> tmp/deploy.
>
>> > No, the usual workflow here is to unpack the tar'ed rootfs at a suitable
>> > location.
>
>> Indeed. I don't think it's what Roberto was asking for though...
>
> Well, sure, that wasn't what Roberto explicitly was asking for, though
> that's a safer way to do what he's after...
>
> After all, if you never learn about best practises, you'll never know...
>
>> >> Depending on your work flow there are a few different solutions
>> >> though. You could extend
>> >> openembedded-core/meta/classes/image_types.bbclass to do what you want
>> >> (e.g. define a new image type or hack "IMAGE_CMD_tar" so that it also
>> >> untars rootfs.tar right after creating it).
>
>> > Well, there's some issues with this approach. In order to uncompress the
>> > tarball and be able to set owner, group, and permissions on all files,
>> > you need to untar the rootfs with root privileges. The same is true when
>> > it comes to creating device nodes.
>
>> Enabling CONFIG_DEVTMPFS in the kernel is pretty standard, so for most
>> people there are no device nodes in the rootfs tarfile and /dev is an
>> empty directory.
>
> Sure, devtmpfs is pretty much standard, though it could nonetheless be
> good to know about.
>
>> Extracting rootfs tarfiles as an unprivileged user has always worked
>> fine for me. Do you have a specific example where root privileges are
>> required?
>
> Well, everything that requires a specific user and permissions. When it
> comes to permissions it's likely most if you need to setuid or setgid on
> binaries / files / directories.
>
> When you unpack as an unpriviliged user, every file will belong to you.
>
> Thus, the recommended (and safe) way to do this, is to unpack as a
> priviliged user.

Yes, you're right. If you everything on the target runs as root then
losing setuid flags isn't going to matter much, but it could be a
problem if the target needs to support non-privileged users.

> Cheers,
> Anders
>
> --
> Anders Darander
> ChargeStorm AB / eStorm AB
> --
> _______________________________________________
> yocto mailing list
> yocto@yoctoproject.org
> https://lists.yoctoproject.org/listinfo/yocto