[Buildroot] How do folks add packages to an existing rootfs during development?

[Buildroot] How do folks add packages to an existing rootfs during development?

[Patrick Doyle]

I have read section 10.7 "Why doesn't Buildroot generate binary
packages (.deb, .ipkg...)?" in the Buildroot users manual, and I
understand and appreciate the answer to that question.

My question is slightly different, and it is based on the following scenario:

I am creating a buildroot environment for my custom product.  I have
generated the rootfs and deployed it to my development board.  Once I
have done that, I have customized a few things to my liking (e.g.
hostname, installed my custom application, generated OpenSSH keys,
etc...).  Now I discover that I also want to add another package to my
environment.

Option 1:
Rerun "make menuconfig; make"; generate a new rootfs; flash it on my
board; and redo the customizations.  (Most of the customizations will
get easier with time, as I roll them into my configuration, but some
of them, such as OpenSSH host keys, will just have to change each time
I do this).

Option 2:
Rerun "make menuconfig; make" and deploy just the new package.
Lather, rinse, repeat, until I've figured out what package sets make
the most sense.  Then do Option 1.

Which option do folks in this community choose in general?

--wpd

[Buildroot] How do folks add packages to an existing rootfs during development?

[Steve Calfee]

On Thu, Feb 11, 2016 at 7:23 AM, Patrick Doyle <wpdster@gmail.com> wrote:
> I have read section 10.7 "Why doesn't Buildroot generate binary
> packages (.deb, .ipkg...)?" in the Buildroot users manual, and I
> understand and appreciate the answer to that question.
>
> My question is slightly different, and it is based on the following scenario:
>
> I am creating a buildroot environment for my custom product.  I have
> generated the rootfs and deployed it to my development board.  Once I
> have done that, I have customized a few things to my liking (e.g.
> hostname, installed my custom application, generated OpenSSH keys,
> etc...).  Now I discover that I also want to add another package to my
> environment.
>
> Option 1:
> Rerun "make menuconfig; make"; generate a new rootfs; flash it on my
> board; and redo the customizations.  (Most of the customizations will
> get easier with time, as I roll them into my configuration, but some
> of them, such as OpenSSH host keys, will just have to change each time
> I do this).
>
> Option 2:
> Rerun "make menuconfig; make" and deploy just the new package.
> Lather, rinse, repeat, until I've figured out what package sets make
> the most sense.  Then do Option 1.
>
> Which option do folks in this community choose in general?
>

When I am doing development, if I have network on the device (even
wifi), I use nfs as both a boot device and as a rootfs. I put my
rootfs on the host in the tftpboot directory. So using your option 2 I
update/add/edit etc a package. Then I simply 'cp
output/target/bin/testapp /tftpboot/testing/bin/' The device does not
even have to be rebooted to use the new app.

If the package is slightly more complicated and might needs some
libraries etc, I just untar the whole rootfs right on top the nfs
destination, just like I do after any fresh build. This can even be
done while the target system is running.

Since there is no package manager you can mess things up and crash the
target. But heck it is development, right. So frequently it is
important to 'make clean; make' and reinstall and reboot the target.
If it still boots and runs you can safely proceed with other
development.

Regards, Steve

[Buildroot] How do folks add packages to an existing rootfs during development?

[Henrique Marks]

----- Mensagem original -----
> De: "Patrick Doyle" <wpdster@gmail.com>
> Para: buildroot at busybox.net
> Enviadas: Quinta-feira, 11 de fevereiro de 2016 13:23:32
> Assunto: [Buildroot] How do folks add packages to an existing rootfs during	development?

> I have read section 10.7 "Why doesn't Buildroot generate binary
> packages (.deb, .ipkg...)?" in the Buildroot users manual, and I
> understand and appreciate the answer to that question.
> 
> My question is slightly different, and it is based on the following scenario:
> 
> I am creating a buildroot environment for my custom product.  I have
> generated the rootfs and deployed it to my development board.  Once I
> have done that, I have customized a few things to my liking (e.g.
> hostname, installed my custom application, generated OpenSSH keys,
> etc...).  Now I discover that I also want to add another package to my
> environment.
> 
> Option 1:
> Rerun "make menuconfig; make"; generate a new rootfs; flash it on my
> board; and redo the customizations.  (Most of the customizations will
> get easier with time, as I roll them into my configuration, but some
> of them, such as OpenSSH host keys, will just have to change each time
> I do this).
> 
> Option 2:
> Rerun "make menuconfig; make" and deploy just the new package.
> Lather, rinse, repeat, until I've figured out what package sets make
> the most sense.  Then do Option 1.
> 
> Which option do folks in this community choose in general?
> 
> --wpd
> _______________________________________________
> buildroot mailing list
> buildroot at busybox.net
> http://lists.busybox.net/mailman/listinfo/buildroot

If possible, try to export the root filesystem via NFS. For this, the card must download the kernel via tftp. If available, the dtb file must also be downloaded via tftp. Then:

cd your_output_folder
sudo mount --bind images/ your_tftp_folder (/var/lib/tftpboot ?)

no filesystem linked in kernel, check this on buildroot menuconfig

[ ] initial RAM filesystem linked into linux kernel (Not checked)

Install and setup nfs-kernel-server package, create directory to be exported
$ mkdir /work/my-nfsboot

Add the following line in /etc/exports
$ /work/my-nfsboot *(rw,insecure,async,no_subtree_check,no_root_squash)

Restart nfs-kernel-server 
$ sudo /etc/init.d/nfs-kernel-server restart

$ sudo tar -zxvf output/images/rootfs.tar.gz -C /work/my-nfsboot

Now it is time to go to your card and configure your bootloader to load the kernel and dtb from tftp, for instance, and mount the rootfs via NFS (from work/my-nfsboot)

Some steps are not strictly necessary, but overall this is the fastest solution.

-- 
Dr. Henrique Marks
henrique.marks at datacom.ind.br
R. Am?rica, 1000 - Eldorado do Sul - RS
CEP: 92990-000 - Brasil
Fone: +55 51 3933 3000 - Ramal 3466

[Buildroot] How do folks add packages to an existing rootfs during development?

[Mauro Condarelli]

Hi Patrick,
comments below.

Il 11/02/2016 16:23, Patrick Doyle ha scritto:
> I have read section 10.7 "Why doesn't Buildroot generate binary
> packages (.deb, .ipkg...)?" in the Buildroot users manual, and I
> understand and appreciate the answer to that question.
>
> My question is slightly different, and it is based on the following scenario:
>
> I am creating a buildroot environment for my custom product.  I have
> generated the rootfs and deployed it to my development board.  Once I
> have done that, I have customized a few things to my liking (e.g.
> hostname, installed my custom application, generated OpenSSH keys,
> etc...).  Now I discover that I also want to add another package to my
> environment.
First of all I do structure my development dir with three subdirs:
buildroot, input and output.
This main subdir is also (usually) under git control, so I have a basic .gitignore containing:

buildroot
output

./buildroot is populated by normal "git clone ..."
./input contains:

./input/board/
./input/configs/
./input/overlay/
./input/package/
./input/patch/
./input/Config.in
./input/external.mk
./input/permissions.makedev
./input/users.table

Initial config is done with "cd buildroot && make O=../output BR2_EXTERNAL=../input yourboard_defconfig"

After that all configurations/compilations are done via: "(cd output && make whatever)"

> Option 1:
> Rerun "make menuconfig; make"; generate a new rootfs; flash it on my
> board; and redo the customizations.  (Most of the customizations will
> get easier with time, as I roll them into my configuration, but some
> of them, such as OpenSSH host keys, will just have to change each time
> I do this).
In general I try to do my configurations editing input/overlay.
This way they will be part of next iteration
When forced to work directly on target I try to backport to the ./input/overlay/ tree
For this I found using mc (Midnight Commander) with one panel connected to target via "Shell link" very effective.
> Option 2:
> Rerun "make menuconfig; make" and deploy just the new package.
> Lather, rinse, repeat, until I've figured out what package sets make
> the most sense.  Then do Option 1.
Here a I use rsync to add whatever needed without clobbering what shouldn't be clobbered.
Using it with Buildroot output is not trivial.
I use the following script:

===================
#!/bin/bash

target=
port=
here=$(pwd)
clean=none

while getopts ":t:p:" opt; do
   case $opt in
     t)
       target=$OPTARG
       ;;
     p)
       port="-p $OPTARG"
       ;;
     \?)
       echo "Invalid option: -$OPTARG" >&2
       exit 1
       ;;
     :)
       echo "Option -$OPTARG requires an argument." >&2
       exit 1
       ;;
   esac
done

if [ -z "$target" ]
then
   echo "usage: $0 -t <target IP> [ -p <target ssh port> ]"
   exit 1
fi

cd /tmp
mkdir TMPfs
cd TMPfs

fakeroot bash <<- EOF
         tar xf $here/output/images/rootfs.tar
         rsync -av -e "ssh $port -i $here/operator -oUserKnownHostsFile=$HOME/.ssh/known_hosts" . root@$target:/
EOF

rm -rf *
cd ..
rmdir TMPfs
===================

I generated a key pair operator/operator.pub and I added operator.pub to ./input/overlay/root/.ssh/authorized_keys

>
> Which option do folks in this community choose in general?
>
> --wpd
> _______________________________________________
> buildroot mailing list
> buildroot at busybox.net
> http://lists.busybox.net/mailman/listinfo/buildroot
>
Hope it Helps

Regards
Mauro

[Buildroot] How do folks add packages to an existing rootfs during development?

[Patrick Doyle]

Thanks to all of you for your suggestions... they all help.

--wpd