* [Buildroot] [PATCH 1/3] Add an entry in images sub-menu for ipk repository building
2013-01-15 16:47 [Buildroot] [PATCH v0] Add support for automated building of Opkg repository Jérôme Pouiller
@ 2013-01-15 16:47 ` Jérôme Pouiller
2013-01-15 16:47 ` [Buildroot] [PATCH 2/3] Add main scripts for opkg " Jérôme Pouiller
` (3 subsequent siblings)
4 siblings, 0 replies; 8+ messages in thread
From: Jérôme Pouiller @ 2013-01-15 16:47 UTC (permalink / raw)
To: buildroot
Just add necessary infrastructure.
Signed-off-by: J?r?me Pouiller <jezz@sysmic.org>
---
fs/Config.in | 1 +
fs/ipk/Config.in | 13 +++++++++++++
fs/ipk/ipk.mk | 22 ++++++++++++++++++++++
3 files changed, 36 insertions(+)
create mode 100644 fs/ipk/Config.in
create mode 100644 fs/ipk/ipk.mk
diff --git a/fs/Config.in b/fs/Config.in
index da4c5ff..7689cc2 100644
--- a/fs/Config.in
+++ b/fs/Config.in
@@ -11,5 +11,6 @@ source "fs/romfs/Config.in"
source "fs/squashfs/Config.in"
source "fs/tar/Config.in"
source "fs/ubifs/Config.in"
+source "fs/ipk/Config.in"
endmenu
diff --git a/fs/ipk/Config.in b/fs/ipk/Config.in
new file mode 100644
index 0000000..b42c9ab
--- /dev/null
+++ b/fs/ipk/Config.in
@@ -0,0 +1,13 @@
+config BR2_TARGET_ROOTFS_IPK_REPO
+ bool "Ipk repository with packages"
+ default y
+ help
+ Create a repository compatible with opkg/dpkg tools.
+
+ Each selected packages is splitted in 4 parts:
+ - main package with debug symbols stripped off
+ - debug symbols
+ - development files
+ - documentation files
+ - locales
+
diff --git a/fs/ipk/ipk.mk b/fs/ipk/ipk.mk
new file mode 100644
index 0000000..8b8d6f2
--- /dev/null
+++ b/fs/ipk/ipk.mk
@@ -0,0 +1,22 @@
+#############################################################
+#
+# Generate a repository for opkg
+#
+#############################################################
+
+define ROOTFS_IPK_REPO_CMD
+ mkdir -p $(BINARIES_DIR)/ipk_repository; \
+ ( cd $(BINARIES_DIR)/ipk_repository && dpkg-scanpackages . ) > $(BINARIES_DIR)/ipk_repository/Packages; \
+ gzip < $(BINARIES_DIR)/ipk_repository/Packages > $(BINARIES_DIR)/ipk_repository/Packages.gz; \
+ echo "Archive: unstable" >> $(BINARIES_DIR)/ipk_repository/Release; \
+ echo "Origin: Buildroot" >> $(BINARIES_DIR)/ipk_repository/Release; \
+ echo "Label: Buildroot autogenerated repository" >> $(BINARIES_DIR)/ipk_repository/Release; \
+ echo "Architecture: $(ARCH)" >> $(BINARIES_DIR)/ipk_repository/Release; \
+ echo "Your repository is ready."; \
+ echo "If you use opkg, add this line to your /etc/opkg/buildroot.conf:"; \
+ echo " src/gz buildroot file://$(BINARIES_DIR)/ipk_repository"; \
+ echo "If you use apt, add this one to /etc/apt/sources.list.d/buildroot.list:"; \
+ echo " deb file://$(BINARIES_DIR)/ipk_repository .";
+endef
+
+$(eval $(call ROOTFS_TARGET,ipk_repo))
--
1.7.9.5
^ permalink raw reply related [flat|nested] 8+ messages in thread* [Buildroot] [PATCH 2/3] Add main scripts for opkg repository building.
2013-01-15 16:47 [Buildroot] [PATCH v0] Add support for automated building of Opkg repository Jérôme Pouiller
2013-01-15 16:47 ` [Buildroot] [PATCH 1/3] Add an entry in images sub-menu for ipk repository building Jérôme Pouiller
@ 2013-01-15 16:47 ` Jérôme Pouiller
2013-01-15 16:47 ` [Buildroot] [PATCH 3/3] Add parsing of Config.in in ipk-post.sh Jérôme Pouiller
` (2 subsequent siblings)
4 siblings, 0 replies; 8+ messages in thread
From: Jérôme Pouiller @ 2013-01-15 16:47 UTC (permalink / raw)
To: buildroot
These scripts are called during installation of packages.
Signed-off-by: J?r?me Pouiller <jezz@sysmic.org>
---
package/pkg-generic.mk | 2 +
support/scripts/ipk-post.sh | 95 +++++++++++++++++++++++++++++++++++++++++++
support/scripts/ipk-pre.sh | 30 ++++++++++++++
3 files changed, 127 insertions(+)
create mode 100755 support/scripts/ipk-post.sh
create mode 100755 support/scripts/ipk-pre.sh
diff --git a/package/pkg-generic.mk b/package/pkg-generic.mk
index 59de0f0..b0eca0a 100644
--- a/package/pkg-generic.mk
+++ b/package/pkg-generic.mk
@@ -140,12 +140,14 @@ $(BUILD_DIR)/%/.stamp_images_installed:
# Install to target dir
$(BUILD_DIR)/%/.stamp_target_installed:
@$(call MESSAGE,"Installing to target")
+ $(if $(BR2_TARGET_ROOTFS_IPK_REPO),support/scripts/ipk-pre.sh $(TARGET_DIR) $(BUILD_DIR) $($(PKG)_DIR_PREFIX) $($(PKG)_RAWNAME) $($(PKG)_VERSION)) $(ARCH)
$(if $(BR2_INIT_SYSTEMD),\
$($(PKG)_INSTALL_INIT_SYSTEMD))
$(if $(BR2_INIT_SYSV)$(BR2_INIT_BUSYBOX),\
$($(PKG)_INSTALL_INIT_SYSV))
$($(PKG)_INSTALL_TARGET_CMDS)
$(foreach hook,$($(PKG)_POST_INSTALL_TARGET_HOOKS),$(call $(hook))$(sep))
+ $(if $(BR2_TARGET_ROOTFS_IPK_REPO),support/scripts/ipk-post.sh $(TARGET_DIR) $(BUILD_DIR) $($(PKG)_DIR_PREFIX) $($(PKG)_RAWNAME) $($(PKG)_VERSION)) $(ARCH)
$(Q)touch $@
# Clean package
diff --git a/support/scripts/ipk-post.sh b/support/scripts/ipk-post.sh
new file mode 100755
index 0000000..a5beacb
--- /dev/null
+++ b/support/scripts/ipk-post.sh
@@ -0,0 +1,95 @@
+#!/bin/bash
+#
+# Licence: GPL
+# Created: 2013-01-15 15:26:45+01:00
+# Main authors:
+# - J?r?me Pouiller <jezz@sysmic.org>
+#
+# Second part of package building.
+#
+# Kill previously launched daemon. Using daemon result, create ipk files of
+# package.
+#
+
+TARGET_DIR=$1
+BUILD_DIR=$2
+PACKAGES_DIR=$3
+PKG_RAWNAME=$4
+PKG_VERSION=$5
+ARCH=$6
+PKG_DIR=$PACKAGES_DIR/$PKG_RAWNAME
+PKG_BUILD_DIR=$BUILD_DIR/$PKG_RAWNAME-$PKG_VERSION
+IPK_DIR=$BUILD_DIR/ipk_build
+PKG=$PKG_RAWNAME
+
+kill $(cat $PKG_BUILD_DIR/.ipk_inotify_pid)
+
+# Create DEBIAN/control files
+for P in $PKG $PKG-i18n $PKG-doc $PKG-dbg $PKG-dev; do
+ rm -fr $IPK_DIR/$P
+ mkdir -p $IPK_DIR/$P/DEBIAN
+ (
+ echo "Package: $P"
+ echo "Version: $PKG_VERSION"
+ echo "Architecture: $ARCH"
+ echo "Maintainer: J?r?me Pouiller <jezz@sysmic.org>"
+ [ $P == $PKG-i18n ] && echo "Depends: $PKG"
+ [ $P == $PKG-dev ] && echo "Depends: $PKG"
+ [ $P == $PKG-dbg ] && echo "Depends: $PKG"
+ [ $P == $PKG-doc ] && echo "Recommends: $PKG"
+ echo "Description: $PKG"
+ [ $P == $PKG-i18n ] && echo -e " .\n This package contains locales files"
+ [ $P == $PKG-dev ] && echo -e " .\n This package contains developement files"
+ [ $P == $PKG-dbg ] && echo -e " .\n This package contains debug symbols"
+ [ $P == $PKG-doc ] && echo -e " .\n This package contains documentation"
+ echo
+ ) > $IPK_DIR/$P/DEBIAN/control
+done
+
+# Place application files in package trees
+cut -f 2 $PKG_BUILD_DIR/.ipk_list_installed_files | sort | uniq | while read FILE_FULL; do
+ FILE=${FILE_FULL##$TARGET_DIR/}
+ DIR=${FILE%/*}
+ [[ -e $FILE_FULL ]] || continue
+ [[ -d $FILE_FULL ]] && continue
+ case /$FILE in
+ /usr/include/*|*.a|*.la|/usr/lib/pkgconfig/*|/usr/share/aclocal/*)
+ mkdir -p $IPK_DIR/$PKG-dev/$DIR
+ cp -pd $FILE_FULL $IPK_DIR/$PKG-dev/$FILE
+ ;;
+ /bin/*|/sbin/*|/lib/*.so*|/usr/bin/*|/usr/sbin/*|/usr/lib/*.so*)
+ mkdir -p $IPK_DIR/$PKG/$DIR
+ if [[ -L $FILE_FULL ]]; then
+ cp -pd $FILE_FULL $IPK_DIR/$PKG/$FILE
+ else
+ mkdir -p $IPK_DIR/$PKG-dbg/$DIR
+ strip $FILE_FULL -o $IPK_DIR/$PKG-dbg/$FILE.dbg --only-keep-debug
+ if [[ /$FILE == *thread*.so* ]]; then
+ strip $FILE_FULL -o $IPK_DIR/$PKG/$FILE --strip-debug
+ else
+ strip $FILE_FULL -o $IPK_DIR/$PKG/$FILE
+ fi
+ fi
+ ;;
+ /usr/*doc/*|/usr/*man/*|/usr/*info/*|/usr/*gtk-doc/*)
+ mkdir -p $IPK_DIR/$PKG-doc/$DIR
+ cp -pd $FILE_FULL $IPK_DIR/$PKG-doc/$FILE
+ ;;
+ /usr/share/locale/*)
+ mkdir -p $IPK_DIR/$PKG-i18n/$DIR
+ cp -pd $FILE_FULL $IPK_DIR/$PKG-i18n/$FILE
+ ;;
+ *)
+ mkdir -p $IPK_DIR/$PKG/$DIR
+ cp -pd $FILE_FULL $IPK_DIR/$PKG/$FILE
+ ;;
+ esac
+done
+
+# Create .deb (same than .ipk) file
+mkdir -p $BUILD_DIR/../images/ipk_repository
+for P in $PKG $PKG-i18n $PKG-doc $PKG-dbg $PKG-dev; do
+ if [[ $(ls $IPK_DIR/$P | wc -l) -gt 1 ]]; then
+ dpkg-deb -b $IPK_DIR/$P $BUILD_DIR/../images/ipk_repository
+ fi
+done
diff --git a/support/scripts/ipk-pre.sh b/support/scripts/ipk-pre.sh
new file mode 100755
index 0000000..27a7b6b
--- /dev/null
+++ b/support/scripts/ipk-pre.sh
@@ -0,0 +1,30 @@
+#!/bin/sh
+#
+# Licence: GPL
+# Created: 2013-01-15 15:25:52+01:00
+# Main authors:
+# - J?r?me Pouiller <jezz@sysmic.org>
+#
+# First part of package building.
+#
+# It launches a daemon to spy write acces to target. Results are written to
+# .ipk_list_installed_files in build subdirectory of package.
+#
+
+TARGET_DIR=$1
+BUILD_DIR=$2
+PACKAGES_DIR=$3
+PKG_RAWNAME=$4
+PKG_VERSION=$5
+PKG_DIR=$PACKAGES_DIR/$PKG_RAWNAME
+PKG_BUILD_DIR=$BUILD_DIR/$PKG_RAWNAME-$PKG_VERSION
+IPK_DIR=$BUILD_DIR/ipk_build
+PKG=$PKG_RAWNAME
+
+inotifywait -mr $TARGET_DIR -e create -e modify -e moved_to --format '%e %w%f' > $PKG_BUILD_DIR/.ipk_list_installed_files &
+echo $! > $PKG_BUILD_DIR/.ipk_inotify_pid
+# FIXME Be sure inotifywait is started
+#sleep 3
+
+
+
--
1.7.9.5
^ permalink raw reply related [flat|nested] 8+ messages in thread* [Buildroot] [PATCH 3/3] Add parsing of Config.in in ipk-post.sh.
2013-01-15 16:47 [Buildroot] [PATCH v0] Add support for automated building of Opkg repository Jérôme Pouiller
2013-01-15 16:47 ` [Buildroot] [PATCH 1/3] Add an entry in images sub-menu for ipk repository building Jérôme Pouiller
2013-01-15 16:47 ` [Buildroot] [PATCH 2/3] Add main scripts for opkg " Jérôme Pouiller
@ 2013-01-15 16:47 ` Jérôme Pouiller
2013-01-17 10:23 ` [Buildroot] [PATCH v0] Add support for automated building of Opkg repository Jérôme Pouiller
2013-02-13 21:43 ` Carsten Schoenert
4 siblings, 0 replies; 8+ messages in thread
From: Jérôme Pouiller @ 2013-01-15 16:47 UTC (permalink / raw)
To: buildroot
Built packages now contains description from Buildroot.
Signed-off-by: J?r?me Pouiller <jezz@sysmic.org>
---
support/scripts/ipk-post.sh | 17 +++++++++++++++++
1 file changed, 17 insertions(+)
diff --git a/support/scripts/ipk-post.sh b/support/scripts/ipk-post.sh
index a5beacb..2bfd736 100755
--- a/support/scripts/ipk-post.sh
+++ b/support/scripts/ipk-post.sh
@@ -38,6 +38,23 @@ for P in $PKG $PKG-i18n $PKG-doc $PKG-dbg $PKG-dev; do
[ $P == $PKG-dbg ] && echo "Depends: $PKG"
[ $P == $PKG-doc ] && echo "Recommends: $PKG"
echo "Description: $PKG"
+ STATE=EMPTY
+ while IFS= read LINE; do
+ if [[ $STATE == GETINDENT ]]; then
+ INDENT="$(echo "$LINE" | grep -o '^[ ]*')"
+ STATE=GETHELP
+ fi
+ if [[ $STATE == GETHELP ]]; then
+ if [[ "$LINE" =~ ^"$INDENT" ]]; then
+ echo " ${LINE#$INDENT}"
+ elif [[ "$LINE" == "" ]]; then
+ echo " ."
+ else
+ break
+ fi
+ fi
+ [[ "$LINE" =~ ^.*-*help-*$ ]] && STATE=GETINDENT
+ done < $PKG_DIR/Config.in
[ $P == $PKG-i18n ] && echo -e " .\n This package contains locales files"
[ $P == $PKG-dev ] && echo -e " .\n This package contains developement files"
[ $P == $PKG-dbg ] && echo -e " .\n This package contains debug symbols"
--
1.7.9.5
^ permalink raw reply related [flat|nested] 8+ messages in thread* [Buildroot] [PATCH v0] Add support for automated building of Opkg repository
2013-01-15 16:47 [Buildroot] [PATCH v0] Add support for automated building of Opkg repository Jérôme Pouiller
` (2 preceding siblings ...)
2013-01-15 16:47 ` [Buildroot] [PATCH 3/3] Add parsing of Config.in in ipk-post.sh Jérôme Pouiller
@ 2013-01-17 10:23 ` Jérôme Pouiller
2013-01-17 12:54 ` Thomas Petazzoni
2013-02-13 21:43 ` Carsten Schoenert
4 siblings, 1 reply; 8+ messages in thread
From: Jérôme Pouiller @ 2013-01-17 10:23 UTC (permalink / raw)
To: buildroot
On Tuesday 15 January 2013 17:47:07 J?r?me Pouiller wrote:
> Dear,
>
> I wrote a first draft of support of building deb or ipk file in Buildroot.
> The result looks like this:
> http://sysmic.org/~jezz/ipk_repository
>
> This patch use inotify to spy which file are installed by package. Once
> installation is done, it copy file to a temporary directory where ipk file
> is built.
[...]
> Result is not so bad, as you can see on URL above and it is not intrusive. I
> would be glad to get your opinion about this work. What do you think about
> it?
[...]
Hello all,
I am surprised to have any reply to this patch proposal. Patches have been
correctly sent? (I didn't receive them but I am sure it come from my local
mailer daemon since they appear on gmame). Or I am just too impatient?
I would like to know if some of you think this feature is interesting/useful?
And do you think I take the right way?
Regards,
--
J?r?me Pouiller
^ permalink raw reply [flat|nested] 8+ messages in thread* [Buildroot] [PATCH v0] Add support for automated building of Opkg repository
2013-01-17 10:23 ` [Buildroot] [PATCH v0] Add support for automated building of Opkg repository Jérôme Pouiller
@ 2013-01-17 12:54 ` Thomas Petazzoni
2013-01-17 14:39 ` Jérôme Pouiller
0 siblings, 1 reply; 8+ messages in thread
From: Thomas Petazzoni @ 2013-01-17 12:54 UTC (permalink / raw)
To: buildroot
Dear J?r?me Pouiller,
On Thu, 17 Jan 2013 11:23:17 +0100, J?r?me Pouiller wrote:
> I am surprised to have any reply to this patch proposal. Patches have
> been correctly sent? (I didn't receive them but I am sure it come
> from my local mailer daemon since they appear on gmame). Or I am just
> too impatient?
Patches have been received. I intend to give a reply to them, but I
need a bit of time to do so. I'll get back with my comments soon.
Best regards,
Thomas
--
Thomas Petazzoni, Free Electrons
Kernel, drivers, real-time and embedded Linux
development, consulting, training and support.
http://free-electrons.com
^ permalink raw reply [flat|nested] 8+ messages in thread
* [Buildroot] [PATCH v0] Add support for automated building of Opkg repository
2013-01-17 12:54 ` Thomas Petazzoni
@ 2013-01-17 14:39 ` Jérôme Pouiller
0 siblings, 0 replies; 8+ messages in thread
From: Jérôme Pouiller @ 2013-01-17 14:39 UTC (permalink / raw)
To: buildroot
On Thursday 17 January 2013 13:54:18 Thomas Petazzoni wrote:
> On Thu, 17 Jan 2013 11:23:17 +0100, J?r?me Pouiller wrote:
> > I am surprised to have any reply to this patch proposal. Patches have
> > been correctly sent? (I didn't receive them but I am sure it come
> > from my local mailer daemon since they appear on gmame). Or I am just
> > too impatient?
>
> Patches have been received. I intend to give a reply to them, but I
> need a bit of time to do so. I'll get back with my comments soon.
Sorry to have been too impatient. :-)
Regards,
--
J?r?me Pouiller
^ permalink raw reply [flat|nested] 8+ messages in thread
* [Buildroot] [PATCH v0] Add support for automated building of Opkg repository
2013-01-15 16:47 [Buildroot] [PATCH v0] Add support for automated building of Opkg repository Jérôme Pouiller
` (3 preceding siblings ...)
2013-01-17 10:23 ` [Buildroot] [PATCH v0] Add support for automated building of Opkg repository Jérôme Pouiller
@ 2013-02-13 21:43 ` Carsten Schoenert
4 siblings, 0 replies; 8+ messages in thread
From: Carsten Schoenert @ 2013-02-13 21:43 UTC (permalink / raw)
To: buildroot
Hello J?r?me,
Am 15.01.2013 17:47, schrieb J?r?me Pouiller:
> Dear,
>
> I wrote a first draft of support of building deb or ipk file in Buildroot. The
> result looks like this:
> http://sysmic.org/~jezz/ipk_repository
This looks impressive and the OPKG packaging would be a really "killer"
feature in buildroot for me! OPKG packages is one of the features I'm
really missing.
> This patch use inotify to spy which file are installed by package. Once
> installation is done, it copy file to a temporary directory where ipk file is
> built.
>
> During this copy, file are splited in 5 ipk files:
> - main package with debug symbols stripped off
> - debug symbols
> - development files
> - documentation files
> - locales
This would be much more than I use currently in an other existing
project, sounds like you wanted to build the *one* big solution. ;)
>
> All these files exist since it is done before target-finalize rule.
>
> Next script create "control" files needed for building. Script parse Config.in
> file to extract help and place it as Description in ipk file.
Creating the control file in runtime would remove the necessary to
rework the current packages, if this works ... great!
> Finaly, dpkg-deb is used to create files. Files get deb extention but they are
> compatible with opkg/ipkg. I don't know if it would be better to use opkg-build
> script.
I would say this depends on the nature of the script, but this can be
tested.
> Result is not so bad, as you can see on URL above and it is not intrusive. I would
> be glad to get your opinion about this work. What do you think about it?
>
> Current restrictions and things to do:
> - Be able to correctly extract Help from complex Config.in
> - Parse Depends field of Config.in
> - Check dependencies using ldd
In the buildsystem-cs [1] (a system for building images for various DVB
setopboxes) a friend of mine use some scripts [2] to get the
dependencies and doing the creating of the packages incl. the comparing
if the package is really new. Maybe there is something useful to find.
> - Add packages for skeleton copy, libc copy and target-finalize rule
> - Detect file written by multiple packages
> - Be able to package setuid binaries
> - Kill inotify if make is interrupted
> - Allow to customize Maintener name, Company name, etc...
> - Allow to append a suffix to package versions
> - Re-use strip commands defined in .config
> - Add gnu-debuglink to binaries
> - I need plenty of Makefile variables. It would be better if ipk-pre.sh and
> ipk-post.sh were converted to Makefiles?
> - Make Architecture field a standard field recognized by Debian
> - Check inotifywait is correctly started before continue installation
> - Be able to do parallel installation
> - Check host tools dependencies
What about a package with the basic rootfs after passing the overlay and
post-build.sh script? Do you have allready think about that?
[1] http://gitorious.org/neutrino-hd/buildsystem-cs
[2] https://gitorious.org/neutrino-hd/buildsystem-cs/trees/master/scripts
--
Mit freundlichen Gr??en
Carsten Sch?nert
^ permalink raw reply [flat|nested] 8+ messages in thread