Openembedded Core Discussions
 help / color / mirror / Atom feed
* [PATCH 1/3] buildhistory: record all builds
@ 2012-03-20 11:00 Koen Kooi
  2012-03-20 11:00 ` [PATCH 2/3] buildhistory: make seperate commits for every entry in the buildhistory dir Koen Kooi
  2012-03-20 11:00 ` [PATCH 3/3] buildhistory: remove duplicate entries from dot graph Koen Kooi
  0 siblings, 2 replies; 7+ messages in thread
From: Koen Kooi @ 2012-03-20 11:00 UTC (permalink / raw)
  To: openembedded-core; +Cc: Koen Kooi

Allow empty commits, this also give a nice speedup since 'git status --porcelain' doesn't need to get run.

Signed-off-by: Koen Kooi <koen@dominion.thruhere.net>
---
 meta/classes/buildhistory.bbclass |   17 +++++++----------
 1 files changed, 7 insertions(+), 10 deletions(-)

diff --git a/meta/classes/buildhistory.bbclass b/meta/classes/buildhistory.bbclass
index f99aa7f..1926d12 100644
--- a/meta/classes/buildhistory.bbclass
+++ b/meta/classes/buildhistory.bbclass
@@ -396,16 +396,13 @@ buildhistory_commit() {
 		if [ ! -d .git ] ; then
 			git init -q
 		fi
-		# Ensure there are new/changed files to commit
-		repostatus=`git status --porcelain`
-		if [ "$repostatus" != "" ] ; then
-			git add ${BUILDHISTORY_DIR}/*
-			HOSTNAME=`hostname 2>/dev/null || echo unknown`
-			git commit ${BUILDHISTORY_DIR}/ -m "Build ${BUILDNAME} of ${DISTRO} ${DISTRO_VERSION} for machine ${MACHINE} on $HOSTNAME" --author "${BUILDHISTORY_COMMIT_AUTHOR}" > /dev/null
-			if [ "${BUILDHISTORY_PUSH_REPO}" != "" ] ; then
-				git push -q ${BUILDHISTORY_PUSH_REPO}
-			fi
-		fi) || true
+		git add ${BUILDHISTORY_DIR}/*
+		HOSTNAME=`hostname 2>/dev/null || echo unknown`
+		git commit ${BUILDHISTORY_DIR}/ --allow-empty -m "Build ${BUILDNAME} of ${DISTRO} ${DISTRO_VERSION} for machine ${MACHINE} on $HOSTNAME" --author "${BUILDHISTORY_COMMIT_AUTHOR}" > /dev/null
+		if [ "${BUILDHISTORY_PUSH_REPO}" != "" ] ; then
+			git push -q ${BUILDHISTORY_PUSH_REPO}
+		fi
+	) || true
 }
 
 python buildhistory_eventhandler() {
-- 
1.7.2.5




^ permalink raw reply related	[flat|nested] 7+ messages in thread

* [PATCH 2/3] buildhistory: make seperate commits for every entry in the buildhistory dir
  2012-03-20 11:00 [PATCH 1/3] buildhistory: record all builds Koen Kooi
@ 2012-03-20 11:00 ` Koen Kooi
  2012-03-20 11:00 ` [PATCH 3/3] buildhistory: remove duplicate entries from dot graph Koen Kooi
  1 sibling, 0 replies; 7+ messages in thread
From: Koen Kooi @ 2012-03-20 11:00 UTC (permalink / raw)
  To: openembedded-core; +Cc: Koen Kooi

This seperates out image changes from package changes making the image diffs a lot easier to read. The downside of this is 3 commits for each and every build.

Signed-off-by: Koen Kooi <koen@dominion.thruhere.net>
---
 meta/classes/buildhistory.bbclass |    4 +++-
 1 files changed, 3 insertions(+), 1 deletions(-)

diff --git a/meta/classes/buildhistory.bbclass b/meta/classes/buildhistory.bbclass
index 1926d12..3aec325 100644
--- a/meta/classes/buildhistory.bbclass
+++ b/meta/classes/buildhistory.bbclass
@@ -398,7 +398,9 @@ buildhistory_commit() {
 		fi
 		git add ${BUILDHISTORY_DIR}/*
 		HOSTNAME=`hostname 2>/dev/null || echo unknown`
-		git commit ${BUILDHISTORY_DIR}/ --allow-empty -m "Build ${BUILDNAME} of ${DISTRO} ${DISTRO_VERSION} for machine ${MACHINE} on $HOSTNAME" --author "${BUILDHISTORY_COMMIT_AUTHOR}" > /dev/null
+		for entry in ${BUILDHISTORY_DIR}/* ; do
+			git commit $entry --allow-empty -m "$(basename $entry): Build ${BUILDNAME} of ${DISTRO} ${DISTRO_VERSION} for machine ${MACHINE} on $HOSTNAME" --author "${BUILDHISTORY_COMMIT_AUTHOR}" > /dev/null
+		done
 		if [ "${BUILDHISTORY_PUSH_REPO}" != "" ] ; then
 			git push -q ${BUILDHISTORY_PUSH_REPO}
 		fi
-- 
1.7.2.5




^ permalink raw reply related	[flat|nested] 7+ messages in thread

* [PATCH 3/3] buildhistory: remove duplicate entries from dot graph
  2012-03-20 11:00 [PATCH 1/3] buildhistory: record all builds Koen Kooi
  2012-03-20 11:00 ` [PATCH 2/3] buildhistory: make seperate commits for every entry in the buildhistory dir Koen Kooi
@ 2012-03-20 11:00 ` Koen Kooi
  2012-03-20 11:48   ` Paul Eggleton
  1 sibling, 1 reply; 7+ messages in thread
From: Koen Kooi @ 2012-03-20 11:00 UTC (permalink / raw)
  To: openembedded-core; +Cc: Koen Kooi

There are various conditions that lead to duplicate entries in the dot graph which need to get fixed, but this patch is a catchall.

Another benefit is that the sort order is now know, leading to less spurious diffs in buildhistory commits.

Signed-off-by: Koen Kooi <koen@dominion.thruhere.net>
---
 meta/classes/buildhistory.bbclass |    7 +++++--
 1 files changed, 5 insertions(+), 2 deletions(-)

diff --git a/meta/classes/buildhistory.bbclass b/meta/classes/buildhistory.bbclass
index 3aec325..6502b04 100644
--- a/meta/classes/buildhistory.bbclass
+++ b/meta/classes/buildhistory.bbclass
@@ -291,6 +291,7 @@ buildhistory_get_image_installed() {
 	# Produce installed package file and size lists and dependency graph
 	echo -n > ${BUILDHISTORY_DIR_IMAGE}/installed-packages.txt
 	echo -n > ${BUILDHISTORY_DIR_IMAGE}/installed-package-sizes.tmp
+	echo -n > ${BUILDHISTORY_DIR_IMAGE}/depends.tmp	
 	echo -e "digraph depends {\n    node [shape=plaintext]" > ${BUILDHISTORY_DIR_IMAGE}/depends.dot
 	for pkg in $INSTALLED_PKGS; do
 		pkgfile=`get_package_filename $pkg`
@@ -302,14 +303,16 @@ buildhistory_get_image_installed() {
 
 		deps=`list_package_depends $pkg | sort | uniq`
 		for dep in $deps ; do
-			echo "$pkg OPP $dep;" | sed -e 's:-:_:g' -e 's:\.:_:g' -e 's:+::g' | sed 's:OPP:->:g' >> ${BUILDHISTORY_DIR_IMAGE}/depends.dot
+			echo "$pkg OPP $dep;" | sed -e 's:-:_:g' -e 's:\.:_:g' -e 's:+::g' | sed 's:OPP:->:g' >> ${BUILDHISTORY_DIR_IMAGE}/depends.tmp
 		done
 
 		recs=`list_package_recommends $pkg | sort | uniq`
 		for rec in $recs ; do
-			echo "$pkg OPP $rec [style=dotted];" | sed -e 's:-:_:g' -e 's:\.:_:g' -e 's:+::g' | sed 's:OPP:->:g' >> ${BUILDHISTORY_DIR_IMAGE}/depends.dot
+			echo "$pkg OPP $rec [style=dotted];" | sed -e 's:-:_:g' -e 's:\.:_:g' -e 's:+::g' | sed 's:OPP:->:g' >> ${BUILDHISTORY_DIR_IMAGE}/depends.tmp
 		done
 	done
+	cat ${BUILDHISTORY_DIR_IMAGE}/depends.tmp | sort | uniq >> ${BUILDHISTORY_DIR_IMAGE}/depends.dot
+	rm ${BUILDHISTORY_DIR_IMAGE}/depends.tmp
 	echo "}" >>  ${BUILDHISTORY_DIR_IMAGE}/depends.dot
 
 	cat ${BUILDHISTORY_DIR_IMAGE}/installed-package-sizes.tmp | sort -n -r | awk '{print $1 "\tKiB " $2}' > ${BUILDHISTORY_DIR_IMAGE}/installed-package-sizes.txt
-- 
1.7.2.5




^ permalink raw reply related	[flat|nested] 7+ messages in thread

* Re: [PATCH 3/3] buildhistory: remove duplicate entries from dot graph
  2012-03-20 11:00 ` [PATCH 3/3] buildhistory: remove duplicate entries from dot graph Koen Kooi
@ 2012-03-20 11:48   ` Paul Eggleton
  2012-03-20 12:11     ` Koen Kooi
  0 siblings, 1 reply; 7+ messages in thread
From: Paul Eggleton @ 2012-03-20 11:48 UTC (permalink / raw)
  To: Koen Kooi; +Cc: openembedded-core

On Tuesday 20 March 2012 12:00:55 Koen Kooi wrote:
> There are various conditions that lead to duplicate entries in the dot graph
> which need to get fixed, but this patch is a catchall.

Dependencies and recommends are already de-duped - are we getting duplicates 
in the package list as well?

> Another benefit is that the sort order is now know, leading to less spurious
> diffs in buildhistory commits.

The package list is already sorted, and so are the dependencies and 
recommends, so how would the sort order be changing?

Cheers,
Paul

-- 

Paul Eggleton
Intel Open Source Technology Centre



^ permalink raw reply	[flat|nested] 7+ messages in thread

* Re: [PATCH 3/3] buildhistory: remove duplicate entries from dot graph
  2012-03-20 11:48   ` Paul Eggleton
@ 2012-03-20 12:11     ` Koen Kooi
  2012-03-20 13:14       ` Paul Eggleton
  0 siblings, 1 reply; 7+ messages in thread
From: Koen Kooi @ 2012-03-20 12:11 UTC (permalink / raw)
  To: Patches and discussions about the oe-core layer


Op 20 mrt. 2012, om 12:48 heeft Paul Eggleton het volgende geschreven:

> On Tuesday 20 March 2012 12:00:55 Koen Kooi wrote:
>> There are various conditions that lead to duplicate entries in the dot graph
>> which need to get fixed, but this patch is a catchall.
> 
> Dependencies and recommends are already de-duped - are we getting duplicates 
> in the package list as well?

From what I'm seeing, only dupes in the dot files:

https://github.com/Angstrom-distribution/buildhistory/commit/b2a256e02e04349db51e663e35b2bb9989acc854#diff-1

>> Another benefit is that the sort order is now know, leading to less spurious
>> diffs in buildhistory commits.
> 
> The package list is already sorted, and so are the dependencies and 
> recommends, so how would the sort order be changing?

Those fields are not sorted in the package itself[1], re-running bitbake systemd-image after these changes:

 angstrom_feed_configs -> opkg [style=dotted];
-angstrom_task_boot -> task_boot;
 angstrom_task_boot -> angstrom_version;
+angstrom_task_boot -> task_boot;
[..]
-htop -> ncurses_terminfo;
-htop -> libtinfo5;
 htop -> libncurses5;
-iw -> libnl_genl_3_200;
+htop -> libtinfo5;
+htop -> ncurses_terminfo;
 iw -> libnl_3_200;

I think I know why [2] doesn't work:

koen@dominion:/data/ssd/OE/buildhistory$ echo b c a | sort
b c a
koen@dominion:/data/ssd/OE/buildhistory$ echo -e b\\nc\\na | sort
a
b
c

Let me know how you want me to respin this series, each commit is an RFC on its own.

regards,

Koen

[1] https://github.com/openembedded/oe-core/commit/06b740d4ca077fb4c89ee6d1065fabb02da45ec6
[2] https://github.com/openembedded/oe-core/commit/830df6067c1ea4a5aab580b42ba7e1e84fe1bcbf


^ permalink raw reply	[flat|nested] 7+ messages in thread

* Re: [PATCH 3/3] buildhistory: remove duplicate entries from dot graph
  2012-03-20 12:11     ` Koen Kooi
@ 2012-03-20 13:14       ` Paul Eggleton
  2012-03-20 13:40         ` Koen Kooi
  0 siblings, 1 reply; 7+ messages in thread
From: Paul Eggleton @ 2012-03-20 13:14 UTC (permalink / raw)
  To: Koen Kooi; +Cc: openembedded-core

On Tuesday 20 March 2012 13:11:38 Koen Kooi wrote:
> I think I know why [2] doesn't work:
> 
> koen@dominion:/data/ssd/OE/buildhistory$ echo b c a | sort
> b c a
> koen@dominion:/data/ssd/OE/buildhistory$ echo -e b\\nc\\na | sort
> a
> b
> c

Ah, yes, that would explain it. I've primarily been testing with rpm for which 
list_package_depends produces the output on separate lines, whereas the ipk 
version is all on one line; it only matters when doing the sort. Oops!

> Let me know how you want me to respin this series, each commit is an RFC on
> its own.

If you wouldn't mind I think the easiest thing is to drop the sort/uniq on the 
depends/recommends and just rely on the one at the end that you're adding. It 
does mean we'll have a one-time resort of the recommends and depends together 
but at least it will work properly for all package backends.

Thanks,
Paul

-- 

Paul Eggleton
Intel Open Source Technology Centre



^ permalink raw reply	[flat|nested] 7+ messages in thread

* Re: [PATCH 3/3] buildhistory: remove duplicate entries from dot graph
  2012-03-20 13:14       ` Paul Eggleton
@ 2012-03-20 13:40         ` Koen Kooi
  0 siblings, 0 replies; 7+ messages in thread
From: Koen Kooi @ 2012-03-20 13:40 UTC (permalink / raw)
  To: Patches and discussions about the oe-core layer


Op 20 mrt. 2012, om 14:14 heeft Paul Eggleton het volgende geschreven:

> On Tuesday 20 March 2012 13:11:38 Koen Kooi wrote:
>> I think I know why [2] doesn't work:
>> 
>> koen@dominion:/data/ssd/OE/buildhistory$ echo b c a | sort
>> b c a
>> koen@dominion:/data/ssd/OE/buildhistory$ echo -e b\\nc\\na | sort
>> a
>> b
>> c
> 
> Ah, yes, that would explain it. I've primarily been testing with rpm for which 
> list_package_depends produces the output on separate lines, whereas the ipk 
> version is all on one line; it only matters when doing the sort. Oops!
> 
>> Let me know how you want me to respin this series, each commit is an RFC on
>> its own.
> 
> If you wouldn't mind I think the easiest thing is to drop the sort/uniq on the 
> depends/recommends and just rely on the one at the end that you're adding

V2 sent. I'm not really happy with the interaction between patch 1 and 2, so I'd appreciate some feedback on it.

regards,

Koen


^ permalink raw reply	[flat|nested] 7+ messages in thread

end of thread, other threads:[~2012-03-20 13:49 UTC | newest]

Thread overview: 7+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2012-03-20 11:00 [PATCH 1/3] buildhistory: record all builds Koen Kooi
2012-03-20 11:00 ` [PATCH 2/3] buildhistory: make seperate commits for every entry in the buildhistory dir Koen Kooi
2012-03-20 11:00 ` [PATCH 3/3] buildhistory: remove duplicate entries from dot graph Koen Kooi
2012-03-20 11:48   ` Paul Eggleton
2012-03-20 12:11     ` Koen Kooi
2012-03-20 13:14       ` Paul Eggleton
2012-03-20 13:40         ` Koen Kooi

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox