Re: [OE-core] Add package managers as a plugin

["Fredrik Gustafsson" <fredrik.gustafsson@axis.com>]

[-- Attachment #1: Type: text/plain, Size: 5072 bytes --]

Hi Alex,
Thanks for supplying a script!

I added apk to your script and then ran
```
for f in `ls`; do
printf "$f: "; cat $f | tail -2 | head -1 | awk -F ' ' '{print $3}' | sed -e 's/elapsed//'
done | sort
```

The result is here:
out.apk.1: 0:13.35
out.apk.2: 0:12.89
out.apk.3: 0:12.99
out.apk.4: 0:12.87
out.apk.5: 0:13.02
out.deb.1: 0:20.27
out.deb.2: 0:20.30
out.deb.3: 0:20.33
out.deb.4: 0:20.06
out.deb.5: 0:20.17
out.ipk.1: 0:18.51
out.ipk.2: 0:18.78
out.ipk.3: 0:18.80
out.ipk.4: 0:18.99
out.ipk.5: 0:18.97
out.rpm.1: 0:16.80
out.rpm.2: 0:16.83
out.rpm.3: 0:16.69
out.rpm.4: 0:16.78
out.rpm.5: 0:17.78

That is not supportive of my statement of ipk being the slowest package manager.
That statement come from timing tests done with our proprietary version of poky
that is somewhat extended when it comes to the do_rootfs task. We're generating
a bit more artifacts than poky. However apk is still ~22% faster than rpm and ~40%
faster than deb according to the data above.

So I tried your script with core-image-sato-sdk-ptest instead that is a bigger image.
The results where more mixed. There is a huge difference between different package
managers:
out.apk.1: 1:13.35
out.apk.2: 1:13.51
out.apk.3: 1:13.23
out.apk.4: 1:14.07
out.apk.5: 1:13.00
out.deb.1: 3:49.37
out.deb.2: 3:50.77
out.deb.3: 3:51.39
out.deb.4: 3:53.40
out.deb.5: 3:53.99
out.ipk.1: 2:38.99
out.ipk.2: 2:39.07
out.ipk.3: 2:35.34
out.ipk.4: 2:36.15
out.ipk.5: 2:34.55
out.rpm.1: 1:58.61
out.rpm.2: 1:59.42
out.rpm.3: 1:59.70
out.rpm.4: 1:58.96
out.rpm.5: 1:58.11

But yeah, I should probably rephrase my cover letter. You'll find a layer for APK that will
apply on top of my patch serie attached to this email.

Also note that there's a difference between the time to generate a package. Which 
we completely ignore in this measurements. It's not as important since it's being
cached, but for a fresh build it will be a difference between different package managers.

BR
Fredrik

___________________________
From: openembedded-core@lists.openembedded.org <openembedded-core@lists.openembedded.org> on behalf of Alex Stewart <alex.stewart@ni.com>
Sent: Wednesday, June 24, 2020 5:18 PM
To: Fredrik Gustafsson; openembedded-core@lists.openembedded.org
Cc: tools-cfpbuild-internal; Hugo Cedervall; Alejandro Del Castillo
Subject: Re: [OE-core] Add package managers as a plugin

On 6/23/20 6:13 AM, Fredrik Gustafsson wrote:
> When building and having a good cache hit, a significant amount of time is
> spent in the phase of generating a rootfs, which is really about the
> performance of the package manager. ipkg is way slower than deb or rpm.
I was interested enough in your comment here to collect a small sampling
of comparative do_rootfs runs on one of my dev machines.

I used a virgin poky clone and tried to replicate the steps you provided
in your OP, timing only the execution of `bitbake -c do_rootfs
core-image-minimal`. I used the debian `time` package for timing. Here's
what I got:

# Summary
PM     Avg do_rootfs time (s)
--     ----------------------
deb    33.89
ipk    33.35
rpm    31.85

# Raw Data
deb    00:33.54
deb    00:34.66
deb    00:33.55
deb    00:33.23
deb    00:34.46
ipk    00:33.96
ipk    00:34.43
ipk    00:33.99
ipk    00:32.17
ipk    00:32.22
rpm    00:29.92
rpm    00:32.31
rpm    00:31.90
rpm    00:33.31
rpm    00:31.80

Based on those results, it seems like all the current PMs are similar in
terms of execution time. Are there major differences between my test
setup and yours? And do you already have a layer with APK available that
I could compare against on my hardware?

This is the crude script I used to automate the test runs:

```
#!/bin/bash
set -euxo pipefail

TIME=/usr/bin/time

recipe=core-image-minimal

# Options are:
#  - 'package_deb' for debian style deb files
#  - 'package_ipk' for ipk files are used by opkg (a debian style
embedded package manager)
#  - 'package_rpm' for rpm style packages
function set_pkg_man() {
     pkg_man=${1}
     sed -i "s/\(^PACKAGE_CLASSES ?= \)\"\(.*\)\"/\1\"$pkg_man\"/"
conf/local.conf
}

function rebuild_recipe() {
     bitbake -c cleanall $recipe
     bitbake $recipe
}

function test_recipe() {
     local name=${1}
     bitbake -c cleansstate $recipe
     bitbake -c clean $recipe
     $TIME bitbake -c do_rootfs -f $recipe 2>&1 | tee outs/out.$name
}

# rpm
set_pkg_man package_rpm
rebuild_recipe

test_recipe rpm.1
test_recipe rpm.2
test_recipe rpm.3
test_recipe rpm.4
test_recipe rpm.5


# ipk
set_pkg_man package_ipk
rebuild_recipe

test_recipe ipk.1
test_recipe ipk.2
test_recipe ipk.3
test_recipe ipk.4
test_recipe ipk.5

# deb
set_pkg_man package_deb
rebuild_recipe

test_recipe deb.1
test_recipe deb.2
test_recipe deb.3
test_recipe deb.4
test_recipe deb.5
```

Thanks,

--
Alex Stewart
Software Engineer - LabVIEW Real-Time OS
National Instruments

alex.stewart@ni.com
office: +1(512)683-8522


[-- Attachment #2: meta-apk.tar.gz --]
[-- Type: application/gzip, Size: 31444 bytes --]