Re: [PATCH] rootfs: always update the opkg index

[Ioan-Adrian Ratiu <adrian.ratiu@ni.com>]

On Mon, 24 Sep 2018, Richard Purdie <richard.purdie@linuxfoundation.org> wrote:
> On Mon, 2018-09-24 at 16:25 +0300, Ioan-Adrian Ratiu wrote:
>> The previous logic assumed that if $BUILD_IMAGES_FROM_FEEDS=1 then a
>> complete set of ipk feeds from which to build the image is already
>> present under $IPK_FEED_URIS at do_rootfs runtime.
>> 
>> $IPK_FEED_URIS usually contains "file://${DEPLOY_DIR_IPK}" which
>> renders the above assumption bad because some recipes in the current
>> build can contain code like do_install[nostamp] = "1" which will
>> cause
>> rebuilds bumping $PR and invalidating the index.
>> 
>> Even when the index is manually re-created before an image build
>> ("bitbake package-index"), the nostamp will cause failures because
>> the
>> dependency gets rebuilt before do_rootfs in the "bitbake <image>"
>> call.
>> 
>> So make the opkg rootfs index logic the same as for rpm/deb, to
>> always
>> update the index in $DEPLOY_DIR_IPK to fix the above nostamp failure.
>> 
>> Feeds outside $DEPLOY_DIR_IPK continue to work as usual.
>
> Is this with master or an older branch?

I've extensively tested this with Sumo, on master I could only verify
the build passes and a minimal image boots.

>
> With master, a copy of the feed is constructed under WORKDIR which is
> then indexed since other image generation or package writes could occur
> and the index isn't static. I'm guessing it doesn't do this for
> external feeds and assumes those feeds are static.

My description of the problem mirrors my understanding of the feed
construction process which might be limited especially if it changed on
master post-Sumo, but the answer is yes, external feeds are assumed
static.

>
> I am wondering what happens if you put a remote (http://) url into this
> though? Does that still work or does it break?

It still works, we're using http paths all the time in our distro
(public feed server is at http://download.ni.com/ni-linux-rt/)

>
> I'm also worried about what happens if the file:// path you point at is
> owned by another user and isn't writable.

Thanks for pointing this out. Looks like it enters an infinite loop: the
do_rootfs task is stuck at 3% and a worker process pegs a core to 100%.

Without this change, do_rootfs gets stuck at 12%, worker pegging a core
if the feed is owned by another user with no write permissions.

This was unexpected :) I'll roll up my sleeves and start digging.

Do you have any pointers about what might be happening?

>
> Cheers,
>
> Richard