Re: [Patch-v2 1/1] mesa: avoid unnecessary rerunning of tasks

[Richard Purdie <richard.purdie@linuxfoundation.org>]

On Thu, 2013-09-05 at 09:33 +0100, Richard Purdie wrote:
> This is certainly better and is heading in the right direction. The two
> issues I can see are:
> 
> a) do_populate_sysroot still needs to run again
> b) do_packagedata installs pkgdata (shlibs) which says mesa provides 
>    some of these libs yet is no shouldn't for the given MACHINE.
> 
> I did think this shouldn't be too difficult to fix however life is never
> quite straightforward.
> 
> For a), rather than change things before we create the sstate package, I
> was thinking we could change than at installation of the sstate package.
> We have SSTATEPOSTINSTFUNCS however these run after we have placed the
> files in the sysroot so its too late. SSTATEPREINSTFUNCS sounds better
> but this actually runs before unpacking the sstate files which is too
> early for us.
> 
> There is only one user of SSTATEPREINSTFUNCS (useradd.bbclass) so I'm
> going to propose we rename that one to a more logical
> SSTATEPREUNPACKFUNCS and add PREINSTFUNCS at a more sensible place. We
> can then add a SSTATEPREINSTFUNCS to the mesa recipe which for
> do_populate_sysroot and do_populate_sysroot_setscene deletes the files
> in question. That should resolve a).
> 
> b) is more tricky since pkgdata is arch specific, not machine specific.
> I had thought it was machine specific but checking now, its not.
> 
> To be quite honest, I don't like what pkgdata does at the moment. We
> have cleaned things up a lot compared to where they used to be but the
> search process the system uses to find information is truly horrid.
> 
> It would be rather tempting to decide to install pkgdata in a machine
> specific directory much like we do with populate_sysroot already. There
> would be minimal performance overhead since the files are small and we
> could then bin all the "search all, then arch, then machine" type lookup
> code we have right now. If we did make it machine specific, we could
> then use the same trick we use in a) but for do_packagedata and instead
> of removing files, it would run some carefully crafted sed operations.
> 
> Right now, we might not need to fix this 100% properly. If emgd installs
> itself as machine specific, it should be first in the providers search
> paths so the second lot of pkgdata should get ignored. emgd is of course
> not machine specific at the moment, nor should it be apart from this
> issue :/.
> 
> This also raises some other questions. The existing code in meta-intel
> adds EMGD into the DEPENDS. It does this so the normal PROVIDES for
> virtual/libgl and friends don't have to change. It does mean the sstate
> checksums for anything depending on mesa change however. If we don't
> have the dependency, things may try and build before emgd is built,
> libGL would be missing and things would break.
> 
> So somehow we need to add in the dependency let leave the sstate
> checksums unchanged. This is possible in the sstate checksum validation
> code but its probably easiest to change that in the core, not
> meta-intel :/.
> 
> I'd also appreciate someone (Ross maybe?) confirming that if we build
> application A against mesa, then change over to a machine that uses emgd
> and swizzle the libs around in the sysroot, do we need to recompile the
> app? This approach is assuming we do not need to do that. If we do,
> there is a different approach we need to take.
> 
> So in summary, this problem is far from simple, we do have most of the
> pieces we need to fix it with some tweaking but some of the changes are
> invasive and might need to be deferred to 1.6. As it stands, the change
> will cause issues due to the changed DEPENDS. We do need to think
> carefully about this and solve it once and for all in a way which works
> optimally but the existing workaround might be best until we can pull
> all the better pieces together.
> 
> I hope this was a relatively clear explanation of the problem, let me
> know if I need to clarify anything. I've cc'd OE-Core since I think more
> people probably need to see/think about this.

It actually get worse (or maybe slightly better) when you think about
the shlibs code.

Imagine application A links against libGL. It therefore gets an RDEPENDS
on the package that provides libGL. With emgd in the mix, the package
providing libGL changes and as things stand today, I suspect it gets
rebuild.

So lets assume that we add leave the mesa pkgdata/shlibs code in place
and actively stop emgd putting conflicting shlibs provides in place
(easy to do).

We could then make the emgd-binary packages RPROVIDE/RCONFLICT the right
mesa packages. We'd then need to make sure the package manager behaviour
is correct in that:

a) If an image is mesa based, installing the app pulls in libGL as
normal, doesn't touch emgd.
b) In an EMGD image, installing the app pulls in libgl, however XSERVER
also pulls in emgd-binary and this then replaces the libGL package.

In theory this should all be possible and would resolve the problem in
b) in my original email without the need to make pkgdata machine
specific (we should do that for other reasons but we're getting off
track).

So perhaps there is a way out of this but it isn't simple...

Cheers,

Richard