State of bitbake world, files not tracked by sstate

State of bitbake world, files not tracked by sstate

[Martin Jansa]

[-- Attachment #1: Type: text/plain, Size: 1927 bytes --]

These are the files which are installed into sysroot without sstate
knowledge, so if you reuse the components which created them from sstate
these files will be missing.

cat << EOF > sstate-sysroot-cruft-whitelist.txt
[^/]*/home/builder
[^/]*/usr/src/kernel/patches
[^/]*/usr/src/kernel/scripts/.*
[^/]*/usr/lib/gdk-pixbuf-2.0/.*/loaders.cache
[^/]*/etc/sgml/sgml-docbook.cat
[^/]*/usr/src/kernel/patches
[^/]*/etc/sgml/sgml-docbook.cat
[^/]*/usr/lib/python3.3/__pycache__
[^/]*/usr/lib/python3.3/[^/]*/__pycache__
[^/]*/usr/lib/python3.3/[^/]*/[^/]*/__pycache__
[^/]*/usr/share/dbus
[^/]*/usr/share/dbus/dbus-bus-introspect.xml
[^/]*/usr/share/dbus/session.conf
[^/]*/usr/bin/crossscripts/guile-config
[^/]*/usr/lib/python2.7/config/libpython2.7.so
[^/]*/var
[^/]*/usr/bin/i586-oe-linux-g77
[^/]*/usr/bin/arm-oe-linux-gnueabi-g77
[^/]*/usr/lib/php/\.channels.*
[^/]*/usr/lib/php/\.registry.*
[^/]*/usr/lib/php/\.depdb.*
[^/]*/usr/lib/php/\.filemap
[^/]*/usr/lib/php/\.lock
[^/]*/usr/lib/gdk-pixbuf-2.0/.*/loaders.cache
[^/]*/usr/include/ruby-1.9.1/i386-linux
[^/]*/usr/include/ruby-1.9.1/i386-linux/ruby
[^/]*/usr/include/ruby-1.9.1/i386-linux/ruby/config.h
[^/]*/usr/include/ruby-1.9.1/ruby/win32.h
[^/]*/usr/lib/ruby/i386-linux
[^/]*/usr/lib/ruby/i386-linux/fake.rb
[^/]*/usr/lib/ruby/i386-linux/libruby.so.1.9.1
[^/]*/usr/lib/ruby/i386-linux/libruby-static.a
[^/]*/usr/lib/ruby/i386-linux/rbconfig.rb
[^/]*/usr/lib/qt4/plugins/webkit
[^/]*/usr/lib/qt5/plugins/webkit
EOF

You can easily verify it in your build (after build without
sstate-cache) with:
openembedded-core/scripts/sstate-sysroot-cruft.sh --tmpdir=tmp-glibc
--whitelist=sstate-sysroot-cruft-whitelist.txt 2>&1

RFC for g77 fix was sent couple months ago (no reply):
http://lists.openembedded.org/pipermail/openembedded-core/2014-April/091480.html

-- 
Martin 'JaMa' Jansa     jabber: Martin.Jansa@gmail.com

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 188 bytes --]

Re: State of bitbake world, files not tracked by sstate

[Richard Purdie]

On Wed, 2014-10-22 at 11:36 +0200, Martin Jansa wrote:
> These are the files which are installed into sysroot without sstate
> knowledge, so if you reuse the components which created them from sstate
> these files will be missing.
> 
> cat << EOF > sstate-sysroot-cruft-whitelist.txt
> [^/]*/home/builder
> [^/]*/usr/src/kernel/patches
> [^/]*/usr/src/kernel/scripts/.*
> [^/]*/usr/lib/gdk-pixbuf-2.0/.*/loaders.cache
> [^/]*/etc/sgml/sgml-docbook.cat
> [^/]*/usr/src/kernel/patches
> [^/]*/etc/sgml/sgml-docbook.cat
> [^/]*/usr/lib/python3.3/__pycache__
> [^/]*/usr/lib/python3.3/[^/]*/__pycache__
> [^/]*/usr/lib/python3.3/[^/]*/[^/]*/__pycache__
> [^/]*/usr/share/dbus
> [^/]*/usr/share/dbus/dbus-bus-introspect.xml
> [^/]*/usr/share/dbus/session.conf
> [^/]*/usr/bin/crossscripts/guile-config
> [^/]*/usr/lib/python2.7/config/libpython2.7.so
> [^/]*/var
> [^/]*/usr/bin/i586-oe-linux-g77
> [^/]*/usr/bin/arm-oe-linux-gnueabi-g77
> [^/]*/usr/lib/php/\.channels.*
> [^/]*/usr/lib/php/\.registry.*
> [^/]*/usr/lib/php/\.depdb.*
> [^/]*/usr/lib/php/\.filemap
> [^/]*/usr/lib/php/\.lock
> [^/]*/usr/lib/gdk-pixbuf-2.0/.*/loaders.cache
> [^/]*/usr/include/ruby-1.9.1/i386-linux
> [^/]*/usr/include/ruby-1.9.1/i386-linux/ruby
> [^/]*/usr/include/ruby-1.9.1/i386-linux/ruby/config.h
> [^/]*/usr/include/ruby-1.9.1/ruby/win32.h
> [^/]*/usr/lib/ruby/i386-linux
> [^/]*/usr/lib/ruby/i386-linux/fake.rb
> [^/]*/usr/lib/ruby/i386-linux/libruby.so.1.9.1
> [^/]*/usr/lib/ruby/i386-linux/libruby-static.a
> [^/]*/usr/lib/ruby/i386-linux/rbconfig.rb
> [^/]*/usr/lib/qt4/plugins/webkit
> [^/]*/usr/lib/qt5/plugins/webkit

I just want to be clear that this list is not 100% accurate. Some of
these files are not tracked by sstate however they are generated by
sstate 'postinsts' so they will be rebuilt upon a rebuild. Examples that
jump out are:

[^/]*/usr/lib/gdk-pixbuf-2.0/.*/loaders.cache
[^/]*/usr/bin/crossscripts/guile-config

Also, 

[^/]*/usr/lib/python3.3/__pycache__
[^/]*/usr/lib/python3.3/[^/]*/__pycache__
[^/]*/usr/lib/python3.3/[^/]*/[^/]*/__pycache__

are automatically generated/updated by python.

[^/]*/usr/src/kernel/scripts/.*

is generated by the modules class amongst other things. There was a
recent different bug report about these files not being cleaned and
triggering file conflict issues but that is a different problem.

I don't know much about the others and there are also some 'real' issues
here.

Cheers,

Richard