[Buildroot] [PATCH 1/6] check-shlibs-deps: new script to check shared library dependencies

["Jérôme Pouiller" <jezz@sysmic.org>]

Hello Samuel,

On Monday 6 February 2017 22:04:23 CET Samuel Martin wrote:
> On Mon, Nov 14, 2016 at 2:22 PM, J?r?me Pouiller <jezz@sysmic.org> wrote:
[...]
> > +# Build dictionnaries from "build/packages-file-list.txt"
> > +def build_dicts(builddir):
> > +    pkgsdict = {}
> > +    filesdict = {}
> > +    with open(os.path.join(builddir, "build", "packages-file-list.txt")) as filelistf:
> > +        for line in filelistf.readlines():
> > +            pkg, fpath = line.split(",", 1)
> > +            fpath = fpath.strip()
> > +            fpath = os.path.join(builddir, "target", fpath)
> > +            fpath = os.path.normpath(os.path.relpath(fpath))
> 
> I wonder why relpath is needed here? since it is resloving relative
> path from the current location, i.e. BR's top directory.
> BTW, is it really the BR's top dir. you want to use here as start path
> to compute the relative path, not $(O)? How does this behave with
> out-of-tree build?

In subsequent case, I need to call realpath() in order to resolve
symlinks. realpath() return an absolute paths, but I prefer to convert
them in relative paths because they are smaller in error messages.

In case above, relpath is necessary because 'builddir' may be absolute.


> > +            add_file(pkgsdict, filesdict, fpath, pkg)
> > +    return filesdict, pkgsdict
> > +
> > +# Return package associated to a file
> > +def get_package(filesdict, fpath):
> > +    if not fpath in filesdict:
> > +        #print("WARNING: %s is not part of any package" % fpath)
> > +        # Do not flood user with warning messages. Especially host-gcc-final
> > +        # does not declare its files and produce many warnings.
> > +        filesdict[fpath] = "unknown"
> > +    return filesdict[fpath]
> > +
> > +# Return list of libraries linked with a binary
> > +def get_shared_libs(builddir, binary):
> > +    libs = set()
> > +    # Host readelf seems able to read foreign binaries (tested with arm/glibc and arm/uclibc)
> > +    pipe = subprocess.Popen([ "readelf", "-d", binary ], stdout=subprocess.PIPE)
> 
> Side note:
> There is this project [1] out-there that looks nice, but I wonder if
> it is worthwhile to add/embedded another python modules dependency to
> BR.

Interesting. (But not enough interesting to rewrite this script :) ) 

> > +    for line in pipe.stdout:
> > +        match = re.match("^.* \(NEEDED\) .*Shared library: \[(.+)\]$",
> > line)
> > +        if match:
> > +            libname = match.group(1)
> > +            # Support case where "/lib" s a symlink to "/usr/lib"
> > +            lpaths = set()
> > +            for dir in [ "usr/lib", "lib" ]:
> > +               
> > lpaths.add(os.path.relpath(os.path.realpath(os.path.join(builddir,
> > "target", dir, libname))))> 
> ditto for relpath/start path.\0

[...]

> > +# Return files found in "target/"
> > +def build_file_list(builddir):
> > +    files = set()
> > +    for dirpath, _, filelist in os.walk(os.path.join(builddir,
> > "target")):
> > +        for f in filelist:
> > +            file = os.path.join(dirpath, f)
> > +            file = os.path.relpath(os.path.realpath(file))
> 
> ditto for relpath/start path.

[...]

> > +    for p, pdeps in sorted(pkgdeps.items()):
> > +        if not deps == None:
> 
> s/deps == None/deps is None/

ok

> or simply: if deps:

I do want to make a difference between [] and None (see below). So I
think "if deps:" does not work.

[...]
> > +parser = argparse.ArgumentParser(description='Show or check binary dependencies based on linked shared libraries')
> > +
> > +parser.add_argument("--builddir", '-b', metavar="BUILDDIR", required=True,
> > +        help="Buildroot output directory")
> > +parser.add_argument("--package", '-p', metavar="PACKAGE",
> > +        help="Check only PACKAGE (else, show dpendencies of all binairies)")
> > +parser.add_argument("--deps", '-d', metavar="DEP1,DEP2,...", nargs='?', default="",
> > +        help="Report errors if found dependencies are not a subset of '--deps'. Pass "". '-p' is mandatory with this option")
> > +parser.add_argument('-w', action='store_true',
> > +        help="Do not return non zero when dependency is missing")
> > +args = parser.parse_args()
> > +if not args.package and args.deps:
> > +    print("ERROR: cannot use --deps wihout --package")
> > +    exit(1)
> > +
> > +if args.deps == "":
> > +    deps = None
> > +elif args.deps == None:
> > +    deps = []
> 
> What is the difference between "deps = None" and "deps = []"?

'None' means user don't to check dependencies (-d is not used) while '[]'
means package shouldn't have dependencies.

However, my current implementation is broken. Following command does not
work as expected:

  check-shlibs-deps -b OUTDIR -p PACKAGE -d ""

I will change that.

> > +else:
> > +    deps = args.deps.split(",")
> > +
[...]

-- 
J?r?me Pouiller, Sysmic
Embedded Linux specialist
http://www.sysmic.fr