From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from 93-97-173-237.zone5.bethere.co.uk ([93.97.173.237] helo=tim.rpsys.net) by linuxtogo.org with esmtp (Exim 4.72) (envelope-from ) id 1Stl1Q-0006KE-7X for openembedded-core@lists.openembedded.org; Tue, 24 Jul 2012 21:43:56 +0200 Received: from localhost (localhost [127.0.0.1]) by tim.rpsys.net (8.13.6/8.13.8) with ESMTP id q6OJWTnm000779; Tue, 24 Jul 2012 20:32:29 +0100 Received: from tim.rpsys.net ([127.0.0.1]) by localhost (tim.rpsys.net [127.0.0.1]) (amavisd-new, port 10024) with LMTP id 00559-02; Tue, 24 Jul 2012 20:32:25 +0100 (BST) Received: from [192.168.3.10] ([192.168.3.10]) (authenticated bits=0) by tim.rpsys.net (8.13.6/8.13.8) with ESMTP id q6OJWLCC000773 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Tue, 24 Jul 2012 20:32:22 +0100 Message-ID: <1343158340.22222.22.camel@ted> From: Richard Purdie To: Yao Zhao Date: Tue, 24 Jul 2012 20:32:20 +0100 In-Reply-To: <500EF6B1.8040703@windriver.com> References: <1343137799-12032-1-git-send-email-yao.zhao@windriver.com> <500EEBE2.5020106@windriver.com> <1343156850.22222.20.camel@ted> <500EF6B1.8040703@windriver.com> X-Mailer: Evolution 3.2.2- Mime-Version: 1.0 X-Virus-Scanned: amavisd-new at rpsys.net Cc: Patches and discussions about the oe-core layer Subject: Re: [PATCH] bzip2-native: fix problems when bzip2-native is installed in parallel X-BeenThere: openembedded-core@lists.openembedded.org X-Mailman-Version: 2.1.11 Precedence: list Reply-To: Patches and discussions about the oe-core layer List-Id: Patches and discussions about the oe-core layer List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 24 Jul 2012 19:43:56 -0000 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit On Tue, 2012-07-24 at 15:25 -0400, Yao Zhao wrote: > On 12-07-24 03:07 PM, Richard Purdie wrote: > > On Tue, 2012-07-24 at 13:39 -0500, Mark Hatle wrote: > >> On 7/24/12 8:57 AM, Burton, Ross wrote: > >>> On 24 July 2012 14:49, Yao Zhao wrote: > >>>> when bzip2-native is installed in parallel to sysroot, it is possible that > >>>> some packages are using bzip2 to unpack, there are chances that bzip2 is > >>>> installed to sysroot but libbz2.so.0 not installed yet because parallel > >>>> installation. > >>>> link bzip2 and bzip2recover statically to avoid this problem and don't lose > >>>> parallel installation. libbz2.so is still available. > >>> Is it me, or is this officially getting silly? This probably happens > >>> for *every* binary in the sysroot that links to a library, which is > >>> probably a fair proportion of them. Statically linking every single > >>> one and then special-casing further problems where a static link isn't > >>> sufficient (see pythonnative) just isn't going to scale. > >> The problem is that there are a handful of things that are needed, and for some > >> reason (valid or not) packages are not "requiring", either because they assume > >> the system is valid or they simply don't know they have the requirement. > >> > >> Both python and perl may be used by a number of auto* utilities as well as by > >> packages themselves. We could attempt to add DEPENDS for all of the cases, but > >> is it really worth it? I suspect we'll end up specify the DEPENDS for 90% of > >> the things in the system, simply due to them incidentally running some system > >> level python or perl script. If we make the python/perl into external items > >> that only get loaded into the environment when we're building python/perl > >> modules then things become increasingly easy. > >> > >> As for the bzip2 issue.. This is another of those "we assume it's valid > >> binaries".. And the problem appears to be when it gets build, it's not valid > >> for a small window of time. > >> > >> From what Yao has explained to me (offline) the issue is that the > >> assume_provided for bzip2-native works just fine, we use the system > >> version...but then python (or other) utilities come in and say they need > >> "bzip2-full-native". Which triggers the build.. and opens a small window of > >> time where bzip2 is being installed, where it isn't functional -- something else > >> comes in and has a requirement of bzip2-native, which is satisfied by the > >> assume_provided and we get into the race where it executes a broken binary. > >> > >> The underlying issue is simply that if we're installing tooling that is either > >> assume-provided (based on an alternative 'full' version) or incidental usage > >> like python or perl we need to take precautions to ensure that the build system > >> never sees the "broken" version during the short install window triggering the > >> race condition. > >> > >> I'll let Yao comment further on this particular issue, but there is an overall > >> class of issues with -native that we need to track to get rid of these subtle races. > > The set of problems are cases where we have things in ASSUME_PROVIDED. > > It is near impossible to have an empty set there and sometimes we need > > to build things ourselves which are already listed there. python-native > > and perl-native are the pathological cases but as also have issues with > > bzip2, tar and likely undiscovered yet, now git. > > > > I'd been searching for bzip2-replacement-native rather than full which > > is why I didn't see this earlier. I suspect in the python/bzip2 case, > > not installing the bzip2 binary might well solve many of the problems as > > python really wants a native libbz2 and the host bzip2 binary is ok. > Hi Richard, > > Can I provide a patch to change bzip2-full-native to bzip2-native to > finish this problem? > I have tested that the bzip2-native is not built any more. No, since I think the libbz2 component that python needs will then not be present and this causes other problems (see the previous commit or mailing list archives for what the exact issue was, I don't remember offhand). Cheers, Richard