From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: by yocto-www.yoctoproject.org (Postfix, from userid 118) id CDFD3E00CF5; Wed, 25 May 2016 02:25:18 -0700 (PDT) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on yocto-www.yoctoproject.org X-Spam-Level: X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00 autolearn=ham version=3.3.1 X-Spam-HAM-Report: * -1.9 BAYES_00 BODY: Bayes spam probability is 0 to 1% * [score: 0.0000] Received: from dan.rpsys.net (5751f4a1.skybroadband.com [87.81.244.161]) by yocto-www.yoctoproject.org (Postfix) with ESMTP id 84DE2E00CF2 for ; Wed, 25 May 2016 02:25:15 -0700 (PDT) Received: from localhost (localhost [127.0.0.1]) by dan.rpsys.net (8.14.4/8.14.4/Debian-4.1ubuntu1) with ESMTP id u4P9PBAV016967; Wed, 25 May 2016 10:25:11 +0100 Received: from dan.rpsys.net ([127.0.0.1]) by localhost (dan.rpsys.net [127.0.0.1]) (amavisd-new, port 10024) with LMTP id D-IhTI74SF-O; Wed, 25 May 2016 10:25:11 +0100 (BST) Received: from hex ([192.168.3.34]) (authenticated bits=0) by dan.rpsys.net (8.14.4/8.14.4/Debian-4.1ubuntu1) with ESMTP id u4P9P5Tx016964 (version=TLSv1/SSLv3 cipher=AES128-GCM-SHA256 bits=128 verify=NOT); Wed, 25 May 2016 10:25:06 +0100 Message-ID: <1464168305.9570.98.camel@linuxfoundation.org> From: Richard Purdie To: "Smith, Elliot" , "Barros Pena, Belen" Date: Wed, 25 May 2016 10:25:05 +0100 In-Reply-To: References: X-Mailer: Evolution 3.16.5-1ubuntu3.1 Mime-Version: 1.0 Cc: "toaster@yoctoproject.org" Subject: Re: improving display of build artifacts X-BeenThere: toaster@yoctoproject.org X-Mailman-Version: 2.1.13 Precedence: list List-Id: Web based interface for BitBake List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 25 May 2016 09:25:18 -0000 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit On Wed, 2016-05-25 at 09:28 +0100, Smith, Elliot wrote: > Thanks for putting this together. > > Some of these should be relatively easy (like 1.1, 1.2, 2.2). > > (NB there is also > https://bugzilla.yoctoproject.org/show_bug.cgi?id=8556, which is > really the same bug: image_fstypes *is* being passed, but we don't > detect the output vmdk file correctly. This should also be relatively > easy to fix.) > > However, the fundamental problem at the moment is that we don't get a > precise notification about which files are built when, or about which > targets they belong to. We rely on trying to guess and erring on the > side of caution, which is why we don't show files we have previously > associated with a build. > > I am going to spend some time trying to figure out a way to provide > this information, as anything we do without this is going to be prone > to error and guesswork (which is why it works as it does at the > moment). I can give a braindump here which may or may not help. In the cases where we need to track artefacts in general we use sstate. sstate works by providing an output directory which the task writes data to, then sstate itself moves the files into the final position. We use sstate for deploy, package, package_write_*, packagedata and so on but one thing we don't use it for are images. Why? Images are large and we don't want the overhead of compressing the image into an sstate tarball or the overhead of downloading such sstate artefacts when its likely cheaper to build the image. There is one task we take a "half way house" approach with which is the kernel source population. Here, it behaves like an sstate task from a dependency graph point of view but its generally cheaper to run the build rather than compress and extract it. I have wondered if we should make images behave a bit more like sstate, in that they could write to an intermediate directory which is then moved into position. That way, toaster could know that a set of files come from a given task for sure. Its obviously a pretty invasive change in behaviour but also partly in keeping with the rework of the image generation we've been pursusing. Is it just the do_image_* output we have the issue with? Cheers, Richard