Re: Deployment for machine X will remove its results from machine Y's deploy dir

[Mike Looijmans <mike.looijmans@topic.nl>]

On 27-11-2014 15:41, Richard Purdie wrote:
> On Thu, 2014-11-27 at 15:22 +0100, Mike Looijmans wrote:
>> On 11/27/2014 02:17 PM, Richard Purdie wrote:
>>> On Thu, 2014-11-27 at 05:02 -0700, Gary Thomas wrote:
>>>> On 2014-11-27 01:35, Mike Looijmans wrote:
>>>>> Here's an example recipe to demonstrate the issue. Save it as "deployme.bb" into a recipe dir. Then build it for two machines. Building it for one machine will remove it from the
>>>>> deployment directory of the other. This problem has been bugging me for months, I had files just "disappear" mysteriously from the deploy directory and seemingly random times, and
>>>>> now I finally figured out what causes it.
>>>>>
>>>>> (cut here)
>>>>>
>>>>> SUMMARY = "Demonstrate a bug in OE deployment"
>>>>> DESCRIPTION = "Build this package for a machine X, then look at the image's \
>>>>>     deploy directory. You'll see a deployme.txt there. Now build it for another \
>>>>>     machine, e.g. "Y". The deployme.txt for machine X will have disappeared \
>>>>>     from the image dir. This appears to be a bug in OE's deployment."
>>>>> LICENSE = "GPLv2"
>>>>> LIC_FILES_CHKSUM = "file://${COREBASE}/LICENSE;md5=4d92cd373abda3937c2bc47fbc49d690"
>>>>>
>>>>> inherit allarch deploy
>>>>>
>>>>> do_compile () {
>>>>>        echo "Hello world!" > deployme.txt
>>>>> }
>>>>>
>>>>> do_deploy () {
>>>>>        install -d ${DEPLOYDIR}
>>>>>        install -m 644 ${B}/deployme.txt ${DEPLOYDIR}/
>>>>> }
>>>>>
>>>>> addtask deploy before do_build after do_compile
>>>>>
>>>>> (cut here)
>>>>
>>>> Very interesting & verified with the latest master.
>>>>
>>>> Have you filed a bug?  https://bugzilla.yoctoproject.org/
>>>
>>> Well, I'm not convinced this is a bug as such. You've created an
>>> "allarch" deploy task, how would you expect this to behave?
>>>
>>> "allarch" means that the output from this task is universal and can be
>>> used on all targets. It will therefore get run once.
>>>
>>> A "deploy" task is machine specific.
>>>
>>> What ends up happening is therefore the task has a stamp is
>>> "universally" created. When you change machine, the checksum of the task
>>> changes, the previous version is removed, the new version is installed.
>>>
>>> So in many ways the system is doing exactly what I would expect it to do
>>> and it isn't a bug in that sense.
>>
>> It's not a bug in the sense that it doesn't do as it was programmed to do.
>
> Its doing *exactly* what the was designed to do. That doesn't match what
> you want/expect though.
>
>>   I understand what's happening here.
>>
>> I just think that the logic here is wrong. If "deploy" is machine specific,
>> then the implicit "undeploy" should be machine specific too, right?
>
> Well, its more complex than that.
>
> deploy.bbclass defaults to DEPLOY_DIR_IMAGE.
>
> DEPLOY_DIR_IMAGE defaults to ${DEPLOY_DIR}/images/${MACHINE}
>
> I actually put off merging the latter since I knew it would cause
> issues, I just couldn't articulate all of them at the time :(.
>
>>> The real question is how should an "allarch" + "deploy" task behave when
>>> you've specified machine specific paths? Perhaps erroring would be
>>> better?
>>
>> That would mean that roughly all deploy tasks will fail.
>
> I'm not sure we have many deploy+allarch tasks so I think "roughly none"
> would be a better description.
>
> deploy is usually used for bootloaders and kernels, both of which are
> not allarch.
>
>> At best they're tied
>> to MACHINE_ARCH, but never to MACHINE itself.
>
> No, they're tried to MACHINE itself, see above.
>
>> Would be strange to put PACKAGE_ARCH="${MACHINE}" in a recipe that clearly has
>> no dependency on machine specific things. And I wrote "${MACHINE}" here on
>> purpose.
>
> Dropping the "allarch" would be better than that.
>
>> I was thinking along the lines of "DEPLOY_DIR_IMAGE must have the same prefix"
>> or so.
>>
>> If I knew the solution, I'd have posted a patch instead of a question or report.
>
> Well, allarch.bbclass could override DEPLOY_DIR_IMAGE to remove
> the /${MACHINE} suffix. That would appear to fix the issues you're
> seeing, at the risk of having a different group of people upset that you
> don't get a complete directory per machine.
>
> It comes down to which behaviour we want. Changing MACHINE in the
> definition of DEPLOY_DIR_IMAGE to PACKAGE_ARCH might be the better
> solution, then it will deploy based on how specific or not specific the
> resulting code it. That will likely upset certain people even more
> though. The other option is to accept that its machine specific and do
> PACKAGE_ARCH="${MACHINE_ARCH}" in the class. That is suboptimal for the
> reasons you describe but would get the behaviour some people want.
>
> I suspect there isn't a right answer here :(
>
> An opinion from Martin/Koen might be useful at this point since it could
> affect them more than others (as well as things like the yocto project
> autobuilder output and output processing/testing/releasing).


If I understand correctly, having two "MACHINE" share the same 
MACHINE_ARCH (which is the case for several intel boards too), is not 
actually allowed?

Because you get the same kernel/bootloader for different machines then. 
That's what I wanted, and that triggered me finding this issue.

Please realize that the example is just an example. The actual problem 
I'm experiencing is with the kernel and bootloader!


-- 
Mike Looijmans