git and dir

git and dir

[Rolf Leggewie]

Hi,

this is something I've been pondering upon today.  I'm hoping for some 
discussion, an opportunity to learn and possibly to make OE even better.

It has always been possible to shoot yourself in the foot with OE.  I 
believe the opportunities will rise with the introduction of git.  The 
reason is that AFAIU, with mtn there was a 1:1 relation between checkout 
dir and upstream branch.  This information was recorded in _MTN/options. 
  git does not have this restriction.  One can freely add remote 
branches to an existing checkout with "git remote add" and switch to 
them with "git checkout $branch".  This can easily corrupt tmp/ and I 
fear it will happen quite often.  I wonder if there is anything 
OE/bitbake can do to alleviate the pain.

Regards

Rolf


PS: I am just thinking if it made sense to add the possibility to 
include the git branch name in TMPDIR just like we can include DISTRO 
now.  I think that would go a long way towards making problems to occur 
only seldomly if at all.

Re: git and dir

[Phil Blundell]

On Thu, 2008-08-14 at 01:33 +0200, Rolf Leggewie wrote:
> It has always been possible to shoot yourself in the foot with OE.  I 
> believe the opportunities will rise with the introduction of git.  The 
> reason is that AFAIU, with mtn there was a 1:1 relation between checkout 
> dir and upstream branch.  This information was recorded in _MTN/options. 
>   git does not have this restriction.  One can freely add remote 
> branches to an existing checkout with "git remote add" and switch to 
> them with "git checkout $branch".  This can easily corrupt tmp/ and I 
> fear it will happen quite often.  I wonder if there is anything 
> OE/bitbake can do to alleviate the pain.

What's the specific corruption that you're worried about?  Any change
which is truly incompatible in TMPDIR ought to result in a change to
SANITY_ABI which will be automatically detected.  Smaller changes ought
not to cause any real problem in the first place.

I'm also not convinced that the operations you mention here with git are
not already possible with monotone.  Isn't this basically what "monotone
update -b BRANCH" does?

Maybe you could give a concrete example of the kind of operation that
you think is going to get users into trouble.

p.

Re: git and dir

[Rolf Leggewie]

Phil Blundell wrote:
> What's the specific corruption that you're worried about?

Maybe my worries are unfounded and that would be great.  I was thinking 
that building for a certain distribution but changing the checked out 
repo underneath its feet back and forth could eventually result in 
incompatible stuff getting into TMPDIR and particularly staging.  Think 
switching between dev, stable, openmoko and some of its variant and 
building stuff in between.  I don't think it is too far-fetched to worry 
this might lead to some incompatabilities in what is under TMPDIR.

Re: git and dir

[Phil Blundell]

On Thu, 2008-08-14 at 13:45 +0200, Rolf Leggewie wrote:
> Phil Blundell wrote:
> > What's the specific corruption that you're worried about?
> 
> Maybe my worries are unfounded and that would be great.  I was thinking 
> that building for a certain distribution but changing the checked out 
> repo underneath its feet back and forth could eventually result in 
> incompatible stuff getting into TMPDIR and particularly staging.  Think 
> switching between dev, stable, openmoko and some of its variant and 
> building stuff in between.  I don't think it is too far-fetched to worry 
> this might lead to some incompatabilities in what is under TMPDIR.

I guess it depends on what you mean by "incompatibilities".

If you start switching between branches like that then, obviously, the
binaries that you create are liable to contain a mismatch of things and
might not be entirely compatible with any of the branches.  For example,
if I check out the dev tree, build glibc from that, then switch to
stable and start trying to build some other packages then I will
probably end up with a dependency on a version of glibc that doesn't
exist in stable.  But this has always been the case and the answer is
simply "don't do that, then".

So, in summary, I don't think there's any new issue here and it doesn't
really seem that any particular action is required to address it.

p.

Re: git and dir

[Mike (mwester)]

Rolf Leggewie wrote:
> Phil Blundell wrote:
>> What's the specific corruption that you're worried about?
> 
> Maybe my worries are unfounded and that would be great.  I was thinking
> that building for a certain distribution but changing the checked out
> repo underneath its feet back and forth could eventually result in
> incompatible stuff getting into TMPDIR and particularly staging.  Think
> switching between dev, stable, openmoko and some of its variant and
> building stuff in between.  I don't think it is too far-fetched to worry
> this might lead to some incompatabilities in what is under TMPDIR.

I see both sides of this discussion.  Rolf brings up a very valid point;
a well-intentioned "newbie" may waste a lot of their time -- and the
time of others offering assistance.  On the other hand, I can see many
cases where switching about on minor branches would be useful, and if
one knows what they are doing, it could be a huge timesaver if TMPDIR
could be reused.

So can we consider a middle ground?  A warning message issued if the
current branch doesn't match the one recorded inside TMPDIR?  This could
be ignored by the experts, but it would be something easy to spot in
(for example) a pastebin offered by a struggling "non-expert" user.

Regarding the specific issue -- the openmoko users have already observed
cases where TMPDIR becomes non-useful because they thought they could
switch among branches/repos without penalty, resulting in incorrect
versions in staging, and other build issues.

Regards,
Mike (mwester)

Re: git and dir

[Rolf Leggewie]

Mike (mwester) wrote:
> So can we consider a middle ground?

How about this?  If $GITBRANCH was available when defining TMPDIR it 
would be just perfectly possible to define it as TMPDIR = 
"${OEHOME}/tmp/${DISTRO}/${GITBRANCH}" and not have any issues.  For 
those who want to share stuff and know what they are doing, it would be 
possible to leave out DISTRO and GITBRANCH fi they want to.

The issue that I see is that although it may have been technically 
possible to switch branches in the same directory with mtn, I would 
consider that a very advanced technique (I consider myself rather 
well-versed in mtn and didn't even know it was possible).  Whereas, this 
is kind of the norm with git.  The easy switching is one of the reasons 
we decided to go forward with git.  Just like Mike said, I think we are 
going to see a lot of wasted time for everyone and I think now is a good 
time to proactively prevent that.

Re: git and dir

[Florian Boor]

Hi,

Rolf Leggewie schrieb:
> "git checkout $branch".  This can easily corrupt tmp/ and I fear it will
> happen quite often.  I wonder if there is anything OE/bitbake can do to
> alleviate the pain.

well, you can't stop people from shooting themselves in their feet entirely. Its
even easier changing your package repository between multiple branches changing
the path in environment or local.conf. Do you want to prevent people from doing
this too?
I do not say the situation is ideal, but if we want to start working on
usability issues we should start with more important ones.

Greetings

Florian

-- 
The dream of yesterday                  Florian Boor
is the hope of today                    Tel: +49 271-771091-15
and the reality of tomorrow.            Fax: +49 271-771091-19
[Robert Hutchings Goddard, 1904]        florian.boor@kernelconcepts.de

1D78 2D4D 6C53 1CA4 5588  D07B A8E7 940C 25B7 9A76