bitbake-setup update problem

bitbake-setup update problem

[Vyacheslav Yurkov]

It turned out to be a painful discovery.
I wanted to rebase my custom branch in oe-core on top of latest master 
before publishing a set of patches. But it seems like after running 
'bitbake-setup update' my branch is nowhere to be seen. Is it supposed 
to be like that? Any chance to get it restored?

Regards,
Slava

Re: [bitbake-devel] bitbake-setup update problem

[Yoann Congal]

[-- Attachment #1: Type: text/plain, Size: 1081 bytes --]

Le lun. 22 déc. 2025 à 09:56, Vyacheslav Yurkov via lists.openembedded.org
<uvv.mail=gmail.com@lists.openembedded.org> a écrit :

> It turned out to be a painful discovery.
> I wanted to rebase my custom branch in oe-core on top of latest master
> before publishing a set of patches. But it seems like after running
> 'bitbake-setup update' my branch is nowhere to be seen. Is it supposed
> to be like that? Any chance to get it restored?
>

Try "git reflog HEAD", you should see every commit HEAD has been on (and,
in theory, the commit at the tip of your branch)


>
> Regards,
> Slava
>
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> View/Reply Online (#18631):
> https://lists.openembedded.org/g/bitbake-devel/message/18631
> Mute This Topic: https://lists.openembedded.org/mt/116898936/4316185
> Group Owner: bitbake-devel+owner@lists.openembedded.org
> Unsubscribe: https://lists.openembedded.org/g/bitbake-devel/unsub [
> yoann.congal@smile.fr]
> -=-=-=-=-=-=-=-=-=-=-=-
>
>

-- 
Yoann Congal
Smile ECS

[-- Attachment #2: Type: text/html, Size: 2345 bytes --]

Re: [bitbake-devel] bitbake-setup update problem

[Vyacheslav Yurkov]

[-- Attachment #1: Type: text/plain, Size: 778 bytes --]

On 22.12.2025 10:09, Yoann Congal wrote:
>
>
> Le lun. 22 déc. 2025 à 09:56, Vyacheslav Yurkov via 
> lists.openembedded.org <http://lists.openembedded.org> 
> <uvv.mail=gmail.com@lists.openembedded.org> a écrit :
>
>     It turned out to be a painful discovery.
>     I wanted to rebase my custom branch in oe-core on top of latest
>     master
>     before publishing a set of patches. But it seems like after running
>     'bitbake-setup update' my branch is nowhere to be seen. Is it
>     supposed
>     to be like that? Any chance to get it restored?
>
>
> Try "git reflog HEAD", you should see every commit HEAD has been on 
> (and, in theory, the commit at the tip of your branch)
That's the first thing I'd tried. The whole layer seems to be a fresh clone.

Slava

[-- Attachment #2: Type: text/html, Size: 1979 bytes --]

Re: [bitbake-devel] bitbake-setup update problem

[Alexander Kanavin]

On Mon, 22 Dec 2025 at 10:14, Vyacheslav Yurkov via
lists.openembedded.org <uvv.mail=gmail.com@lists.openembedded.org>
wrote:
> Try "git reflog HEAD", you should see every commit HEAD has been on (and, in theory, the commit at the tip of your branch)
>
>
> That's the first thing I'd tried. The whole layer seems to be a fresh clone.

Sadly yes, this is how bitbake's git fetcher performs unpack(). It's a
forced overwrite, with no consideration for whether there have been
local changes. We do need a guard in bb-setup for this, I guess the
good one would check the revision that was previously unpacked (it's
written into config/sources-fixed-revisions.json), and if the revision
in the actual layer doesn't match then it would ask the user in the
same way that it asks about making build/conf/ changes.

Alex

Re: [bitbake-devel] bitbake-setup update problem

[Alexander Kanavin]

On Mon, 22 Dec 2025 at 12:18, Alexander Kanavin via
lists.openembedded.org <alex.kanavin=gmail.com@lists.openembedded.org>
wrote:
> Sadly yes, this is how bitbake's git fetcher performs unpack(). It's a
> forced overwrite, with no consideration for whether there have been
> local changes. We do need a guard in bb-setup for this, I guess the
> good one would check the revision that was previously unpacked (it's
> written into config/sources-fixed-revisions.json), and if the revision
> in the actual layer doesn't match then it would ask the user in the
> same way that it asks about making build/conf/ changes.

If the revision does match, it should also check if there are
additional local branches. If either is true then ask to choose one
of:
1. Stop the update right now.
2. Move the existing layer directory to a backup location.

Alex

Re: [bitbake-devel] bitbake-setup update problem

[Alexander Kanavin]

On Mon, 22 Dec 2025 at 12:25, Alexander Kanavin via
lists.openembedded.org <alex.kanavin=gmail.com@lists.openembedded.org>
wrote:
> > Sadly yes, this is how bitbake's git fetcher performs unpack(). It's a
> > forced overwrite, with no consideration for whether there have been
> > local changes. We do need a guard in bb-setup for this, I guess the
> > good one would check the revision that was previously unpacked (it's
> > written into config/sources-fixed-revisions.json), and if the revision
> > in the actual layer doesn't match then it would ask the user in the
> > same way that it asks about making build/conf/ changes.
>
> If the revision does match, it should also check if there are
> additional local branches. If either is true then ask to choose one
> of:
> 1. Stop the update right now.
> 2. Move the existing layer directory to a backup location.

I posted a patch which adds the local modifications check.
Unfortunately I couldn't figure out how to check for additional local
branches: it checks only if there are uncommitted changes, or changes
on the current branch. If you have ideas I'd appreciate (simply
running 'git branch' and counting the lines does not work, as git
fetcher's unpack first unpacks master, then checks out the final
branch if necessary, so both one and two branches can mean no local
modifications).

Alex

Re: [bitbake-devel] bitbake-setup update problem

[Alexander Kanavin]

On Fri, 2 Jan 2026 at 21:19, Alexander Kanavin via
lists.openembedded.org <alex.kanavin=gmail.com@lists.openembedded.org>
wrote:

> I posted a patch which adds the local modifications check.
> Unfortunately I couldn't figure out how to check for additional local
> branches: it checks only if there are uncommitted changes, or changes
> on the current branch. If you have ideas I'd appreciate (simply
> running 'git branch' and counting the lines does not work, as git
> fetcher's unpack first unpacks master, then checks out the final
> branch if necessary, so both one and two branches can mean no local
> modifications).

The more I think about it, the more I'm convinced git repositories
managed by bitbake-setup should have a commit hook that prevents
making commits altogether and tells users to use 'bitbake-setup init
-L repo local-path'.

The local modifications check should remain, to guard for cases where
there are uncommitted changes.

I'll make a v3.

Alex

Re: [bitbake-devel] bitbake-setup update problem

[Vyacheslav Yurkov]

On 08.01.2026 13:14, Alexander Kanavin wrote:
> On Fri, 2 Jan 2026 at 21:19, Alexander Kanavin via
> lists.openembedded.org <alex.kanavin=gmail.com@lists.openembedded.org>
> wrote:
>
>> I posted a patch which adds the local modifications check.
>> Unfortunately I couldn't figure out how to check for additional local
>> branches: it checks only if there are uncommitted changes, or changes
>> on the current branch. If you have ideas I'd appreciate (simply
>> running 'git branch' and counting the lines does not work, as git
>> fetcher's unpack first unpacks master, then checks out the final
>> branch if necessary, so both one and two branches can mean no local
>> modifications).
> The more I think about it, the more I'm convinced git repositories
> managed by bitbake-setup should have a commit hook that prevents
> making commits altogether and tells users to use 'bitbake-setup init
> -L repo local-path'.
>
> The local modifications check should remain, to guard for cases where
> there are uncommitted changes.
>
> I'll make a v3.
>
> Alex

This is why I don't like 'repo' and other wrappers that require that. I 
don't want to use another tool on top of git.

Slava

RE: [bitbake-devel] bitbake-setup update problem

[Peter Kjellerstedt]

> -----Original Message-----
> From: bitbake-devel@lists.openembedded.org <bitbake-devel@lists.openembedded.org> On Behalf Of Vyacheslav Yurkov via lists.openembedded.org
> Sent: den 18 januari 2026 09:48
> To: Alexander Kanavin <alex.kanavin@gmail.com>
> Cc: bitbake-devel@lists.openembedded.org; Yoann Congal <yoann.congal@smile.fr>
> Subject: Re: [bitbake-devel] bitbake-setup update problem
> 
> On 08.01.2026 13:14, Alexander Kanavin wrote:
> > On Fri, 2 Jan 2026 at 21:19, Alexander Kanavin via
> > lists.openembedded.org <alex.kanavin=gmail.com@lists.openembedded.org>
> > wrote:
> >
> >> I posted a patch which adds the local modifications check.
> >> Unfortunately I couldn't figure out how to check for additional local
> >> branches: it checks only if there are uncommitted changes, or changes
> >> on the current branch. If you have ideas I'd appreciate (simply
> >> running 'git branch' and counting the lines does not work, as git
> >> fetcher's unpack first unpacks master, then checks out the final
> >> branch if necessary, so both one and two branches can mean no local
> >> modifications).
> > The more I think about it, the more I'm convinced git repositories
> > managed by bitbake-setup should have a commit hook that prevents
> > making commits altogether and tells users to use 'bitbake-setup init
> > -L repo local-path'.
> >
> > The local modifications check should remain, to guard for cases where
> > there are uncommitted changes.
> >
> > I'll make a v3.
> >
> > Alex
> 
> This is why I don't like 'repo' and other wrappers that require that. I
> don't want to use another tool on top of git.
> 
> Slava

I'm not sure why you are pointing at repo here, because this is actually 
a use case that repo handles extremely well. We have been using repo for 
over ten years to maintain the manifests for all our builds, and (AFAIK) 
we have not had a single case of repo loosing any data due to doing an 
update.

What repo does is first, if there are uncommitted changes, then it will 
not update that specific repository until those changes have either been 
removed or committed. Second, it rebases all local commits during the 
update. This can of course result in a conflict, which then needs to be 
handled manually, just like any other Git conflict resolution. What's 
more, repo also handles a repository being removed and later readded 
without loosing any metadata. This is because repo actually does not 
keep most of Git's metadata in the .git directory for each repository, 
but rather in its own .repo directory. This allows for going back and 
forth in manifest revisions even when there are changes to which 
repositories are being checked out, without fear of losing any data.

//Peter

Re: [bitbake-devel] bitbake-setup update problem

[Alexander Kanavin]

On Tue, 20 Jan 2026 at 14:48, Peter Kjellerstedt
<peter.kjellerstedt@axis.com> wrote:

> I'm not sure why you are pointing at repo here, because this is actually
> a use case that repo handles extremely well. We have been using repo for
> over ten years to maintain the manifests for all our builds, and (AFAIK)
> we have not had a single case of repo loosing any data due to doing an
> update.
>
> What repo does is first, if there are uncommitted changes, then it will
> not update that specific repository until those changes have either been
> removed or committed. Second, it rebases all local commits during the
> update. This can of course result in a conflict, which then needs to be
> handled manually, just like any other Git conflict resolution. What's
> more, repo also handles a repository being removed and later readded
> without loosing any metadata. This is because repo actually does not
> keep most of Git's metadata in the .git directory for each repository,
> but rather in its own .repo directory. This allows for going back and
> forth in manifest revisions even when there are changes to which
> repositories are being checked out, without fear of losing any data.

There was some discussion at the yesterday's tech call: basically
adding a 'non-destructive unpack' feature to bitbake's git fetcher
would be most welcome, and then bitbake-setup could use it, and then
the restriction on making commits could be reverted. Such a feature
probably would work like the description above.

Unfortunately right now I'm swamped with some high priority
autobuilder issues (and there's a large backlog of other bitbake-setup
improvements to be reviewed and possibly adjusted from feedback that I
would really want to get in before the freeze), so can't promise where
and how it'll happen if it's left to me to implement.

Alex

Re: [bitbake-devel] bitbake-setup update problem

[Vyacheslav Yurkov]

[-- Attachment #1: Type: text/plain, Size: 506 bytes --]

On 20.01.2026 14:48, Peter Kjellerstedt wrote:
> I'm not sure why you are pointing at repo here, because this is actually
> a use case that repo handles extremely well. We have been using repo for
> over ten years to maintain the manifests for all our builds, and (AFAIK)
> we have not had a single case of repo loosing any data due to doing an
> update.
I didn't mean that it loses the data. I meant the need to run some other 
command _using_ repo so it is aware of what happens in the repository.

Slava

[-- Attachment #2: Type: text/html, Size: 944 bytes --]