Dropbox and "Your local changes to the following files would be overwritten by merge"

Dropbox and "Your local changes to the following files would be overwritten by merge"

[Yuri Kanivetsky]

Hi,

I've started to run into issues with git relatively recently. Let's
take the issue I'm experiencing at the moment. I'm trying to rewrite a
history (git rebase -i some-hash, move 4 commits up, rename the last 2
of them), and it fails to apply the next commit on the todo list after
these 4 (I've redacted the message):

[detached HEAD 1888888] name-3
 Date: Mon May 9 22:41:03 2022 +0300
 1 file changed, 1 insertion(+)
[detached HEAD 00b5555] name-4
 Date: Sat Jul 23 13:19:42 2022 +0300
 1 file changed, 1 insertion(+)
error: Your local changes to the following files would be overwritten by merge:
        some paths
Please commit your changes or stash them before you merge.
Aborting
hint: Could not execute the todo command
hint:
hint:     pick 8222222222222222222222222222222222222222 name-5
hint:
hint: It has been rescheduled; To edit the command before continuing, please
hint: edit the todo list first:
hint:
hint:     git rebase --edit-todo
hint:     git rebase --continue
Could not apply 8222222... name-5

The files it complains about, they're versioned, and they're not
changed when it stops. If I copy the repository (cp -r), and repeat
the steps, it sometimes succeeds, sometimes not. If I were to split it
into smaller steps I believe it would succeed. In other words I got
this message occasionally for no apparent reason. Usually it succeeds
if I simply repeat the steps. Sometimes I need to do it in smaller
steps.

First I thought it's an issue with vim-fugitive, but I can reproduce
it with git alone. The other culprits I can think of are magit (I
believe I used it with this repository), vim, and now that I think
about it... Dropbox? The repository is in a dropbox directory. I
understand that this is uncommon, but a couple of my repositories are
in a dropbox directory and I'd like to put the reasons aside. Can you
suggest what I can do to find the reason it fails? Do you know what
exactly triggers this message (the content changed, modification time,
ctime)?

I'm running Alpine Linux and git-2.36.1.

Regards,
Yuri

RE: Dropbox and "Your local changes to the following files would be overwritten by merge"

[rsbecker]

On July 23, 2022 8:40 AM, Yuri Kanivetsky wrote:
>I've started to run into issues with git relatively recently. Let's take the issue I'm
>experiencing at the moment. I'm trying to rewrite a history (git rebase -i some-
>hash, move 4 commits up, rename the last 2 of them), and it fails to apply the next
>commit on the todo list after these 4 (I've redacted the message):
>
>[detached HEAD 1888888] name-3
> Date: Mon May 9 22:41:03 2022 +0300
> 1 file changed, 1 insertion(+)
>[detached HEAD 00b5555] name-4
> Date: Sat Jul 23 13:19:42 2022 +0300
> 1 file changed, 1 insertion(+)
>error: Your local changes to the following files would be overwritten by merge:
>        some paths
>Please commit your changes or stash them before you merge.
>Aborting
>hint: Could not execute the todo command
>hint:
>hint:     pick 8222222222222222222222222222222222222222 name-5
>hint:
>hint: It has been rescheduled; To edit the command before continuing, please
>hint: edit the todo list first:
>hint:
>hint:     git rebase --edit-todo
>hint:     git rebase --continue
>Could not apply 8222222... name-5
>
>The files it complains about, they're versioned, and they're not changed when it
>stops. If I copy the repository (cp -r), and repeat the steps, it sometimes succeeds,
>sometimes not. If I were to split it into smaller steps I believe it would succeed. In
>other words I got this message occasionally for no apparent reason. Usually it
>succeeds if I simply repeat the steps. Sometimes I need to do it in smaller steps.
>
>First I thought it's an issue with vim-fugitive, but I can reproduce it with git alone.
>The other culprits I can think of are magit (I believe I used it with this repository),
>vim, and now that I think about it... Dropbox? The repository is in a dropbox
>directory. I understand that this is uncommon, but a couple of my repositories are
>in a dropbox directory and I'd like to put the reasons aside. Can you suggest what I
>can do to find the reason it fails? Do you know what exactly triggers this message
>(the content changed, modification time, ctime)?
>
>I'm running Alpine Linux and git-2.36.1.

Is DropBox 100% up to date when the operation is run? It uses timestamps to determine whether files changed. Git uses SHA1. So, they each have a different perspective on what is changed. A git status before the operation would tell you what is modified prior to the merge, which could be something DropBox is updating or is pending an update. You could try to do a git stash push prior to the merge and then a git stash pop to isolate whatever DropBox is doing at the time.
--Randall

Re: Dropbox and "Your local changes to the following files would be overwritten by merge"

[brian m. carlson]

[-- Attachment #1: Type: text/plain, Size: 1391 bytes --]

On 2022-07-23 at 12:40:20, Yuri Kanivetsky wrote:
> Hi,

Hey,

> First I thought it's an issue with vim-fugitive, but I can reproduce
> it with git alone. The other culprits I can think of are magit (I
> believe I used it with this repository), vim, and now that I think
> about it... Dropbox? The repository is in a dropbox directory. I
> understand that this is uncommon, but a couple of my repositories are
> in a dropbox directory and I'd like to put the reasons aside. Can you
> suggest what I can do to find the reason it fails? Do you know what
> exactly triggers this message (the content changed, modification time,
> ctime)?

It's known that Dropbox and other cloud syncing services can corrupt
repositories.  They can cause files to be removed, renamed, or reappear
after being deleted, and I suspect that's what's happening here.

I'd definitely suggest moving your repository out of the Dropbox folder
and seeing if you can reproduce the problem.  That will tell you if
that's the problem you're having.

I'd recommend moving the data out of the Dropbox repository regardless,
since, as I mentioned, it is known to cause corruption, and you won't
know it until it's too late.  You may want to additionally run "git
fsck" to see if that finds any problems other than dangling objects.
-- 
brian m. carlson (he/him or they/them)
Toronto, Ontario, CA

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 263 bytes --]

Re: Dropbox and "Your local changes to the following files would be overwritten by merge"

[Junio C Hamano]

"brian m. carlson" <sandals@crustytoothpaste.net> writes:

> It's known that Dropbox and other cloud syncing services can corrupt
> repositories.  They can cause files to be removed, renamed, or reappear
> after being deleted, and I suspect that's what's happening here.
> ...
> I'd recommend moving the data out of the Dropbox repository regardless,
> since, as I mentioned, it is known to cause corruption, and you won't
> know it until it's too late.  You may want to additionally run "git
> fsck" to see if that finds any problems other than dangling objects.

An anticipated follow-up question is "but then how do I replicate
the repository across my machines? that is the reason why I have it
in Dropbox in the first place", and the above is a hard-to-follow
advice until it has a good answer.

E.g. after finished working at a site, prepare to take the "latest"
to the next place you will work at in a bundle,

    $ git bundle create 2022-07-23.bndl --all HEAD
    $ mv 2022-07-23.bndl $my_dropbox_directory/.

and when you get to the next place to work at, you'd "fetch" from it
into the repository you keep, or you can choose *NOT* to maintain
any local copy there at all and "git clone" from the bundle every
time you restart at a different place.

Re: Dropbox and "Your local changes to the following files would be overwritten by merge"

[Junio C Hamano]

Junio C Hamano <gitster@pobox.com> writes:

> E.g. after finished working at a site, prepare to take the "latest"
> to the next place you will work at in a bundle,
>
>     $ git bundle create 2022-07-23.bndl --all HEAD
>     $ mv 2022-07-23.bndl $my_dropbox_directory/.
>
> and when you get to the next place to work at, you'd "fetch" from it
> into the repository you keep, or you can choose *NOT* to maintain
> any local copy there at all and "git clone" from the bundle every
> time you restart at a different place.

A practical real-world solution would certainly involve keeping
reasonably fresh copies of repositories at each work location, and
"git fetch --mirror" into it from the bundle.  The "bundle create"
command would most likely have some bottom revisions to reduce the
size of the resulting bundle files, i.e. if you know that all copies
of repositories you'd work with surely have the version you tagged
"v53.8" a few weeks ago, then the "bundle create" command may have
"--not v53.8" at the end.

In other words, the above is *NOT* meant to be a full solution, but
merely to illustrate a possible direction to go in.

Re: Dropbox and "Your local changes to the following files would be overwritten by merge"

[Yuri Kanivetsky]

Hi,

I'm not sure if it's appropriate with mailing lists, but I'd like to
respond to you all with one email. Correct me if I'm doing it wrong.

A couple of points first. The issue happens sometimes. And if my
memory serves me right, it might be that at some point it fails, but
then I repeat the steps, and it succeeds. Then, there seems to be no
reason for Dropbox to change the files. That is, it just failed, I'm
in the middle of a rebase, and Dropbox says that all devices but one
did something at least about a day ago. The laptop is up for a while
now. And anyway, it's the only machine where I change the repository.
If I did change something on other machines, that was long ago.

rsbecker, it fails particularly when Dropbox is up to date. In case it
matters, I don't do merges, I do rebases. And if I were to expect
Dropbox to change the versioned files, I wouldn't do the git stash
thing.

brian, I'm planning to move the repository out of the dropbox, but
first I'd like to figure out what exactly is happening. Move it out to
confirm if it's a Dropbox issue? That wouldn't be a 100% proof. But I
bet it'd succeed. Anyway, it does seem like a Dropbox issue. And I'm
planning to keep an eye on whether it happens only with dropbox
repositories (most of my repositories are not in the dropbox).

> You may want to additionally run "git fsck" to see if that finds any problems other than dangling objects.

I'm in the middle of an issue, so to say, rebase stopped and `git
fsck` reports one missing tree, besides dangling objects.

Junio, thanks for the suggestion to use bundles. I didn't know about
them, and it looks like an interesting option I should consider. But
again, first I'd like to figure out what exactly is happening.

Actually, I also tried to build git to add some debug statements, but
got stuck with some missing dependencies.

At the end I'd like to provide details about the latest case. Maybe
you can infer something from it. It looks like it tried to apply a
commit it already applied a second time. I have a hard time
interpreting the current state. It's at commit "b", but
.git/rebase-merge/stopped-sha points to commit "c", and
.git/rebase-merge/done says it applied commit "b" twice. Although I'm
not sure I'm interpreting it right.

What I did:

$ git rebase -i --root # and changed the first pick into edit
$ git add some-file

$ git rebase --continue
error: Your local changes to the following files would be overwritten by merge:
        some-other-file
Please commit your changes or stash them before you merge.
Aborting
hint: Could not execute the todo command
hint:
hint:     pick 0008743 commit c
hint:
hint: It has been rescheduled; To edit the command before continuing, please
hint: edit the todo list first:
hint:
hint:     git rebase --edit-todo
hint:     git rebase --continue
Could not apply 0008743... commit c

$ git log --all --graph --oneline --decorate
* 3774372 (HEAD) commit b
* 2ccb75d commit a
...
* 9adb050 commit d
* 0008743 commit c
* 3e0cc95 commit b
* 6df3ef9 commit a

.git/rebase-merge/done:
edit 6df3ef9 commit a
pick 3e0cc95 commit b
pick 0008743 commit c
pick 3e0cc95 commit b

.git/rebase-merge/git-rebase-todo:
pick 0008743 commit c
pick 9adb050 commit d
...

.git/rebase-merge/git-rebase-todo.backup:
pick 6df3ef9 commit a
pick 3e0cc95 commit b
pick 0008743 commit c
pick 9adb050 commit d
...

.git/rebase-merge/message:
commit c

.git/rebase-merge/msgnum:
3

.git/rebase-merge/rewritten-list:
6df3ef9 2ccb75d
3e0cc95 3774372

.git/rebase-merge/stopped-sha:
0008743

Regards,
Yuri

On Sat, Jul 23, 2022 at 9:19 PM Junio C Hamano <gitster@pobox.com> wrote:
>
> Junio C Hamano <gitster@pobox.com> writes:
>
> > E.g. after finished working at a site, prepare to take the "latest"
> > to the next place you will work at in a bundle,
> >
> >     $ git bundle create 2022-07-23.bndl --all HEAD
> >     $ mv 2022-07-23.bndl $my_dropbox_directory/.
> >
> > and when you get to the next place to work at, you'd "fetch" from it
> > into the repository you keep, or you can choose *NOT* to maintain
> > any local copy there at all and "git clone" from the bundle every
> > time you restart at a different place.
>
> A practical real-world solution would certainly involve keeping
> reasonably fresh copies of repositories at each work location, and
> "git fetch --mirror" into it from the bundle.  The "bundle create"
> command would most likely have some bottom revisions to reduce the
> size of the resulting bundle files, i.e. if you know that all copies
> of repositories you'd work with surely have the version you tagged
> "v53.8" a few weeks ago, then the "bundle create" command may have
> "--not v53.8" at the end.
>
> In other words, the above is *NOT* meant to be a full solution, but
> merely to illustrate a possible direction to go in.
>
>