[BUG] Clones from local repositories do not work correctly under Windows

[BUG] Clones from local repositories do not work correctly under Windows

[L. E. Segovia]

Thank you for filling out a Git bug report!
Please answer the following questions to help us understand your issue.

What did you do before the bug happened? (Steps to reproduce your issue)

Under Windows 10 21H2, I cloned a Git repository off an existing local
copy, then applied a patch to it using `git am`.

What did you expect to happen? (Expected behavior)

Operations on the repo like `git status` and `git log` should work out
of the box.

What happened instead? (Actual behavior)

I keep getting this error:

> error: unable to normalize alternate object path:
/c/Users/Amalia/.cache/cerbero-sources/librsvg-2.40.20/.git/objects

With `git status` it happens only once, but with `git log` it's a wall
worth of that error before the log is actually generated and paged.

This error makes Git complain, but tools depending on it (in particular
Cargo) outright crash. For instance, this is the error I got when
attempting to build the Rust crate above:

> Error: CliError { error: Some(failed to determine package fingerprint
for build script for librsvg v2.58.0-beta.1
(E:\cerbero\build\sources\msvc_x86_64\librsvg-2.40.20\rsvg)
>
> Caused by:
>     0: failed to determine the most recently modified file in
E:\cerbero\build\sources\msvc_x86_64\librsvg-2.40.20\rsvg
>     1: failed to determine list of files in
E:\cerbero\build\sources\msvc_x86_64\librsvg-2.40.20\rsvg
>     2: object not found - no match for id
(32467a0191907fd571b502e395d033a06dfee655); class=Odb (9); code=NotFound
(-3)), exit_code: 101 }

What's different between what you expected and what actually happened?

It seems that under Windows, the code in
https://github.com/git/git/blame/436d4e5b14df49870a897f64fe92c0ddc7017e4c/object-file.c#L525
is not able to resolve the absolute path of the "remote" (the original
copy) at all. I checked if it was a filesystem traversal issue by
issuing the clone against a folder in the same drive, but there was no
change.

Anything else you want to add:

Possibly introduced in 2.39.0:
https://github.com/git/git/commit/199337d6ec5c656e52b914b5dac3820cc5e363f3

Please review the rest of the bug report below.
You can delete any lines you don't wish to share.


[System Info]
git version:
git version 2.44.0.windows.1
cpu: x86_64
built from commit: ad0bbfffa543db6979717be96df630d3e5741331
sizeof-long: 4
sizeof-size_t: 8
shell-path: /bin/sh
feature: fsmonitor--daemon
uname: Windows 10.0 19045
compiler info: gnuc: 13.2
libc info: no libc information available
$SHELL (typically, interactive shell): <unset>


[Enabled Hooks]

-- 
amyspark 🌸 https://www.amyspark.me

Re: [BUG] Clones from local repositories do not work correctly under Windows

[Johannes Sixt]

Am 11.04.24 um 19:03 schrieb L. E. Segovia:
> Thank you for filling out a Git bug report!
> Please answer the following questions to help us understand your issue.
> 
> What did you do before the bug happened? (Steps to reproduce your issue)
> 
> Under Windows 10 21H2, I cloned a Git repository off an existing local
> copy, then applied a patch to it using `git am`.

This is an imprecise description of what you did. I did my best to
follow these instructions, but could not reproduce the problem mentioned
below.

You should post the commands you have given with their output.

> 
> What did you expect to happen? (Expected behavior)
> 
> Operations on the repo like `git status` and `git log` should work out
> of the box.
> 
> What happened instead? (Actual behavior)
> 
> I keep getting this error:
> 
>> error: unable to normalize alternate object path:
> /c/Users/Amalia/.cache/cerbero-sources/librsvg-2.40.20/.git/objects

What is this .cache here?

-- Hannes

Re: [BUG] Clones from local repositories do not work correctly under Windows

[L. E. Segovia]

[-- Attachment #1: Type: text/plain, Size: 3291 bytes --]

Hi Johannes,

Under a MSYS2 shell, follow these steps (feel free to replace the choice
of remote repo with any other supporting a Rust crate):

1. `cd` to a directory of your choice (let's call it `dirA`).
2. `git clone https://gitlab.gnome.org/GNOME/librsvg.git .`
3. `git checkout -b cerbero_build`.
4. `cd ..`
5. `git clone <full path to dirA> -s -b cerbero_build dirB`
6. `cd dirB`
7. `git status`

Now switch to any other shell of your choice, such as Git Bash, CMD, or
PowerShell, and try issuing commands such as `git status`. It will
result in a wall of "error: unable to normalize..."

To further the error state, try `git am` the attached diff to the repo
under the MSYS2 shell. It will succeed, but afterwards you'll also start
to get fatal errors from `git status`:

> error: unable to normalize alternate object path: <dirA>
> error: Could not read 819d81aefc255dee9d8845363038f7fdec809673
> fatal: Failed to traverse parents of commit
90042c71ef79cebeba782c98b26f0532de0e6a25

Attempting to build the repo using the well-known pair `meson setup
build; meson compile -C build` (assuming you have all the dependencies
around) will, at this step, crash at the very start with the following
Rust stacktrace:

> Error: CliError { error: Some(failed to determine package fingerprint for build script for librsvg v2.58.0-beta.1 (<dirB>/rsvg)
> 
> Caused by:
>     0: failed to determine the most recently modified file in <dirB>\rsvg
>     1: failed to determine list of files in <dirB>\rsvg
>     2: object not found - no match for id (32467a0191907fd571b502e395d033a06dfee655); class=Odb (9); code=NotFound (-3)), exit_code: 101 }

It should be noted, however, than exchanging the shells (cloning in Git
Bash, but then checking and patching under MSYS) appears to not cause
any errors. That leads me to believe there's some part of Git that is
not handling line separators under MSYS specifically.

> What is this .cache here?

This is part of the Cerbero package management system, used by GStreamer
to craft its SDK:

https://gitlab.freedesktop.org/gstreamer/cerbero.git

The above mentioned steps were reverse engineered from its caching steps.

Hope this is of use.

Best,

amyspark

On 12/04/2024 03:56, Johannes Sixt wrote:
> Am 11.04.24 um 19:03 schrieb L. E. Segovia:
>> Thank you for filling out a Git bug report!
>> Please answer the following questions to help us understand your issue.
>>
>> What did you do before the bug happened? (Steps to reproduce your issue)
>>
>> Under Windows 10 21H2, I cloned a Git repository off an existing local
>> copy, then applied a patch to it using `git am`.
> 
> This is an imprecise description of what you did. I did my best to
> follow these instructions, but could not reproduce the problem mentioned
> below.
> 
> You should post the commands you have given with their output.
> 
>>
>> What did you expect to happen? (Expected behavior)
>>
>> Operations on the repo like `git status` and `git log` should work out
>> of the box.
>>
>> What happened instead? (Actual behavior)
>>
>> I keep getting this error:
>>
>>> error: unable to normalize alternate object path:
>> /c/Users/Amalia/.cache/cerbero-sources/librsvg-2.40.20/.git/objects
> 
> What is this .cache here?
> 
> -- Hannes
> 

-- 
amyspark 🌸 https://www.amyspark.me

[-- Attachment #2: 0001-test.patch --]
[-- Type: text/plain, Size: 465 bytes --]

From 6519af2fbab17cd4b1861e6b9cbfbb2c76906edf Mon Sep 17 00:00:00 2001
From: "L. E. Segovia" <amy@amyspark.me>
Date: Fri, 12 Apr 2024 09:43:51 -0300
Subject: [PATCH] test

---
 AUTHORS | 1 +
 1 file changed, 1 insertion(+)

diff --git a/AUTHORS b/AUTHORS
index 9971ce58..7b2a9266 100644
--- a/AUTHORS
+++ b/AUTHORS
@@ -68,6 +68,7 @@ Kurosawa Takeshi
 liucougar
 LRN
 Laszlo Peter
+L. E. Segovia
 Luis Menina
 Maarten Bosmans
 Maciej Piechotka
-- 
2.44.0.windows.1

Re: [BUG] Clones from local repositories do not work correctly under Windows

[Johannes Sixt]

Am 12.04.24 um 15:05 schrieb L. E. Segovia:
> Under a MSYS2 shell, follow these steps (feel free to replace the choice
> of remote repo with any other supporting a Rust crate):
> 
> 1. `cd` to a directory of your choice (let's call it `dirA`).
> 2. `git clone https://gitlab.gnome.org/GNOME/librsvg.git .`
> 3. `git checkout -b cerbero_build`.
> 4. `cd ..`
> 5. `git clone <full path to dirA> -s -b cerbero_build dirB`
> 6. `cd dirB`
> 7. `git status`
> 
> Now switch to any other shell of your choice, such as Git Bash, CMD, or
> PowerShell, and try issuing commands such as `git status`. It will
> result in a wall of "error: unable to normalize..."

Thanks, that is much more helpful. Here are my assumption, my
explanation of the symptoms, and the conclusion (which is basically "if
it hurts, don't do it"):

I assume that you have two installations of Git. One installation is
used when you use the git command from the MSYS2 shell; it is the MSYS2
version. The other installation is Git for Windows or equivalent that is
used by all the other shells.

When you clone a repository with -s, a .git/objects/info/alternates file
is initialized that contains the path to the original repository. You
can see its contents with `git count-objects -v`. I tested the
reproduction recipe with my own build (which is a MinGW build, much like
Git for Windows), and it looks like this:

C:\Temp\repoclone>git count-objects -v
count: 0
size: 0
in-pack: 0
packs: 0
size-pack: 0
prune-packable: 0
garbage: 0
size-garbage: 0
alternate: C:/Temp/repoorig/.git/objects

See the last line? It begins with a drive letter. When you call the
command with the MSYS2 version, it will look more like this:

alternate: /c/Temp/repoorig/.git/objects

This is the MSYS2/Cygwin absolute path with a "drive letter".
Unfortunately, this kind of path is unintelligible for Git for Windows.
It expects absolute paths to begin with drive letter-colon or a
double-slash or double-backslash. For this reason, it reports "unable to
normalize alternate object path".

The conclusion is: Do not use two different flavors of Git on a
repository that is set up with a link to an alternate repository.

BTW, I did not do the followup test with `git am`. (Actually, I did, but
there were no errors.) So, I cannot tell why your MSYS2 version suddenly
also reports the "unable to normalize" error.

-- Hannes

Re: [BUG] Clones from local repositories do not work correctly under Windows

[L. E. Segovia]

Hi Johannes,

Indeed there are two -- one lives under the MSYS2 prefix, the other
under Program Files. However, Cargo seems to access the Git data
directly, so I believe you're right -- it's not able to understand the
path syntax.

As for MSYS2, so far my testing points to an inability to chain back to
the original clone. Given the mixup of installs, it's best to focus on
fixing that.

Thanks for the explanation!

amyspark

On 12/04/2024 13:16, Johannes Sixt wrote:
> Am 12.04.24 um 15:05 schrieb L. E. Segovia:
>> Under a MSYS2 shell, follow these steps (feel free to replace the choice
>> of remote repo with any other supporting a Rust crate):
>>
>> 1. `cd` to a directory of your choice (let's call it `dirA`).
>> 2. `git clone https://gitlab.gnome.org/GNOME/librsvg.git .`
>> 3. `git checkout -b cerbero_build`.
>> 4. `cd ..`
>> 5. `git clone <full path to dirA> -s -b cerbero_build dirB`
>> 6. `cd dirB`
>> 7. `git status`
>>
>> Now switch to any other shell of your choice, such as Git Bash, CMD, or
>> PowerShell, and try issuing commands such as `git status`. It will
>> result in a wall of "error: unable to normalize..."
> 
> Thanks, that is much more helpful. Here are my assumption, my
> explanation of the symptoms, and the conclusion (which is basically "if
> it hurts, don't do it"):
> 
> I assume that you have two installations of Git. One installation is
> used when you use the git command from the MSYS2 shell; it is the MSYS2
> version. The other installation is Git for Windows or equivalent that is
> used by all the other shells.
> 
> When you clone a repository with -s, a .git/objects/info/alternates file
> is initialized that contains the path to the original repository. You
> can see its contents with `git count-objects -v`. I tested the
> reproduction recipe with my own build (which is a MinGW build, much like
> Git for Windows), and it looks like this:
> 
> C:\Temp\repoclone>git count-objects -v
> count: 0
> size: 0
> in-pack: 0
> packs: 0
> size-pack: 0
> prune-packable: 0
> garbage: 0
> size-garbage: 0
> alternate: C:/Temp/repoorig/.git/objects
> 
> See the last line? It begins with a drive letter. When you call the
> command with the MSYS2 version, it will look more like this:
> 
> alternate: /c/Temp/repoorig/.git/objects
> 
> This is the MSYS2/Cygwin absolute path with a "drive letter".
> Unfortunately, this kind of path is unintelligible for Git for Windows.
> It expects absolute paths to begin with drive letter-colon or a
> double-slash or double-backslash. For this reason, it reports "unable to
> normalize alternate object path".
> 
> The conclusion is: Do not use two different flavors of Git on a
> repository that is set up with a link to an alternate repository.
> 
> BTW, I did not do the followup test with `git am`. (Actually, I did, but
> there were no errors.) So, I cannot tell why your MSYS2 version suddenly
> also reports the "unable to normalize" error.
> 
> -- Hannes
> 

-- 
amyspark 🌸 https://www.amyspark.me

Re: [BUG] Clones from local repositories do not work correctly under Windows

[Junio C Hamano]

Johannes Sixt <j6t@kdbg.org> writes:

> alternate: C:/Temp/repoorig/.git/objects
> alternate: /c/Temp/repoorig/.git/objects
>
> This is the MSYS2/Cygwin absolute path with a "drive letter".
> Unfortunately, this kind of path is unintelligible for Git for Windows.
> It expects absolute paths to begin with drive letter-colon or a
> double-slash or double-backslash. For this reason, it reports "unable to
> normalize alternate object path".
>
> The conclusion is: Do not use two different flavors of Git on a
> repository that is set up with a link to an alternate repository.

I do not complain to the conclusion, and I do not use Windows, but
it makes me wonder if there is a way to spell that full path that
can be understood by both implementations.  The two implementations
are not incompatible in the actual object contents and refnames and
other things in .git/ directory, are they?

In short, does MSYS2/Cygwin understand paths in "C:/Temp/..." style,
and if so, writing that out, even though it may not what it
considers the native format, would make the world a happier place.

RE: [BUG] Clones from local repositories do not work correctly under Windows

[rsbecker]

On Friday, April 12, 2024 1:33 PM, Junio C Hamano wrote:
>Johannes Sixt <j6t@kdbg.org> writes:
>
>> alternate: C:/Temp/repoorig/.git/objects
>> alternate: /c/Temp/repoorig/.git/objects
>>
>> This is the MSYS2/Cygwin absolute path with a "drive letter".
>> Unfortunately, this kind of path is unintelligible for Git for Windows.
>> It expects absolute paths to begin with drive letter-colon or a
>> double-slash or double-backslash. For this reason, it reports "unable
>> to normalize alternate object path".
>>
>> The conclusion is: Do not use two different flavors of Git on a
>> repository that is set up with a link to an alternate repository.
>
>I do not complain to the conclusion, and I do not use Windows, but it makes
me
>wonder if there is a way to spell that full path that can be understood by
both
>implementations.  The two implementations are not incompatible in the
actual
>object contents and refnames and other things in .git/ directory, are they?
>
>In short, does MSYS2/Cygwin understand paths in "C:/Temp/..." style, and if
so,
>writing that out, even though it may not what it considers the native
format, would
>make the world a happier place.

I'm my environment (<2 month old Cygwin update), the path would be
/cygdrive/c/temp... I have not had issues using that style with git. Cygwin
itself does not understand C:\Temp (resolves to C:temp), but does understand
C:/Temp in bash and various utilities. It also understands (and expects) ~
to mean /home/user (C:\Cygwin64\home\user) in the Cygwin space, not relative
to C:\Users\user for resolving .gitconfig.

Re: [BUG] Clones from local repositories do not work correctly under Windows

[L. E. Segovia]

Hi all,

In Cerbero, we use cygpath or a bespoke conversion mechanism, depending
on availability:

https://gitlab.freedesktop.org/gstreamer/cerbero/-/blob/main/cerbero/utils/__init__.py#L142-149

I'm not sure if there's a function in MSYS2's libc exposing that.

amyspark

On 12/04/2024 14:41, rsbecker@nexbridge.com wrote:
> On Friday, April 12, 2024 1:33 PM, Junio C Hamano wrote:
>> Johannes Sixt <j6t@kdbg.org> writes:
>>
>>> alternate: C:/Temp/repoorig/.git/objects
>>> alternate: /c/Temp/repoorig/.git/objects
>>>
>>> This is the MSYS2/Cygwin absolute path with a "drive letter".
>>> Unfortunately, this kind of path is unintelligible for Git for Windows.
>>> It expects absolute paths to begin with drive letter-colon or a
>>> double-slash or double-backslash. For this reason, it reports "unable
>>> to normalize alternate object path".
>>>
>>> The conclusion is: Do not use two different flavors of Git on a
>>> repository that is set up with a link to an alternate repository.
>>
>> I do not complain to the conclusion, and I do not use Windows, but it makes
> me
>> wonder if there is a way to spell that full path that can be understood by
> both
>> implementations.  The two implementations are not incompatible in the
> actual
>> object contents and refnames and other things in .git/ directory, are they?
>>
>> In short, does MSYS2/Cygwin understand paths in "C:/Temp/..." style, and if
> so,
>> writing that out, even though it may not what it considers the native
> format, would
>> make the world a happier place.
> 
> I'm my environment (<2 month old Cygwin update), the path would be
> /cygdrive/c/temp... I have not had issues using that style with git. Cygwin
> itself does not understand C:\Temp (resolves to C:temp), but does understand
> C:/Temp in bash and various utilities. It also understands (and expects) ~
> to mean /home/user (C:\Cygwin64\home\user) in the Cygwin space, not relative
> to C:\Users\user for resolving .gitconfig.
> 

-- 
amyspark 🌸 https://www.amyspark.me

Re: [BUG] Clones from local repositories do not work correctly under Windows

[Johannes Sixt]

Am 12.04.24 um 19:32 schrieb Junio C Hamano:
> Johannes Sixt <j6t@kdbg.org> writes:
> 
>> alternate: C:/Temp/repoorig/.git/objects
>> alternate: /c/Temp/repoorig/.git/objects
>>
>> This is the MSYS2/Cygwin absolute path with a "drive letter".
>> Unfortunately, this kind of path is unintelligible for Git for Windows.
>> It expects absolute paths to begin with drive letter-colon or a
>> double-slash or double-backslash. For this reason, it reports "unable to
>> normalize alternate object path".
>>
>> The conclusion is: Do not use two different flavors of Git on a
>> repository that is set up with a link to an alternate repository.
> 
> I do not complain to the conclusion, and I do not use Windows, but
> it makes me wonder if there is a way to spell that full path that
> can be understood by both implementations.  The two implementations
> are not incompatible in the actual object contents and refnames and
> other things in .git/ directory, are they?
> 
> In short, does MSYS2/Cygwin understand paths in "C:/Temp/..." style,
> and if so, writing that out, even though it may not what it
> considers the native format, would make the world a happier place.

Yes, MSYS2 should understand C:/Temp. This can be achieved easily by
doing the clone with Git for Windows. A small complication is, though,
that the actual contents of the .git/objects/info/alternates is
constructed from the literal command line arguments. In my test I used

   git clone -s C:\Temp\repoorig repoclone

which writes

   C:\Temp\repoorig/.git/objects

to the file. The path we see in the output of count-objects -v is
obviously a normalized version. Users will have to say

   git clone -s C:/Temp/repoorig repoclone

to get the more widely understood version into the file.

-- Hannes