Extremely long subtree split duration

Extremely long subtree split duration

[Peter Kassak]

Windows has always been a bit slower with subtree split, but now we
encountered an extreme difference.

The test is on the opensource repository of bitnami charts:

git clone https://github.com/bitnami/charts.git
cd charts
git subtree split -P bitnami/minio -b split

On live Ubuntu 24.04 booted from usb stick, it took about one and a
half minutes.

On Windows 11 installed on internal nvme, it took almost 4 hours.

Yes, it sounds crazy, but it really took three hours and 53 minutes on
the same hardware.

Main difference (except OS) is git version. On Ubuntu is latest
installed via apt 2.43.0, on Windows is 2.47.0.windows.2.

I have tried several different OS in docker, virtualbox, different HW,
different git versions..

Long story short, it looks like since version 2.44.0-rc0, probably
after merging 'zf/subtree-split-fix', subtree split is much much
slower.

And practically unusable on Windows.

Can this be fixed?

Thanks, Peter "Cache" Kassak

Re: Extremely long subtree split duration

[Kristoffer Haugsbakk]

On Fri, Nov 8, 2024, at 22:40, Peter Kassak wrote:
> Main difference (except OS) is git version. On Ubuntu is latest
> installed via apt 2.43.0, on Windows is 2.47.0.windows.2.

I tried a bit on Ubuntu 22.04, Git 2.47.0.  I gave it 14 minutes and then
it was apparently about 10000/32714 done.

-- 
Kristoffer Haugsbakk

Re: Extremely long subtree split duration

[Kristoffer Haugsbakk]

On Fri, Nov 8, 2024, at 23:07, Kristoffer Haugsbakk wrote:
> On Fri, Nov 8, 2024, at 22:40, Peter Kassak wrote:
>> Main difference (except OS) is git version. On Ubuntu is latest
>> installed via apt 2.43.0, on Windows is 2.47.0.windows.2.
>
> I tried a bit on Ubuntu 22.04, Git 2.47.0.  I gave it 14 minutes and then
> it was apparently about 10000/32714 done.
>
> --
> Kristoffer Haugsbakk

I tried again and ran the whole command.  It took 3 hours and 43
minutes.  My `/bin/sh` is dash(1).

I then checked out the first parent of the commit that you pointed to:
bcf524023e (Merge branch 'zf/subtree-split-fix', 2024-02-02).  In other
words I checked out bbc8c05670 (Merge branch 'jc/ls-files-doc-update',
2024-02-02).

A *split* on that took 3m49s.

-- 
Kristoffer Haugsbakk

Re: Extremely long subtree split duration

[Peter Kassak]

Hello,

I didn't compile my own version from sources, I wrote that it was
probably after 'zf/subtree-split-fix', because it was related to
subtree split.

And I managed to find out by trying that the change happened after
version 2.43.0 and was included in version 2.44.0-rc0, so i assumed
that.

Go on and try same split with 2.43.0.

Peter "Cache" Kassak.

On Sat, Nov 9, 2024 at 3:28 PM Kristoffer Haugsbakk
<kristofferhaugsbakk@fastmail.com> wrote:
>
> On Fri, Nov 8, 2024, at 23:07, Kristoffer Haugsbakk wrote:
> > On Fri, Nov 8, 2024, at 22:40, Peter Kassak wrote:
> >> Main difference (except OS) is git version. On Ubuntu is latest
> >> installed via apt 2.43.0, on Windows is 2.47.0.windows.2.
> >
> > I tried a bit on Ubuntu 22.04, Git 2.47.0.  I gave it 14 minutes and then
> > it was apparently about 10000/32714 done.
> >
> > --
> > Kristoffer Haugsbakk
>
> I tried again and ran the whole command.  It took 3 hours and 43
> minutes.  My `/bin/sh` is dash(1).
>
> I then checked out the first parent of the commit that you pointed to:
> bcf524023e (Merge branch 'zf/subtree-split-fix', 2024-02-02).  In other
> words I checked out bbc8c05670 (Merge branch 'jc/ls-files-doc-update',
> 2024-02-02).
>
> A *split* on that took 3m49s.
>
> --
> Kristoffer Haugsbakk

Re: Extremely long subtree split duration

[Kristoffer Haugsbakk]

Hi again

On Sat, Nov 9, 2024, at 22:10, Peter Kassak wrote:
> Hello,
>
> I didn't compile my own version from sources, I wrote that it was
> probably after 'zf/subtree-split-fix', because it was related to
> subtree split.
>
> And I managed to find out by trying that the change happened after
> version 2.43.0 and was included in version 2.44.0-rc0, so i assumed
> that.
>
> Go on and try same split with 2.43.0.
>
> Peter "Cache" Kassak.

Back when I read this I thought I checked these.  So I didn’t do
anything more.

I just came back to this since I found this project idea about
rewriting this contrib/ script in C (make it a builtin):

https://github.com/gitgitgadget/git/issues/1410

I’ll link this thread on that issue.

-- 
Kristoffer Haugsbakk

Re: Extremely long subtree split duration

[Kristoffer Haugsbakk]

I don’t use this command and I am an outsider to the code (to all of the
code in this project).  But the commit that introduced this apparent
performance regression does a lot of `git log -1` calls. Very apparent
when I ran it since I got a lot of stderr output about missing note refs
(since I use them in another project).

For what it’s worth.