Bug Report

Bug Report

[Tiago d'Almeida]

[-- Attachment #1: Type: text/plain, Size: 991 bytes --]

Hello!

I'm Tiago and sorry for any mistake because it is my first bug report to git.

So after git updated, I begin to get some errors and unexpected git
status output.

I made a test git repo to test this, and here are the results:

```
mkdir git_bug && cd git_bug
git init -q && git maintenance register && echo "Done!"
touch <files>
mkdir <folders>
touch <files in folders>
git status --short

## No commits yet on main
?? LICENSE
?? README.md
?? folder/
?? folder_1/

git add <files>
git status

## No commits yet on main
?? LICENSE
?? README.md
?? folder/
?? folder_1/

git commit -s -m "First commit"

On branch main

Initial commit

Untracked files:
LICENSE
README.md
folder/
folder_1/

nothing added to commit but untracked files present

```

Attached to this email follow the `git bugreport` and global `config`
files, and the git_bug repo.

I've checked my global config file and it seems all ok, but if there
is some error or contradicition, please tell me.

Thanks in advance!

[-- Attachment #2: git-bugreport-2023-06-27-1643.txt --]
[-- Type: text/plain, Size: 971 bytes --]

Thank you for filling out a Git bug report!
Please answer the following questions to help us understand your issue.

What did you do before the bug happened? (Steps to reproduce your issue)

A normal git routine.
- git add <files>
- git commit -s -m "Commit"

What did you expect to happen? (Expected behavior)

No errors and the files being added and commited.

What happened instead? (Actual behavior)

No files added. Error when tried to commit.

What's different between what you expected and what actually happened?

The error.

Anything else you want to add:

This just happened after updating git to version 2.41 


[System Info]
git version 2.41.0
cpu: x86_64
no commit associated with this build
sizeof-long: 8
sizeof-size_t: 8
shell-path: /bin/sh
uname: Linux 6.3.8-200.fc38.x86_64 #1 SMP PREEMPT_DYNAMIC Thu Jun 15 02:15:40 UTC 2023 x86_64
compiler info: gnuc: 13.1
libc info: glibc: 2.37
$SHELL (typically, interactive shell): /usr/bin/fish


[Enabled Hooks]

[-- Attachment #3: global_config --]
[-- Type: application/octet-stream, Size: 7090 bytes --]

# TIAGO d'ALMEIDA GIT CONFIG


# These directives includes the user.name and user.email according to the repo location.
[include]
  path = "./config_d/personal"

[includeIf "gitdir:~/Projects/work/"]
  path = "./config_d/work"

[includeIf "gitdir:~/Work/"]
  path = "./config_d/work"

[alias]
  a   = add
  aa  = add --all .
  ai  = add -i
  aac = ! git add --all . && git commit -m
  br  = branch
  cl  = clone
  cm  = commit -s -m
  cml = commit -s
  cp  = cherry-pick
  d   = diff
  f   = fetch
  fa  = fetch --all
  i   = ! git init -q && git maintenance register && echo "Done!"
  l   = log --graph --oneline -n 10
  ls  = log --graph --oneline --all
  lch = log --pretty="format:(%h) %N %an" --show-notes=*
  m   = maintenance run
  ms  = maintenance register
  mu  = maintenance unregister
  nt  = notes
  nta = notes append
  nch = notes append --ref=changelog
  p   = push
  pa  = push --all
  rb  = rebase -i
  r   = reset
  r1  = reset HEAD~
  r2  = reset HEAD~~
  rh  = reset --hard
  rh1 = reset HEAD~ --hard
  rh2 = reset HEAD~~ --hard
  rt  = restore
  s   = status --short
  sl  = status --long
  st  = stash              # equivalent to `stash push`
  stp = stash pop
  stl = stash list
  sta = stash apply
  sts = stash show
  sw  = switch             # switch and restore is the new checkout
  w   = ! watch -n 5 -p -c git -c color.ui=always status

  amend   = commit -s --amend --no-edit
  fixup   = commit -s --fixup

  since   = log --graph --oneline --since="1 week ago"

  track   = add -N
  unstage = restore --staged
  forget  = rm -r --cached

  del     = delete
  delete  = ! git maintenance unregister && rm -rf .git && echo "Done!"
  watch   = ! watch -n 5 -p -c git -c color.ui=always status

  show    = show --show-signature

#	brd = branch -d
#	brD = branch -D
# br_desc = branch --edit-description
#	merged = branch --merged
#	dev = !git checkout dev && git pull origin dev
#	staging = !git checkout staging && git pull origin staging
#	master = !git checkout master && git pull origin 
#	po = push origin
#	pu = !git push origin `git branch --show-current`
#	pod = push origin dev
#	pos = push origin staging
#	pom = push origin main
#	poh = push origin HEAD
#	pogm = !git push origin gh-pages && git checkout master && git pull origin master && git rebase gh-pages && git push origin master && git checkout gh-pages
#	pomg = !git push origin master && git checkout gh-pages && git pull origin gh-pages && git rebase master && git push origin gh-pages && git checkout master
#	plo = pull origin
#	plod = pull origin dev
#	plos = pull origin staging
#	plom = pull origin master
#	ploh = pull origin HEAD
#	f = "!git ls-files | grep -i"
# gr = grep -Ii

[advice]
  addIgnoredFile = false
  statusHints = false

[branch]
  autoSetupMerge = always

[core]
	askPass = ""
  compression = 6
  editor = nano +1
  eol = lf
  filemode = true
  fsmonitor = true
  fsmonitorHookVersion = 2
  fsyncMethod = batch
#  hooksPath = "./hooks" --> can't be this path, correct this in the future; see man git-config
#  notes = refs/notes/commits
  pager = less --tabs=2
#  quotePath = off
  safecrlf = true
  sparseCheckout = true
  sparseCheckoutCone = true
  splitIndex = true
  symlinks = true
  whitespace = space-before-tab, tab-in-indent, trailing-space, tabwidth=2

[add]
#  ignoreErrors = true

[apply]
  ignoreWhitespace = false

[checkout]
  defaultRemote = origin
  workers = 0
  thresholdForParallelism = 100
  
[color]
  ui = auto

[color "advice"]
  hint = dim white

[color "status"]
  header       = italic white
  added        = green
  updated      = cyan
  changed      = brightyellow
  untracked    = dim white
  branch       = bold magenta
  nobranch     = red
  localBranch  = bold magenta
  remoteBranch = brightgreen
  unmerged     = cyan

[column]
  ui = auto, column, dense

[commit]
  gpgSign = true
#  template = "./commit"

[credential]
#  helper = "cache --timeout 18000 --socket ~/.config/git/socket"
  helper = "cache --timeout 18000"

[diff]
  context = 5
  mnemonicPrefix = true
  relative = true
  renames = copies
  algoritm = patience
  wsErrorHighlight = all
  colorMoved = true

[feature]
  experimental = true
  manyFiles = true

[fetch]
  prune = true
  pruneTags = true
  output = compact
  parallel = 3
  writeCommitGraph = true

[filter "lfs"]
  clean = git-lfs clean -- %f
  smudge = git-lfs smudge -- %f
  process = git-lfs filter-process
  required = true

[gc]
  bigPackThreshold    = 1g
  logExpiry           = 1.week
  cruftPacks          = true
  pruneExpire         = 1.week
  worktreePruneExpire = 1.month

[gpg]
  program = gpg2
  minTrustLevel = fully

[help]
  autoCorrect = prompt

[http]
#  cookieFile = ~/.config/git/cookie
  saveCookies = true
  version = HTTP/2
  sslVersion = tlsv1.3
  sslVerify = true

[i18n]
  commitEncoding = utf-8
  logOutputEncoding = utf-8

[index]
  sparse = true
  threads = true

[init]
  defaultBranch = main
#  templateDir = ~/.config/git/template/

[interactive]
  singleKey = true

[lfs "https://github.com/"]
  locksverify = true

[lfs "https://gitlab.com/"]
  locksverify = true

[log]
  decorate = auto
  graphColors = brightwhite, red, green, blue, magenta, yellow, cyan

[lsrefs]
  unborn = advertise

[maintenance]
  auto = false
  strategy = incremental
# commit-graph        - hourly
# prefetch            - hourly
# gc                  - off
# loose-objects       - daily
# incremental-repack  - daily
# pack-refs           - weekly

[maintenance.gc]
  enabled = true
  schedule= weekly

[merge]
  conflictStyle = merge
  ff = only
  verifySignatures = true
  branchdesc = true
  log = true
  renormalize = true
  stat = true
  autoStash = true

#[notes]
#  # https://dev.to/leehambley/effortlessly-maintain-a-high-quality-change-log-with-git-notes-4bm5
##  mergeStrategy = cat_sort_uniq
##  rewriteMode = cat_sort_uniq
##  rewriteRef = refs/notes/commits
#
#[notes "rewrite"]
#  amend = true
#  rebase = true

[pack]
  useBitmaps = true
  useSparse = true
  writeBitmapHashCache =true
  writeReverseIndex = true

# pass command-line tool 
[pass]
  signcommits = true

[protocol]
  version = 2

[pull]
  ff      = only
  rebase  = false
  twohead = ort

[push]
  autoSetupRemote = true
  default = current
  followTags = true
  gpgSign = if-asked
  negotiate = true

[rebase]
  backend = merge
  autoSquash = true
  autoStash = true
  missingCommitsCheck = warn
  abbreviateCommands = true

[remote]
  pushDefault = origin

#[remote "origin"]
#  tagOpt = "--tags"
##  fetch = +refs/heads/*:refs/remotes/origin/*
##  fetch = +refs/notes/*:refs/notes/*
##  push = +refs/heads/*
##  push = +refs/notes/*:refs/notes/*
#
[repack]
  writeBitmaps = true

[rerere]
  autoUpdate = true
  enabled = true

[safe]
  bareRepository = explicit

[splitIndex]
  maxPercentChange = 20

[ssh]
  variant = ssh

[status]
  branch    = true
  showStash = true

[transfer]
  credentialsInUrl = die
  fsckObjects = true

[tag]
  forceSignAnnotated = true
  gpgSign = true

[user]
  useConfigOnly = true
  signingkey = ***********

[versionsort]
  suffix = "-pre"
  suffix = "-rc"
  suffix = ""

[-- Attachment #4: git_bug.tar.gz --]
[-- Type: application/gzip, Size: 11601 bytes --]

index.skipHash doesn't work with split index, was Re: Bug Report

[Jeff King]

On Tue, Jun 27, 2023 at 05:02:30PM +0100, Tiago d'Almeida wrote:

> Attached to this email follow the `git bugreport` and global `config`
> files, and the git_bug repo.

Thanks for providing your config; it was very important to reproducing.
The bug comes from the combination of "core.splitIndex" and
"index.skipHash" (the latter is triggered in your config by
"feature.manyFiles").

Here's a quick reproduction:

  git init repo
  cd repo
  touch file
  git -c core.splitIndex=true -c index.skipHash=true add file

That should add "file" to the index but doesn't. Removing either the
splitIndex option or the skipHash option makes it work. I didn't dig
further than that.

Adding the author of skipHash to the cc.

-Peff

Re: index.skipHash doesn't work with split index, was Re: Bug Report

[Jeff King]

[re-adding list to cc, so everybody can benefit from the answer]

On Mon, Jul 03, 2023 at 07:43:42PM +0100, Tiago d'Almeida wrote:

> Meanwhile, just a question:
> 
> How can I correct the corrupted git repos that are in my computer with
> unstaged changes?

Probably the best path forward is:

  1. Disable one or both of the problematic config options:

       git config --global core.splitIndex false

  2. Blow away the old index, and have git re-populate it from the HEAD
     commit:

       rm -f .git/index
       git reset

     Usually "git reset" would be enough here, but since we don't know
     what funny or possibly corrupted state the index file is in,
     deleting it first seems wise). Because the default for git-reset is
     "--mixed", this won't touch your working tree files at all.

  3. You should then be able to run "git status" to see any differences
     between the working tree and the fresh index (including any
     untracked ones). And now "git add", etc, should work.

-Peff

Re: index.skipHash doesn't work with split index, was Re: Bug Report

[Johannes Schindelin]

Hi,

On Thu, 29 Jun 2023, Jeff King wrote:

> On Tue, Jun 27, 2023 at 05:02:30PM +0100, Tiago d'Almeida wrote:
>
> > Attached to this email follow the `git bugreport` and global `config`
> > files, and the git_bug repo.
>
> Thanks for providing your config; it was very important to reproducing.
> The bug comes from the combination of "core.splitIndex" and
> "index.skipHash" (the latter is triggered in your config by
> "feature.manyFiles").
>
> Here's a quick reproduction:
>
>   git init repo
>   cd repo
>   touch file
>   git -c core.splitIndex=true -c index.skipHash=true add file

I ran into this issue while debugging the `commit -am` issue I worked on
in https://github.com/gitgitgadget/git/pull/1554.

The reason is that `write_shared_index()` calls `do_write_index()` without
any additional flags (see
https://github.com/git/git/blob/v2.41.0/read-cache.c#L3300) and
`do_write_index()` heeds the `index.skipHash` setting always (see
https://github.com/git/git/blob/v2.41.0/read-cache.c#L2900).

I briefly experimented with this diff, which is ugly and should not be
used as is, but it seemed to fix the issue for me:

-- snip --
diff --git a/read-cache.c b/read-cache.c
index ee6bcf40351..92a4aa2f25a 100644
--- a/read-cache.c
+++ b/read-cache.c
@@ -3292,14 +3294,17 @@ static int write_shared_index(struct index_state *istate,
 			      struct tempfile **temp, unsigned flags)
 {
 	struct split_index *si = istate->split_index;
-	int ret, was_full = !istate->sparse_index;
+	int ret, was_full = !istate->sparse_index, saved_skip_hash;

 	move_cache_to_base_index(istate);
 	convert_to_sparse(istate, 0);

 	trace2_region_enter_printf("index", "shared/do_write_index",
 				   the_repository, "%s", get_tempfile_path(*temp));
+	saved_skip_hash = si->base->repo->settings.index_skip_hash;
+	si->base->repo->settings.index_skip_hash = 0;
 	ret = do_write_index(si->base, *temp, WRITE_NO_EXTENSION, flags);
+	si->base->repo->settings.index_skip_hash = saved_skip_hash;
 	trace2_region_leave_printf("index", "shared/do_write_index",
 				   the_repository, "%s", get_tempfile_path(*temp));

-- snap --

The reason why this is needed is that the shared index _must_ have an
identifer that the split index can use, and that's that index hash.
Skipping it breaks that pattern.

Probably a much better idea than above-mentioned diff would be to add a
new flag as a sibling to `COMMIT_LOCK` (i.e. here:
https://github.com/git/git/blob/v2.41.0/cache.h#L346-L348) and use that
only in `write_shared_index()` to force the index hash to be computed and
written.

I won't have time to work on this, though.

Ciao,
Johannes

>
> That should add "file" to the index but doesn't. Removing either the
> splitIndex option or the skipHash option makes it work. I didn't dig
> further than that.
>
> Adding the author of skipHash to the cc.
>
> -Peff
>

Re: index.skipHash doesn't work with split index, was Re: Bug Report

[Junio C Hamano]

Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:

> The reason why this is needed is that the shared index _must_ have an
> identifer that the split index can use, and that's that index hash.
> Skipping it breaks that pattern.

A very good summary.