* Re: Git Rename Detection Bug
From: Junio C Hamano @ 2023-11-12 23:09 UTC (permalink / raw)
To: Elijah Newren
Cc: Philip Oakley, Jeremy Pridmore, git@vger.kernel.org,
Paul Baumgartner
In-Reply-To: <CABPp-BEtva2WTGQG3Qs4EbZLK_RJC9vuA-2OYxkTPExgowwvqQ@mail.gmail.com>
Elijah Newren <newren@gmail.com> writes:
>> Could I suggest that we are missing a piece of terminology, to wit,
>> BLOBSAME. It's a compatriot to TREESAME, as used in `git log` for
>> history simplification (based on a tree's pathspec, most commonly a
>> commit's top level path).
>
> We could add it, but I'm not sure how it helps. We already had 'exact
> rename' which seems to fit the bill as well, and 'blob' is something
> someone new to Git is unlikely to know.
Also, as Philip said, TREESAME is a concept foreign to rename
detection codepath. It is a property of a commit (not a tree) and
tells us if it has the same tree object as its relevant parents (in
which case it can be simplified away if it is a merge). I do not
mind rename codepath using a jargon (or two) to express "in trees A
and B, this subtree of A records the same tree object as a subtree
of B at a different path (i.e., the contents of these two subtrees
at different paths are the same)" but the word used to express that
should not be TREESAME to avoid confusion. And the other word to
express "this path in tree A records a blob object that is identical
to this other path in tree B" should not be BLOBSAME, as the word
strongly hints it is somehow related to TREESAME.
Thanks.
^ permalink raw reply
* Re: first-class conflicts?
From: Junio C Hamano @ 2023-11-12 23:25 UTC (permalink / raw)
To: Theodore Ts'o; +Cc: Sandra Snan, git, Dragan Simic, rsbecker
In-Reply-To: <20231112152143.GD35991@mit.edu>
"Theodore Ts'o" <tytso@mit.edu> writes:
> On Sat, Nov 11, 2023 at 10:31:54AM +0900, Junio C Hamano wrote:
>> ...
>> I wonder if it would help users to add a new configuration option
>> for those who want to live safer that tells "commit -a" to leave
>> unmerged paths alone and require the unmerged paths to be added
>> explicitly (which may have to extend to cover things like "add -u"
>> and "add .").
>>
>> Perhaps not. I often find myself doing "git add -u" after resolving
>> conflicts and re-reading the result, without an explicit pathspec.
>
> Maybe the configuration option would also forbit "git add -u" from
> adding diffs with conflict markers unless --force is added?
Historically we left it to pre-commit hooks, but I agree that
protection at the time of "git add" may be more helpful.
I also alluded to being careful about "git add" with an overly vague
pathspec (like "." to add everything addable under the sun), but I
do not think it is possible to define "overly vague" in a way that
satisfies everybody (would "git add \*.h" be still overly vague when
5% of your header files have conflicts in the merge you are
concluding?) and keep the new users safe.
Unless the configuration forbids patterns and say "each and every
individual path must be named to add and resolve conflicted paths",
that is. Come to think of it, that may not be too bad.
> I dunno. I personally wouldn't use it myself, because I've always
> made a point of running "git diff", or "git status", and almost
> always, a command like "make -j16 && make -j16 check" (or an aliased
> equivalent) before commiting a merge.
>
> But that's because I'm a paranoid s.o.b. and in my long career, I've
> learned is that "you can't be paranoid enough", and "hope is not a
> strategy". :-)
Being careful and paranoid is good ;-) I wouldn't use it myself,
either, but the discussion started while trying to allay new users'
worries about recording a half-resolved state by mistake, and in
that context, I think it would have non-empty audiences.
Thanks.
^ permalink raw reply
* Re: [RFC PATCH] status: avoid reporting worktrees as "Untracked files"
From: Junio C Hamano @ 2023-11-12 23:52 UTC (permalink / raw)
To: Eric Sunshine; +Cc: Edmundo Carmona Antoranz, git
In-Reply-To: <CAPig+cQbwcJOQiYyb7bma3pH1hxjE_X_yeAp3JeHWVCeJtySfQ@mail.gmail.com>
Eric Sunshine <sunshine@sunshineco.com> writes:
> I doubt you're the only one, but, based upon, list emails over the
> years, it seems that both in-main-tree and outside-main-tree (often
> sibling) worktrees are common. More recently, we've also heard from
> people who don't even have a main-worktree; instead, they hang their
> multiple worktrees off of a bare repository (which is an
> explicitly-supported use-case); i.e.:
>
> git clone --bare https://.../foobar.git
> git -C foobar.git worktree add worktree1
> git -C foobar.git worktree add worktree2
> ...
I am not sure why you brought in that layout in this discussion,
because it places worktree1 and worktree2 next to each other, just
like placing worktree1 and worktree2 next to the non-bare repository.
git clone https://.../foobar.git foobar
git -C foobar worktree add worktree1
git -C foobar worktree add worktree2
The layout to create worktrees attached to a bare repository and add
them next to each other, and the same starting from a non-bare
repository, share an important trait. They do not have an untracked
and untrackable "cruft" in their working tree, unlike the crazy
layout that places worktrees of the repository inside the working
tree of the primary worktree as untracked subdirectories.
Really, what is the advantage of doing so? It is not like the build
recipe recorded in the primary worktree can work recursively on
different branches that are checked out---worktree names and paths
at which they are checked out are totally local matter, and the
upstream project that supplies the build recipe would not know or
care.
Even worse, when the project wants to add a new subdirectory or a
file, the name chosen for the subdirectory may happen to collide
with the name of an untracked subdirectory you happened to have used
(again, because the worktree names and locations are totally local
matter, the upstream project are unaware of them and cannot avoid
such name clashes even if they cared). You can imagine the
confusion that happens to your next "git pull".
Compared to such an insanity, attaching worktrees to a bare
repository, so that all worktrees are equals and there is no
"primary" worktree that you cannot remove, behave just as normal as
a set of worktrees attached to a non-bare repository and sit outside
the primary worktree, often as immediate siblings.
^ permalink raw reply
* Re: [PATCH v4 3/4] rebase: test autosquash with and without -i
From: Junio C Hamano @ 2023-11-13 1:20 UTC (permalink / raw)
To: Andy Koppe; +Cc: git, newren
In-Reply-To: <20231111132720.78877-4-andy.koppe@gmail.com>
Andy Koppe <andy.koppe@gmail.com> writes:
> Amend t3415-rebase-autosquash.sh to test the --autosquash option and
> rebase.autoSquash config with and without -i.
>
> Signed-off-by: Andy Koppe <andy.koppe@gmail.com>
> ---
> t/t3415-rebase-autosquash.sh | 38 ++++++++++++++++++++++++++----------
> 1 file changed, 28 insertions(+), 10 deletions(-)
>
> diff --git a/t/t3415-rebase-autosquash.sh b/t/t3415-rebase-autosquash.sh
> index a364530d76..fcc40d6fe1 100755
> --- a/t/t3415-rebase-autosquash.sh
> +++ b/t/t3415-rebase-autosquash.sh
> @@ -43,7 +43,7 @@ test_auto_fixup () {
>
> git tag $1 &&
> test_tick &&
> - git rebase $2 -i HEAD^^^ &&
> + git rebase $2 HEAD^^^ &&
> git log --oneline >actual &&
> if test -n "$no_squash"
> then
> @@ -61,15 +61,24 @@ test_auto_fixup () {
> }
>
> test_expect_success 'auto fixup (option)' '
> - test_auto_fixup final-fixup-option --autosquash
> + test_auto_fixup fixup-option --autosquash &&
> + test_auto_fixup fixup-option-i "--autosquash -i"
> '
Nice. As test_auto_fixup (and test_auto_squash we see later) clears
the slate with "git reset --hard base" before it starts its work, it
is easy to rerun them with and without "-i".
^ permalink raw reply
* Re: [PATCH v4 4/4] rebase: rewrite --(no-)autosquash documentation
From: Junio C Hamano @ 2023-11-13 1:21 UTC (permalink / raw)
To: Andy Koppe; +Cc: git, newren
In-Reply-To: <20231111132720.78877-6-andy.koppe@gmail.com>
Andy Koppe <andy.koppe@gmail.com> writes:
> +In the rebase todo list, the actions of squash, fixup and amend commits are
> +changed from `pick` to `squash`, `fixup` or `fixup -C`, respectively, and they
> +are moved right after the commit they modify. The `--interactive` option can
> +be used to review and edit the todo list before proceeding.
OK, this is good---exactly what I was looking for while reviewing
the "lose -i from the description" step earlier in the series.
Nicely done.
^ permalink raw reply
* What's cooking in git.git (Nov 2023, #05; Mon, 13)
From: Junio C Hamano @ 2023-11-13 4:29 UTC (permalink / raw)
To: git
Here are the topics that have been cooking in my tree. Commits
prefixed with '+' are in 'next' (being in 'next' is a sign that a
topic is stable enough to be used and are candidate to be in a
future release). Commits prefixed with '-' are only in 'seen', and
aren't considered "accepted" at all and may be annotated with an URL
to a message that raises issues but they are no means exhaustive. A
topic without enough support may be discarded after a long period of
no activity (of course they can be resubmit when new interests
arise).
The Git 2.43-rc1 has been tagged and soon Git 2.43-rc2 will follow
this week.
Copies of the source code to Git live in many repositories, and the
following is a list of the ones I push into or their mirrors. Some
repositories have only a subset of branches.
With maint, master, next, seen, todo:
git://git.kernel.org/pub/scm/git/git.git/
git://repo.or.cz/alt-git.git/
https://kernel.googlesource.com/pub/scm/git/git/
https://github.com/git/git/
https://gitlab.com/git-vcs/git/
With all the integration branches and topics broken out:
https://github.com/gitster/git/
Even though the preformatted documentation in HTML and man format
are not sources, they are published in these repositories for
convenience (replace "htmldocs" with "manpages" for the manual
pages):
git://git.kernel.org/pub/scm/git/git-htmldocs.git/
https://github.com/gitster/git-htmldocs.git/
Release tarballs are available at:
https://www.kernel.org/pub/software/scm/git/
--------------------------------------------------
[New Topics]
* jk/chunk-bounds-more (2023-11-09) 9 commits
(merged to 'next' on 2023-11-13 at 3df4b18bea)
+ commit-graph: mark chunk error messages for translation
+ commit-graph: drop verify_commit_graph_lite()
+ commit-graph: check order while reading fanout chunk
+ commit-graph: use fanout value for graph size
+ commit-graph: abort as soon as we see a bogus chunk
+ commit-graph: clarify missing-chunk error messages
+ commit-graph: drop redundant call to "lite" verification
+ midx: check consistency of fanout table
+ commit-graph: handle overflow in chunk_size checks
(this branch is used by tb/pair-chunk-expect.)
Code clean-up for jk/chunk-bounds topic.
Will cook in 'next'.
source: <20231109070310.GA2697602@coredump.intra.peff.net>
* ps/httpd-tests-on-nixos (2023-11-11) 3 commits
(merged to 'next' on 2023-11-13 at 81bd6f5334)
+ t9164: fix inability to find basename(1) in Subversion hooks
+ t/lib-httpd: stop using legacy crypt(3) for authentication
+ t/lib-httpd: dynamically detect httpd and modules path
Portability tweak.
Will merge to 'master'.
source: <cover.1699596457.git.ps@pks.im>
* ss/format-patch-use-encode-headers-for-cover-letter (2023-11-10) 1 commit
- format-patch: fix ignored encode_email_headers for cover letter
"git format-patch --encode-email-headers" ignored the option when
preparing the cover letter, which has been corrected.
Will merge to 'next'.
source: <20231109111950.387219-1-contact@emersion.fr>
* ps/ban-a-or-o-operator-with-test (2023-11-11) 4 commits
- Makefile: stop using `test -o` when unlinking duplicate executables
- contrib/subtree: convert subtree type check to use case statement
- contrib/subtree: stop using `-o` to test for number of args
- global: convert trivial usages of `test <expr> -a/-o <expr>`
Test and shell scripts clean-up.
Will merge to 'next'.
source: <cover.1699609940.git.ps@pks.im>
* vd/glossary-dereference-peel (2023-11-10) 1 commit
- glossary: add definitions for dereference & peel
"To dereference" and "to peel" were sometimes used in in-code
comments and documentation but without description in the glossary.
Expecting an update.
cf. <xmqq1qcyxxri.fsf@gitster.g>
source: <pull.1610.git.1699574277143.gitgitgadget@gmail.com>
* ak/rebase-autosquash (2023-11-13) 4 commits
- rebase: rewrite --(no-)autosquash documentation
- rebase: test autosquash with and without -i
- rebase: support --autosquash without -i
- rebase: fully ignore rebase.autoSquash without -i
source: <20231111132720.78877-1-andy.koppe@gmail.com>
--------------------------------------------------
[Stalled]
* pw/rebase-sigint (2023-09-07) 1 commit
- rebase -i: ignore signals when forking subprocesses
If the commit log editor or other external programs (spawned via
"exec" insn in the todo list) receive internactive signal during
"git rebase -i", it caused not just the spawned program but the
"Git" process that spawned them, which is often not what the end
user intended. "git" learned to ignore SIGINT and SIGQUIT while
waiting for these subprocesses.
Expecting a reroll.
cf. <12c956ea-330d-4441-937f-7885ab519e26@gmail.com>
source: <pull.1581.git.1694080982621.gitgitgadget@gmail.com>
* tk/cherry-pick-sequence-requires-clean-worktree (2023-06-01) 1 commit
- cherry-pick: refuse cherry-pick sequence if index is dirty
"git cherry-pick A" that replays a single commit stopped before
clobbering local modification, but "git cherry-pick A..B" did not,
which has been corrected.
Expecting a reroll.
cf. <999f12b2-38d6-f446-e763-4985116ad37d@gmail.com>
source: <pull.1535.v2.git.1685264889088.gitgitgadget@gmail.com>
* jc/diff-cached-fsmonitor-fix (2023-09-15) 3 commits
- diff-lib: fix check_removed() when fsmonitor is active
- Merge branch 'jc/fake-lstat' into jc/diff-cached-fsmonitor-fix
- Merge branch 'js/diff-cached-fsmonitor-fix' into jc/diff-cached-fsmonitor-fix
(this branch uses jc/fake-lstat.)
The optimization based on fsmonitor in the "diff --cached"
codepath is resurrected with the "fake-lstat" introduced earlier.
It is unknown if the optimization is worth resurrecting, but in case...
source: <xmqqr0n0h0tw.fsf@gitster.g>
--------------------------------------------------
[Cooking]
* vd/for-each-ref-unsorted-optimization (2023-11-07) 9 commits
- t/perf: add perf tests for for-each-ref
- for-each-ref: add option to fully dereference tags
- ref-filter.c: filter & format refs in the same callback
- ref-filter.c: refactor to create common helper functions
- ref-filter.h: add functions for filter/format & format-only
- ref-filter.h: move contains caches into filter
- ref-filter.h: add max_count and omit_empty to ref_format
- for-each-ref: clarify interaction of --omit-empty & --count
- ref-filter.c: really don't sort when using --no-sort
"git for-each-ref --no-sort" still sorted the refs alphabetically
which paid non-trivial cost. It has been redefined to show output
in an unspecified order, to allow certain optimizations to take
advantage of.
Expecting a reroll.
cf. <dbcbcf0e-aeee-4bb9-9e39-e2e85194d083@github.com>
source: <pull.1609.git.1699320361.gitgitgadget@gmail.com>
* jw/git-add-attr-pathspec (2023-11-04) 1 commit
(merged to 'next' on 2023-11-13 at b61be94e4d)
+ attr: enable attr pathspec magic for git-add and git-stash
"git add" and "git stash" learned to support the ":(attr:...)"
magic pathspec.
Will cook in 'next'.
source: <20231103163449.1578841-1-jojwang@google.com>
* ps/ci-gitlab (2023-11-09) 8 commits
(merged to 'next' on 2023-11-10 at ea7ed67945)
+ ci: add support for GitLab CI
+ ci: install test dependencies for linux-musl
+ ci: squelch warnings when testing with unusable Git repo
+ ci: unify setup of some environment variables
+ ci: split out logic to set up failed test artifacts
+ ci: group installation of Docker dependencies
+ ci: make grouping setup more generic
+ ci: reorder definitions for grouping functions
Add support for GitLab CI.
Will cook in 'next'.
source: <cover.1699514143.git.ps@pks.im>
* ps/ref-tests-update (2023-11-03) 10 commits
(merged to 'next' on 2023-11-13 at dc26e55d6f)
+ t: mark several tests that assume the files backend with REFFILES
+ t7900: assert the absence of refs via git-for-each-ref(1)
+ t7300: assert exact states of repo
+ t4207: delete replace references via git-update-ref(1)
+ t1450: convert tests to remove worktrees via git-worktree(1)
+ t: convert tests to not access reflog via the filesystem
+ t: convert tests to not access symrefs via the filesystem
+ t: convert tests to not write references via the filesystem
+ t: allow skipping expected object ID in `ref-store update-ref`
+ Merge branch 'ps/show-ref' into ps/ref-tests-update
Update ref-related tests.
Will cook in 'next'.
source: <cover.1698914571.git.ps@pks.im>
* jx/fetch-atomic-error-message-fix (2023-10-19) 2 commits
- fetch: no redundant error message for atomic fetch
- t5574: test porcelain output of atomic fetch
"git fetch --atomic" issued an unnecessary empty error message,
which has been corrected.
Expecting an update.
cf. <ZTjQIrCgSANAT8wR@tanuki>
source: <ced46baeb1c18b416b4b4cc947f498bea2910b1b.1697725898.git.zhiyou.jx@alibaba-inc.com>
* js/bugreport-in-the-same-minute (2023-10-16) 1 commit
- bugreport: include +i in outfile suffix as needed
Instead of auto-generating a filename that is already in use for
output and fail the command, `git bugreport` learned to fuzz the
filename to avoid collisions with existing files.
Expecting a reroll.
cf. <ZTtZ5CbIGETy1ucV.jacob@initialcommit.io>
source: <20231016214045.146862-2-jacob@initialcommit.io>
* kh/t7900-cleanup (2023-10-17) 9 commits
- t7900: fix register dependency
- t7900: factor out packfile dependency
- t7900: fix `print-args` dependency
- t7900: fix `pfx` dependency
- t7900: factor out common schedule setup
- t7900: factor out inheritance test dependency
- t7900: create commit so that branch is born
- t7900: setup and tear down clones
- t7900: remove register dependency
Test clean-up.
Perhaps discard?
cf. <655ca147-c214-41be-919d-023c1b27b311@app.fastmail.com>
source: <cover.1697319294.git.code@khaugsbakk.name>
* tb/merge-tree-write-pack (2023-10-23) 5 commits
- builtin/merge-tree.c: implement support for `--write-pack`
- bulk-checkin: introduce `index_tree_bulk_checkin_incore()`
- bulk-checkin: introduce `index_blob_bulk_checkin_incore()`
- bulk-checkin: generify `stream_blob_to_pack()` for arbitrary types
- bulk-checkin: extract abstract `bulk_checkin_source`
"git merge-tree" learned "--write-pack" to record its result
without creating loose objects.
Broken when an object created during a merge is needed to continue merge
cf. <CABPp-BEfy9VOvimP9==ry_rZXu=metOQ8s=_-XiG_Pdx9c06Ww@mail.gmail.com>
source: <cover.1698101088.git.me@ttaylorr.com>
* tb/pair-chunk-expect (2023-11-10) 8 commits
- midx: read `OOFF` chunk with `pair_chunk_expect()`
- midx: read `OIDL` chunk with `pair_chunk_expect()`
- commit-graph: read `BIDX` chunk with `pair_chunk_expect()`
- commit-graph: read `GDAT` chunk with `pair_chunk_expect()`
- commit-graph: read `CDAT` chunk with `pair_chunk_expect()`
- commit-graph: read `OIDL` chunk with `pair_chunk_expect()`
- chunk-format: introduce `pair_chunk_expect()` helper
- Merge branch 'jk/chunk-bounds-more' into HEAD
(this branch uses jk/chunk-bounds-more.)
Further code clean-up.
Needs review.
source: <cover.1699569246.git.me@ttaylorr.com>
* tb/path-filter-fix (2023-10-18) 17 commits
- bloom: introduce `deinit_bloom_filters()`
- commit-graph: reuse existing Bloom filters where possible
- object.h: fix mis-aligned flag bits table
- commit-graph: drop unnecessary `graph_read_bloom_data_context`
- commit-graph.c: unconditionally load Bloom filters
- bloom: prepare to discard incompatible Bloom filters
- bloom: annotate filters with hash version
- commit-graph: new filter ver. that fixes murmur3
- repo-settings: introduce commitgraph.changedPathsVersion
- t4216: test changed path filters with high bit paths
- t/helper/test-read-graph: implement `bloom-filters` mode
- bloom.h: make `load_bloom_filter_from_graph()` public
- t/helper/test-read-graph.c: extract `dump_graph_info()`
- gitformat-commit-graph: describe version 2 of BDAT
- commit-graph: ensure Bloom filters are read with consistent settings
- revision.c: consult Bloom filters for root commits
- t/t4216-log-bloom.sh: harden `test_bloom_filters_not_used()`
The Bloom filter used for path limited history traversal was broken
on systems whose "char" is unsigned; update the implementation and
bump the format version to 2.
Needs (hopefully final and quick) review.
source: <cover.1697653929.git.me@ttaylorr.com>
* cc/git-replay (2023-11-03) 14 commits
- replay: stop assuming replayed branches do not diverge
- replay: add --contained to rebase contained branches
- replay: add --advance or 'cherry-pick' mode
- replay: use standard revision ranges
- replay: make it a minimal server side command
- replay: remove HEAD related sanity check
- replay: remove progress and info output
- replay: add an important FIXME comment about gpg signing
- replay: change rev walking options
- replay: introduce pick_regular_commit()
- replay: die() instead of failing assert()
- replay: start using parse_options API
- replay: introduce new builtin
- t6429: remove switching aspects of fast-rebase
Introduce "git replay", a tool meant on the server side without
working tree to recreate a history.
Comments?
source: <20231102135151.843758-1-christian.couder@gmail.com>
* ak/color-decorate-symbols (2023-10-23) 7 commits
- log: add color.decorate.pseudoref config variable
- refs: exempt pseudorefs from pattern prefixing
- refs: add pseudorefs array and iteration functions
- log: add color.decorate.ref config variable
- log: add color.decorate.symbol config variable
- log: use designated inits for decoration_colors
- config: restructure color.decorate documentation
A new config for coloring.
Needs review.
source: <20231023221143.72489-1-andy.koppe@gmail.com>
* js/update-urls-in-doc-and-comment (2023-09-26) 4 commits
- doc: refer to internet archive
- doc: update links for andre-simon.de
- doc: update links to current pages
- doc: switch links to https
Stale URLs have been updated to their current counterparts (or
archive.org) and HTTP links are replaced with working HTTPS links.
Needs review.
source: <pull.1589.v2.git.1695553041.gitgitgadget@gmail.com>
* la/trailer-cleanups (2023-10-20) 3 commits
- trailer: use offsets for trailer_start/trailer_end
- trailer: find the end of the log message
- commit: ignore_non_trailer computes number of bytes to ignore
Code clean-up.
Comments?
source: <pull.1563.v5.git.1697828495.gitgitgadget@gmail.com>
* eb/hash-transition (2023-10-02) 30 commits
- t1016-compatObjectFormat: add tests to verify the conversion between objects
- t1006: test oid compatibility with cat-file
- t1006: rename sha1 to oid
- test-lib: compute the compatibility hash so tests may use it
- builtin/ls-tree: let the oid determine the output algorithm
- object-file: handle compat objects in check_object_signature
- tree-walk: init_tree_desc take an oid to get the hash algorithm
- builtin/cat-file: let the oid determine the output algorithm
- rev-parse: add an --output-object-format parameter
- repository: implement extensions.compatObjectFormat
- object-file: update object_info_extended to reencode objects
- object-file-convert: convert commits that embed signed tags
- object-file-convert: convert commit objects when writing
- object-file-convert: don't leak when converting tag objects
- object-file-convert: convert tag objects when writing
- object-file-convert: add a function to convert trees between algorithms
- object: factor out parse_mode out of fast-import and tree-walk into in object.h
- cache: add a function to read an OID of a specific algorithm
- tag: sign both hashes
- commit: export add_header_signature to support handling signatures on tags
- commit: convert mergetag before computing the signature of a commit
- commit: write commits for both hashes
- object-file: add a compat_oid_in parameter to write_object_file_flags
- object-file: update the loose object map when writing loose objects
- loose: compatibilty short name support
- loose: add a mapping between SHA-1 and SHA-256 for loose objects
- repository: add a compatibility hash algorithm
- object-names: support input of oids in any supported hash
- oid-array: teach oid-array to handle multiple kinds of oids
- object-file-convert: stubs for converting from one object format to another
Teach a repository to work with both SHA-1 and SHA-256 hash algorithms.
Needs review.
source: <878r8l929e.fsf@gmail.froward.int.ebiederm.org>
* jx/remote-archive-over-smart-http (2023-10-04) 4 commits
- archive: support remote archive from stateless transport
- transport-helper: call do_take_over() in connect_helper
- transport-helper: call do_take_over() in process_connect
- transport-helper: no connection restriction in connect_helper
"git archive --remote=<remote>" learned to talk over the smart
http (aka stateless) transport.
Needs review.
source: <cover.1696432593.git.zhiyou.jx@alibaba-inc.com>
* jx/sideband-chomp-newline-fix (2023-10-04) 3 commits
- pkt-line: do not chomp newlines for sideband messages
- pkt-line: memorize sideband fragment in reader
- test-pkt-line: add option parser for unpack-sideband
Sideband demultiplexer fixes.
Needs review.
source: <cover.1696425168.git.zhiyou.jx@alibaba-inc.com>
* js/config-parse (2023-09-21) 5 commits
- config-parse: split library out of config.[c|h]
- config.c: accept config_parse_options in git_config_from_stdin
- config: report config parse errors using cb
- config: split do_event() into start and flush operations
- config: split out config_parse_options
The parsing routines for the configuration files have been split
into a separate file.
Needs review.
source: <cover.1695330852.git.steadmon@google.com>
* jc/fake-lstat (2023-09-15) 1 commit
- cache: add fake_lstat()
(this branch is used by jc/diff-cached-fsmonitor-fix.)
A new helper to let us pretend that we called lstat() when we know
our cache_entry is up-to-date via fsmonitor.
Needs review.
source: <xmqqcyykig1l.fsf@gitster.g>
* js/doc-unit-tests (2023-11-10) 3 commits
(merged to 'next' on 2023-11-10 at 7d00ffd06b)
+ ci: run unit tests in CI
+ unit tests: add TAP unit test framework
+ unit tests: add a project plan document
(this branch is used by js/doc-unit-tests-with-cmake.)
Process to add some form of low-level unit tests has started.
Will cook in 'next'.
source: <cover.1699555664.git.steadmon@google.com>
* js/doc-unit-tests-with-cmake (2023-11-10) 7 commits
(merged to 'next' on 2023-11-10 at b4503c9c8c)
+ cmake: handle also unit tests
+ cmake: use test names instead of full paths
+ cmake: fix typo in variable name
+ artifacts-tar: when including `.dll` files, don't forget the unit-tests
+ unit-tests: do show relative file paths
+ unit-tests: do not mistake `.pdb` files for being executable
+ cmake: also build unit tests
(this branch uses js/doc-unit-tests.)
Update the base topic to work with CMake builds.
Will cook in 'next'.
source: <pull.1579.v3.git.1695640836.gitgitgadget@gmail.com>
* jc/rerere-cleanup (2023-08-25) 4 commits
- rerere: modernize use of empty strbuf
- rerere: try_merge() should use LL_MERGE_ERROR when it means an error
- rerere: fix comment on handle_file() helper
- rerere: simplify check_one_conflict() helper function
Code clean-up.
Not ready to be reviewed yet.
source: <20230824205456.1231371-1-gitster@pobox.com>
* rj/status-bisect-while-rebase (2023-10-16) 1 commit
- status: fix branch shown when not only bisecting
"git status" is taught to show both the branch being bisected and
being rebased when both are in effect at the same time.
Needs review.
source: <2e24ca9b-9c5f-f4df-b9f8-6574a714dfb2@gmail.com>
--------------------------------------------------
[Discarded]
* jc/strbuf-comment-line-char (2023-11-01) 4 commits
. strbuf: move env-using functions to environment.c
. strbuf: make add_lines() public
. strbuf_add_commented_lines(): drop the comment_line_char parameter
. strbuf_commented_addf(): drop the comment_line_char parameter
Code simplification that goes directly against a past libification
topic. It is hard to judge because the "libification" is done
piecewise without seemingly clear design principle.
Will discard.
source: <cover.1698791220.git.jonathantanmy@google.com>
^ permalink raw reply
* Re: [PATCH v2 1/4] global: convert trivial usages of `test <expr> -a/-o <expr>`
From: Patrick Steinhardt @ 2023-11-13 7:12 UTC (permalink / raw)
To: Junio C Hamano; +Cc: Jeff King, git
In-Reply-To: <xmqqr0kxrvij.fsf@gitster.g>
[-- Attachment #1: Type: text/plain, Size: 1522 bytes --]
On Sat, Nov 11, 2023 at 09:20:04AM +0900, Junio C Hamano wrote:
> Junio C Hamano <gitster@pobox.com> writes:
>
> > I am not so surprised that this one was missed, though. I didn't
> > point this one out during my review of the previous round, either,
> > and not everybody is as careful as you are.
>
> Ah, sorry, thist came out in a way I did not mean to.
>
> I didn't mean "I did not point it out explicitly. It is not
> surprising if a contributor who was not careful did not find it on
> their own and took initiative to fix it themselves".
>
> I meant "I failed to spot it myself hence I didn't point it out in
> my review---I was not being so careful to aim for thoroughly cover
> and find all the similar issues".
>
> In any case, I'll tweak it while queueing. Thanks for noticing.
Thanks indeed, I missed this instance as well when I scanned for any
additional subshells.
Patrick
> diff --git i/t/valgrind/valgrind.sh w/t/valgrind/valgrind.sh
> index 9fbf90cee7..3c8ee19975 100755
> --- i/t/valgrind/valgrind.sh
> +++ w/t/valgrind/valgrind.sh
> @@ -23,7 +23,7 @@ memcheck)
> VALGRIND_MAJOR=$(expr "$VALGRIND_VERSION" : '[^0-9]*\([0-9]*\)')
> VALGRIND_MINOR=$(expr "$VALGRIND_VERSION" : '[^0-9]*[0-9]*\.\([0-9]*\)')
> test 3 -gt "$VALGRIND_MAJOR" ||
> - ( test 3 -eq "$VALGRIND_MAJOR" && test 4 -gt "$VALGRIND_MINOR" ) ||
> + { test 3 -eq "$VALGRIND_MAJOR" && test 4 -gt "$VALGRIND_MINOR"; } ||
> TOOL_OPTIONS="$TOOL_OPTIONS --track-origins=yes"
> ;;
> *)
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]
^ permalink raw reply
* Re: [PATCH v4 0/3] t: improve compatibility with NixOS
From: Patrick Steinhardt @ 2023-11-13 7:15 UTC (permalink / raw)
To: Junio C Hamano; +Cc: git, Jeff King
In-Reply-To: <xmqqh6lttaj4.fsf@gitster.g>
[-- Attachment #1: Type: text/plain, Size: 1741 bytes --]
On Sat, Nov 11, 2023 at 09:10:23AM +0900, Junio C Hamano wrote:
> Patrick Steinhardt <ps@pks.im> writes:
>
> > Three changes compared to v3:
> >
> > - Switched from `test <expr> -a <expr>` to `test <expr> && test
> > <expr>`.
> >
> > - Improved the commit message to explain why the new
> > runtime-detected paths are only used as a fallback.
> >
> > - Rebased on top of 0e3b67e2aa (ci: add support for GitLab CI,
> > 2023-11-09), which has been merged to `next` and caused conflicts.
>
> Please, no. "The other topic has been merged to 'next'" is *not* a
> good reason to do this. Before that, the other topic was in 'seen'
> and causing conflicts already, so getting into 'next' did not create
> any new reason for rebasing.
>
> I'll manage this time, but please do not do such a rebase unless you
> are asked to do so. The rebase will force me to do (1) detach from
> 'next' and apply these, (2) discard the result and detach from the
> base of where the previous iteration is queued, (3) apply the new
> iteration with "am -3" to undo the rebase, before I can compare the
> new iteration with the old iteration.
Fair enough. I assumed that it would ease your workload instead of
creating more work for you. But I'll keep in mind that it doesn't and
refrain from doing this in the future.
> > Technically speaking this series also depends on 0763c3a2c4 (http:
> > update curl http/2 info matching for curl 8.3.0, 2023-09-15), without
> > which the tests will fail on NixOS machines with a recent libcurl.
>
> Thanks for that note. This topic has been queued on top of
> v2.43.0-rc1 which has 0763c3a2c4, so we'd be safe.
>
> Will queue.
Thanks.
Patrick
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]
^ permalink raw reply
* Re: [PATCH v4 1/3] t/lib-httpd: dynamically detect httpd and modules path
From: Patrick Steinhardt @ 2023-11-13 7:15 UTC (permalink / raw)
To: Junio C Hamano; +Cc: git, Jeff King
In-Reply-To: <xmqqo7g1tb06.fsf@gitster.g>
[-- Attachment #1: Type: text/plain, Size: 1168 bytes --]
On Sat, Nov 11, 2023 at 09:00:09AM +0900, Junio C Hamano wrote:
> Patrick Steinhardt <ps@pks.im> writes:
>
> > +if test -x "$DEFAULT_HTTPD_PATH"
> > +then
> > + DETECTED_HTTPD_ROOT="$("$DEFAULT_HTTPD_PATH" -V | sed -n 's/^ -D HTTPD_ROOT="\(.*\)"$/\1/p')"
> > +fi
>
> With this patch, my test run starts like so:
>
> rm -f -r 'test-results'
> *** prove ***
> apache2: Could not open configuration file /etc/apache2/apache2.conf: No such file or directory
> ...
>
> I find the error message leaking mildly annoying, and would suggest
> doing something like the following on top.
>
> diff --git c/t/lib-httpd.sh w/t/lib-httpd.sh
> index 0a74922d7f..03493ee72b 100644
> --- c/t/lib-httpd.sh
> +++ w/t/lib-httpd.sh
> @@ -68,7 +68,7 @@ done
>
> if test -x "$DEFAULT_HTTPD_PATH"
> then
> - DETECTED_HTTPD_ROOT="$("$DEFAULT_HTTPD_PATH" -V | sed -n 's/^ -D HTTPD_ROOT="\(.*\)"$/\1/p')"
> + DETECTED_HTTPD_ROOT="$("$DEFAULT_HTTPD_PATH" -V 2>/dev/null | sed -n 's/^ -D HTTPD_ROOT="\(.*\)"$/\1/p')"
> fi
>
> for DEFAULT_HTTPD_MODULE_PATH in '/usr/libexec/apache2' \
Yup, makes sense. Thanks for the tweak.
Patrick
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]
^ permalink raw reply
* [PATCH 00/13] Enabling z/OS workflow for git
From: Haritha via GitGitGadget @ 2023-11-13 10:24 UTC (permalink / raw)
To: git; +Cc: Haritha
z/OS is an IBM mainframe operating system, also known as OS/390. Our team
has been actively involved in porting Git to z/OS and we have made
significant modifications to facilitate this process. The patch below is the
initial configuration for z/OS. I also have few follow up changes and I will
send that after these changes are approved. Please let me know if there are
any concerns.
Haritha D (13):
Enabling z/OS workflow for git
Enable builds for z/OS.
spaces and errors fix Handled git pipeline errors
fixes for build errors Handled git pipeline errorse
fixes for build errors
spaces and errors fix Handled git pipeline errors
spaces and errors fix Handled git pipeline errors
platform_name fix Handled git pipeline errors
strncpy fix Handled git pipeline errors
strncpy fix Handled git pipeline errors
strncpy fix Handled git pipeline errors
Handled git pipeline errors - Memory leak
Handled git pipeline errors - z/OS enable
Makefile | 21 +++++++++---
builtin.h | 3 ++
builtin/archive.c | 6 ++++
builtin/hash-object.c | 28 +++++++++++++++
combine-diff.c | 4 +++
config.c | 7 ++++
config.mak.uname | 20 +++++++++++
configure.ac | 3 ++
convert.c | 58 +++++++++++++++++++++++++++----
copy.c | 3 ++
diff.c | 11 ++++++
entry.c | 26 ++++++++++++++
environment.c | 3 ++
fetch-negotiator.h | 2 +-
fetch-pack.c | 4 +--
git-compat-util.h | 8 +++++
negotiator/default.c | 4 +--
negotiator/noop.c | 4 +--
negotiator/skipping.c | 4 +--
object-file.c | 80 ++++++++++++++++++++++++++++++++++++++++++-
read-cache.c | 3 ++
utf8.c | 11 ++++++
22 files changed, 292 insertions(+), 21 deletions(-)
base-commit: a9ecda2788e229afc9b611acaa26d0d9d4da53ed
Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-git-1537%2FHarithaIBM%2Fenablezos-v1
Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-git-1537/HarithaIBM/enablezos-v1
Pull-Request: https://github.com/git/git/pull/1537
--
gitgitgadget
^ permalink raw reply
* [PATCH 01/13] Enabling z/OS workflow for git
From: Haritha D via GitGitGadget @ 2023-11-13 10:24 UTC (permalink / raw)
To: git; +Cc: Haritha, Haritha D
In-Reply-To: <pull.1537.git.git.1699871056.gitgitgadget@gmail.com>
From: Haritha D <harithamma.d@ibm.com>
Our team ported Git to z/OS w/ significant modifcation.
This patch is Initial Config
Signed-off-by: Haritha D <harithamma.d@ibm.com>
---
config.mak.uname | 20 ++++++++++++++++++++
1 file changed, 20 insertions(+)
diff --git a/config.mak.uname b/config.mak.uname
index 3bb03f423a0..6ba9b707006 100644
--- a/config.mak.uname
+++ b/config.mak.uname
@@ -625,6 +625,26 @@ ifeq ($(uname_S),NONSTOP_KERNEL)
SANE_TOOL_PATH = /usr/coreutils/bin:/usr/local/bin
SHELL_PATH = /usr/coreutils/bin/bash
endif
+
+ifeq ($(uname_S),OS/390)
+ PERL_PATH = perl
+ PERL_PATH_FOR_SCRIPTS = /bin/env perl
+ SHELL_PATH = bash
+ SHELL_PATH_FOR_SCRIPTS = /bin/env bash
+ PYTHON_PATH = python
+ NO_SYS_POLL_H = YesPlease
+ NO_STRCASESTR = YesPlease
+ NO_REGEX = YesPlease
+ NO_MMAP = YesPlease
+ NO_NSEC = YesPlease
+ NO_STRLCPY = YesPlease
+ NO_MKDTEMP = YesPlease
+ NO_MEMMEM = YesPlease
+ NO_GECOS_IN_PWENT = YesPlease
+ HAVE_STRINGS_H = YesPlease
+ NEEDS_MODE_TRANSLATION = YesPlease
+endif
+
ifeq ($(uname_S),MINGW)
ifeq ($(shell expr "$(uname_R)" : '1\.'),2)
$(error "Building with MSys is no longer supported")
--
gitgitgadget
^ permalink raw reply related
* [PATCH 03/13] spaces and errors fix Handled git pipeline errors
From: Haritha D via GitGitGadget @ 2023-11-13 10:24 UTC (permalink / raw)
To: git; +Cc: Haritha, Haritha D
In-Reply-To: <pull.1537.git.git.1699871056.gitgitgadget@gmail.com>
From: Haritha D <harithamma.d@ibm.com>
This PR has fixes to enable build on z/OS
Signed-off-by: Harithamma D <harithamma.d@ibm.com>
---
builtin/hash-object.c | 6 +++---
convert.c | 22 +++++++++++++++++-----
entry.c | 22 +++++++++++-----------
object-file.c | 6 +++---
4 files changed, 34 insertions(+), 22 deletions(-)
diff --git a/builtin/hash-object.c b/builtin/hash-object.c
index b33b32ff977..9129658a37c 100644
--- a/builtin/hash-object.c
+++ b/builtin/hash-object.c
@@ -62,8 +62,8 @@ static void hash_fd(int fd, const char *type, const char *path, unsigned flags,
# include <stdio.h>
# include <stdlib.h>
- int setbinaryfd(int fd)
- {
+int setbinaryfd(int fd)
+{
attrib_t attr;
int rc;
@@ -74,7 +74,7 @@ static void hash_fd(int fd, const char *type, const char *path, unsigned flags,
rc = __fchattr(fd, &attr, sizeof(attr));
return rc;
- }
+}
# endif
#endif
diff --git a/convert.c b/convert.c
index 4f14ff6f1ed..17cc849efed 100644
--- a/convert.c
+++ b/convert.c
@@ -1315,15 +1315,28 @@ static struct attr_check *check;
static const char* get_platform() {
struct utsname uname_info;
+ char *result;
+ if(!uname_info.sysname)
+ {
+ result = (char *)malloc(strlen(uname_info.sysname)+1);
+ int index=0;
+ while(index <= strlen(uname_info.sysname))
+ {
+ *result = uname_info.sysname[index];
+ ++result;
+ ++index;
+ }
+ }
if (uname(&uname_info))
die(_("uname() failed with error '%s' (%d)\n"),
strerror(errno),
errno);
- if (!strcmp(uname_info.sysname, "OS/390"))
- return "zos";
- return uname_info.sysname;
+ if (!strcmp(uname_info.sysname, "OS/390"))
+ result="zos";
+
+ return result;
}
@@ -1331,11 +1344,10 @@ void convert_attrs(struct index_state *istate,
struct conv_attrs *ca, const char *path)
{
struct attr_check_item *ccheck = NULL;
- struct strbuf platform_working_tree_encoding = STRBUF_INIT;
+ struct strbuf platform_working_tree_encoding = STRBUF_INIT;
strbuf_addf(&platform_working_tree_encoding, "%s-working-tree-encoding", get_platform());
-
if (!check) {
check = attr_check_initl("crlf", "ident", "filter",
"eol", "text", "working-tree-encoding", platform_working_tree_encoding.buf,
diff --git a/entry.c b/entry.c
index df6feb2234b..f2a7b2adbf5 100644
--- a/entry.c
+++ b/entry.c
@@ -130,17 +130,17 @@ int fstat_checkout_output(int fd, const struct checkout *state, struct stat *st)
void tag_file_as_working_tree_encoding(struct index_state *istate, char* path, int fd) {
struct conv_attrs ca;
convert_attrs(istate, &ca, path);
- if (ca.attr_action != CRLF_BINARY) {
- if (ca.working_tree_encoding)
- __chgfdcodeset(fd, ca.working_tree_encoding);
- else
- __setfdtext(fd);
- }
- else {
- __setfdbinary(fd);
- }
-
- __disableautocvt(fd);
+ if (ca.attr_action != CRLF_BINARY) {
+ if (ca.working_tree_encoding)
+ __chgfdcodeset(fd, ca.working_tree_encoding);
+ else
+ __setfdtext(fd);
+ }
+ else {
+ __setfdbinary(fd);
+ }
+
+ __disableautocvt(fd);
}
#endif
diff --git a/object-file.c b/object-file.c
index 28e69ed1e33..562d1344422 100644
--- a/object-file.c
+++ b/object-file.c
@@ -2557,15 +2557,15 @@ int index_path(struct index_state *istate, struct object_id *oid,
switch (st->st_mode & S_IFMT) {
case S_IFREG:
#ifdef __MVS__
- validate_codeset(istate, path, &autocvtToASCII);
+ validate_codeset(istate, path, &autocvtToASCII);
#endif
fd = open(path, O_RDONLY);
if (fd < 0)
return error_errno("open(\"%s\")", path);
#ifdef __MVS__
- if (!autocvtToASCII)
- __disableautocvt(fd);
+ if (!autocvtToASCII)
+ __disableautocvt(fd);
#endif
if (index_fd(istate, oid, fd, st, OBJ_BLOB, path, flags) < 0)
--
gitgitgadget
^ permalink raw reply related
* [PATCH 04/13] fixes for build errors Handled git pipeline errorse
From: Haritha D via GitGitGadget @ 2023-11-13 10:24 UTC (permalink / raw)
To: git; +Cc: Haritha, Haritha D
In-Reply-To: <pull.1537.git.git.1699871056.gitgitgadget@gmail.com>
From: Haritha D <harithamma.d@ibm.com>
This PR has fixes to enable build on z/OS
Signed-off-by: Harithamma D <harithamma.d@ibm.com>
---
convert.c | 4 ++--
fetch-negotiator.h | 2 +-
fetch-pack.c | 2 +-
3 files changed, 4 insertions(+), 4 deletions(-)
diff --git a/convert.c b/convert.c
index 17cc849efed..da05f6c2e51 100644
--- a/convert.c
+++ b/convert.c
@@ -1313,13 +1313,13 @@ static int git_path_check_ident(struct attr_check_item *check)
static struct attr_check *check;
-static const char* get_platform() {
+static const char* get_platform(void) {
struct utsname uname_info;
char *result;
if(!uname_info.sysname)
{
- result = (char *)malloc(strlen(uname_info.sysname)+1);
int index=0;
+ result = (char *)malloc(strlen(uname_info.sysname)+1);
while(index <= strlen(uname_info.sysname))
{
*result = uname_info.sysname[index];
diff --git a/fetch-negotiator.h b/fetch-negotiator.h
index e348905a1f0..2e19ef247f9 100644
--- a/fetch-negotiator.h
+++ b/fetch-negotiator.h
@@ -47,7 +47,7 @@ struct fetch_negotiator {
*/
int (*ack)(struct fetch_negotiator *, struct commit *);
- void (*release)(struct fetch_negotiator *);
+ void (*release_negotiator)(struct fetch_negotiator *);
/* internal use */
void *data;
diff --git a/fetch-pack.c b/fetch-pack.c
index 26999e3b659..f40b90dfa65 100644
--- a/fetch-pack.c
+++ b/fetch-pack.c
@@ -1232,7 +1232,7 @@ static struct ref *do_fetch_pack(struct fetch_pack_args *args,
all_done:
if (negotiator)
- negotiator->release(negotiator);
+ negotiator->release_negotiator(negotiator);
return ref;
}
--
gitgitgadget
^ permalink raw reply related
* [PATCH 05/13] fixes for build errors
From: Haritha D via GitGitGadget @ 2023-11-13 10:24 UTC (permalink / raw)
To: git; +Cc: Haritha, Haritha D
In-Reply-To: <pull.1537.git.git.1699871056.gitgitgadget@gmail.com>
From: Haritha D <harithamma.d@ibm.com>
This patch has fixes for build errors
Signed-off-by: Haritha D <harithamma.d@ibm.com>
---
convert.c | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/convert.c b/convert.c
index da05f6c2e51..d3f204b4c29 100644
--- a/convert.c
+++ b/convert.c
@@ -1315,8 +1315,8 @@ static struct attr_check *check;
static const char* get_platform(void) {
struct utsname uname_info;
- char *result;
- if(!uname_info.sysname)
+ char *result = NULL;
+ if(!uname_info.sysname[0])
{
int index=0;
result = (char *)malloc(strlen(uname_info.sysname)+1);
--
gitgitgadget
^ permalink raw reply related
* [PATCH 06/13] spaces and errors fix Handled git pipeline errors
From: Haritha D via GitGitGadget @ 2023-11-13 10:24 UTC (permalink / raw)
To: git; +Cc: Haritha, Haritha D
In-Reply-To: <pull.1537.git.git.1699871056.gitgitgadget@gmail.com>
From: Haritha D <harithamma.d@ibm.com>
This PR has fixes to enable build on z/OS
Signed-off-by: Harithamma D <harithamma.d@ibm.com>
---
convert.c | 2 +-
fetch-pack.c | 2 +-
2 files changed, 2 insertions(+), 2 deletions(-)
diff --git a/convert.c b/convert.c
index d3f204b4c29..7fe107710ec 100644
--- a/convert.c
+++ b/convert.c
@@ -1314,7 +1314,7 @@ static int git_path_check_ident(struct attr_check_item *check)
static struct attr_check *check;
static const char* get_platform(void) {
- struct utsname uname_info;
+ struct utsname uname_info = {0};
char *result = NULL;
if(!uname_info.sysname[0])
{
diff --git a/fetch-pack.c b/fetch-pack.c
index f40b90dfa65..c1f2e714f8e 100644
--- a/fetch-pack.c
+++ b/fetch-pack.c
@@ -1853,7 +1853,7 @@ static struct ref *do_fetch_pack_v2(struct fetch_pack_args *args,
die("fsck failed");
if (negotiator)
- negotiator->release(negotiator);
+ negotiator->release_negotiator(negotiator);
oidset_clear(&common);
return ref;
--
gitgitgadget
^ permalink raw reply related
* [PATCH 02/13] Enable builds for z/OS.
From: Haritha D via GitGitGadget @ 2023-11-13 10:24 UTC (permalink / raw)
To: git; +Cc: Haritha, Haritha D
In-Reply-To: <pull.1537.git.git.1699871056.gitgitgadget@gmail.com>
From: Haritha D <harithamma.d@ibm.com>
This commit enables git to build on z/OS.
It takes advantage of enahanced ASCII
services on z/OS to auto-convert input
files to ASCII
It also adds support for
[platform]-working-tree-encoding.
Platform is substituted with uname_info.sysname,
so it will only apply to the given platform.
Also adds support for scripts that are not in
standard locations so that /bin/env bash
can be specified.
Signed-off-by: Harithamma D <harithamma.d@ibm.com>
---
Makefile | 21 +++++++++---
builtin.h | 3 ++
builtin/archive.c | 6 ++++
builtin/hash-object.c | 28 +++++++++++++++
combine-diff.c | 4 +++
config.c | 7 ++++
configure.ac | 3 ++
convert.c | 44 ++++++++++++++++++++----
copy.c | 3 ++
diff.c | 11 ++++++
entry.c | 26 ++++++++++++++
environment.c | 3 ++
git-compat-util.h | 8 +++++
negotiator/default.c | 4 +--
negotiator/noop.c | 4 +--
negotiator/skipping.c | 4 +--
object-file.c | 80 ++++++++++++++++++++++++++++++++++++++++++-
read-cache.c | 3 ++
utf8.c | 11 ++++++
19 files changed, 255 insertions(+), 18 deletions(-)
diff --git a/Makefile b/Makefile
index 9c6a2f125f8..30aa76da4f4 100644
--- a/Makefile
+++ b/Makefile
@@ -20,6 +20,8 @@ include shared.mak
#
# Define SHELL_PATH to a POSIX shell if your /bin/sh is broken.
#
+# Define SHELL_PATH_FOR_SCRIPTS to a POSIX shell if your /bin/sh is broken.
+#
# Define SANE_TOOL_PATH to a colon-separated list of paths to prepend
# to PATH if your tools in /usr/bin are broken.
#
@@ -215,6 +217,8 @@ include shared.mak
#
# Define PERL_PATH to the path of your Perl binary (usually /usr/bin/perl).
#
+# Define PERL_PATH_FOR_SCRIPTS to a Perl binary if your /usr/bin/perl is broken.
+#
# Define NO_PERL if you do not want Perl scripts or libraries at all.
#
# Define NO_PERL_CPAN_FALLBACKS if you do not want to install bundled
@@ -903,9 +907,15 @@ BINDIR_PROGRAMS_NO_X += git-cvsserver
ifndef SHELL_PATH
SHELL_PATH = /bin/sh
endif
+ifndef SHELL_PATH_FOR_SCRIPTS
+ SHELL_PATH_FOR_SCRIPTS = /bin/sh
+endif
ifndef PERL_PATH
PERL_PATH = /usr/bin/perl
endif
+ifndef PERL_PATH_FOR_SCRIPTS
+ PERL_PATH_FOR_SCRIPTS = /usr/bin/perl
+endif
ifndef PYTHON_PATH
PYTHON_PATH = /usr/bin/python
endif
@@ -1336,7 +1346,7 @@ THIRD_PARTY_SOURCES += sha1dc/%
# xdiff and reftable libs may in turn depend on what is in libgit.a
GITLIBS = common-main.o $(LIB_FILE) $(XDIFF_LIB) $(REFTABLE_LIB) $(LIB_FILE)
-EXTLIBS =
+EXTLIBS = $(ZOPEN_EXTRA_LIBS)
GIT_USER_AGENT = git/$(GIT_VERSION)
@@ -2226,9 +2236,10 @@ perllibdir_relative_SQ = $(subst ','\'',$(perllibdir_relative))
gitwebdir_SQ = $(subst ','\'',$(gitwebdir))
gitwebstaticdir_SQ = $(subst ','\'',$(gitwebstaticdir))
-SHELL_PATH_SQ = $(subst ','\'',$(SHELL_PATH))
+SHELL_PATH_SQ = $(subst ','\'',$(SHELL_PATH_FOR_SCRIPTS))
TEST_SHELL_PATH_SQ = $(subst ','\'',$(TEST_SHELL_PATH))
PERL_PATH_SQ = $(subst ','\'',$(PERL_PATH))
+PERL_PATH_FOR_SCRIPTS_SQ = $(subst ','\'',$(PERL_PATH_FOR_SCRIPTS))
PYTHON_PATH_SQ = $(subst ','\'',$(PYTHON_PATH))
TCLTK_PATH_SQ = $(subst ','\'',$(TCLTK_PATH))
DIFF_SQ = $(subst ','\'',$(DIFF))
@@ -2448,7 +2459,7 @@ hook-list.h: generate-hooklist.sh Documentation/githooks.txt
SCRIPT_DEFINES = $(SHELL_PATH_SQ):$(DIFF_SQ):\
$(localedir_SQ):$(USE_GETTEXT_SCHEME):$(SANE_TOOL_PATH_SQ):\
- $(gitwebdir_SQ):$(PERL_PATH_SQ):$(PAGER_ENV):\
+ $(gitwebdir_SQ):$(PERL_PATH_FOR_SCRIPTS_SQ):$(PAGER_ENV):\
$(perllibdir_SQ)
GIT-SCRIPT-DEFINES: FORCE
@FLAGS='$(SCRIPT_DEFINES)'; \
@@ -2465,7 +2476,7 @@ sed -e '1s|#!.*/sh|#!$(SHELL_PATH_SQ)|' \
-e 's/@@USE_GETTEXT_SCHEME@@/$(USE_GETTEXT_SCHEME)/g' \
-e $(BROKEN_PATH_FIX) \
-e 's|@@GITWEBDIR@@|$(gitwebdir_SQ)|g' \
- -e 's|@@PERL@@|$(PERL_PATH_SQ)|g' \
+ -e 's|@@PERL@@|$(PERL_PATH_FOR_SCRIPTS_SQ)|g' \
-e 's|@@PAGER_ENV@@|$(PAGER_ENV_SQ)|g' \
$@.sh >$@+
endef
@@ -2519,7 +2530,7 @@ PERL_DEFINES += $(gitexecdir) $(perllibdir) $(localedir)
$(SCRIPT_PERL_GEN): % : %.perl GIT-PERL-DEFINES GIT-PERL-HEADER GIT-VERSION-FILE
$(QUIET_GEN) \
sed -e '1{' \
- -e ' s|#!.*perl|#!$(PERL_PATH_SQ)|' \
+ -e ' s|#!.*perl|#!$(PERL_PATH_FOR_SCRIPTS_SQ)|' \
-e ' r GIT-PERL-HEADER' \
-e ' G' \
-e '}' \
diff --git a/builtin.h b/builtin.h
index d560baa6618..806af1a262d 100644
--- a/builtin.h
+++ b/builtin.h
@@ -250,5 +250,8 @@ int cmd_verify_pack(int argc, const char **argv, const char *prefix);
int cmd_show_ref(int argc, const char **argv, const char *prefix);
int cmd_pack_refs(int argc, const char **argv, const char *prefix);
int cmd_replace(int argc, const char **argv, const char *prefix);
+#ifdef __MVS__
+ extern int setbinaryfd(int);
+#endif
#endif
diff --git a/builtin/archive.c b/builtin/archive.c
index 90761fdfee0..53ec794356f 100644
--- a/builtin/archive.c
+++ b/builtin/archive.c
@@ -14,6 +14,12 @@
static void create_output_file(const char *output_file)
{
int output_fd = xopen(output_file, O_CREAT | O_WRONLY | O_TRUNC, 0666);
+#ifdef __MVS__
+ #if (__CHARSET_LIB == 1)
+ if (setbinaryfd(output_fd))
+ die_errno(_("could not tag archive file '%s'"), output_file);
+ #endif
+#endif
if (output_fd != 1) {
if (dup2(output_fd, 1) < 0)
die_errno(_("could not redirect output"));
diff --git a/builtin/hash-object.c b/builtin/hash-object.c
index 5ffec99dcea..b33b32ff977 100644
--- a/builtin/hash-object.c
+++ b/builtin/hash-object.c
@@ -57,11 +57,39 @@ static void hash_fd(int fd, const char *type, const char *path, unsigned flags,
maybe_flush_or_die(stdout, "hash to stdout");
}
+#ifdef __MVS__
+# if (__CHARSET_LIB == 1)
+# include <stdio.h>
+# include <stdlib.h>
+
+ int setbinaryfd(int fd)
+ {
+ attrib_t attr;
+ int rc;
+
+ memset(&attr, 0, sizeof(attr));
+ attr.att_filetagchg = 1;
+ attr.att_filetag.ft_ccsid = FT_BINARY;
+ attr.att_filetag.ft_txtflag = 0;
+
+ rc = __fchattr(fd, &attr, sizeof(attr));
+ return rc;
+ }
+# endif
+#endif
+
+
static void hash_object(const char *path, const char *type, const char *vpath,
unsigned flags, int literally)
{
int fd;
fd = xopen(path, O_RDONLY);
+#ifdef __MVS__
+# if (__CHARSET_LIB == 1)
+ if (setbinaryfd(fd))
+ die_errno("Cannot set to binary '%s'", path);
+# endif
+#endif
hash_fd(fd, type, vpath, flags, literally);
}
diff --git a/combine-diff.c b/combine-diff.c
index f90f4424829..73445a517c7 100644
--- a/combine-diff.c
+++ b/combine-diff.c
@@ -1082,6 +1082,10 @@ static void show_patch_diff(struct combine_diff_path *elem, int num_parent,
ssize_t done;
int is_file, i;
+#ifdef __MVS__
+ __disableautocvt(fd);
+#endif
+
elem->mode = canon_mode(st.st_mode);
/* if symlinks don't work, assume symlink if all parents
* are symlinks
diff --git a/config.c b/config.c
index f9a1cca4e8a..37c124a37c0 100644
--- a/config.c
+++ b/config.c
@@ -1521,6 +1521,13 @@ static int git_default_core_config(const char *var, const char *value,
return 0;
}
+ #ifdef __MVS__
+ if (!strcmp(var, "core.ignorefiletags")) {
+ ignore_file_tags = git_config_bool(var, value);
+ return 0;
+ }
+ #endif
+
if (!strcmp(var, "core.safecrlf")) {
int eol_rndtrp_die;
if (value && !strcasecmp(value, "warn")) {
diff --git a/configure.ac b/configure.ac
index 276593cd9dd..ed380504be6 100644
--- a/configure.ac
+++ b/configure.ac
@@ -463,6 +463,9 @@ else
CC_LD_DYNPATH=-Wl,+b,
else
CC_LD_DYNPATH=
+ if test "$(uname -s)" = "OS/390"; then
+ CC_LD_DYNPATH=-L
+ fi
AC_MSG_WARN([linker does not support runtime path to dynamic libraries])
fi
fi
diff --git a/convert.c b/convert.c
index a8870baff36..4f14ff6f1ed 100644
--- a/convert.c
+++ b/convert.c
@@ -377,12 +377,15 @@ static int check_roundtrip(const char *enc_name)
static const char *default_encoding = "UTF-8";
static int encode_to_git(const char *path, const char *src, size_t src_len,
- struct strbuf *buf, const char *enc, int conv_flags)
+ struct strbuf *buf, const char *enc, enum convert_crlf_action attr_action, int conv_flags)
{
char *dst;
size_t dst_len;
int die_on_error = conv_flags & CONV_WRITE_OBJECT;
+ if (attr_action == CRLF_BINARY) {
+ return 0;
+ }
/*
* No encoding is specified or there is nothing to encode.
* Tell the caller that the content was not modified.
@@ -403,6 +406,11 @@ static int encode_to_git(const char *path, const char *src, size_t src_len,
return 0;
trace_encoding("source", path, enc, src, src_len);
+#ifdef __MVS__
+ // Don't convert ISO8859-1 on z/OS
+ if (strcasecmp("ISO8859-1", enc) == 0)
+ return 0;
+#endif
dst = reencode_string_len(src, src_len, default_encoding, enc,
&dst_len);
if (!dst) {
@@ -468,11 +476,14 @@ static int encode_to_git(const char *path, const char *src, size_t src_len,
}
static int encode_to_worktree(const char *path, const char *src, size_t src_len,
- struct strbuf *buf, const char *enc)
+ struct strbuf *buf, enum convert_crlf_action attr_action, const char *enc)
{
char *dst;
size_t dst_len;
+ if (attr_action == CRLF_BINARY) {
+ return 0;
+ }
/*
* No encoding is specified or there is nothing to encode.
* Tell the caller that the content was not modified.
@@ -1302,18 +1313,37 @@ static int git_path_check_ident(struct attr_check_item *check)
static struct attr_check *check;
+static const char* get_platform() {
+ struct utsname uname_info;
+
+ if (uname(&uname_info))
+ die(_("uname() failed with error '%s' (%d)\n"),
+ strerror(errno),
+ errno);
+
+ if (!strcmp(uname_info.sysname, "OS/390"))
+ return "zos";
+ return uname_info.sysname;
+}
+
+
void convert_attrs(struct index_state *istate,
struct conv_attrs *ca, const char *path)
{
struct attr_check_item *ccheck = NULL;
+ struct strbuf platform_working_tree_encoding = STRBUF_INIT;
+
+ strbuf_addf(&platform_working_tree_encoding, "%s-working-tree-encoding", get_platform());
+
if (!check) {
check = attr_check_initl("crlf", "ident", "filter",
- "eol", "text", "working-tree-encoding",
+ "eol", "text", "working-tree-encoding", platform_working_tree_encoding.buf,
NULL);
user_convert_tail = &user_convert;
git_config(read_convert_config, NULL);
}
+ strbuf_release(&platform_working_tree_encoding);
git_check_attr(istate, path, check);
ccheck = check->items;
@@ -1334,6 +1364,8 @@ void convert_attrs(struct index_state *istate,
ca->crlf_action = CRLF_TEXT_CRLF;
}
ca->working_tree_encoding = git_path_check_encoding(ccheck + 5);
+ if (git_path_check_encoding(ccheck + 6))
+ ca->working_tree_encoding = git_path_check_encoding(ccheck + 6);
/* Save attr and make a decision for action */
ca->attr_action = ca->crlf_action;
@@ -1427,7 +1459,7 @@ int convert_to_git(struct index_state *istate,
len = dst->len;
}
- ret |= encode_to_git(path, src, len, dst, ca.working_tree_encoding, conv_flags);
+ ret |= encode_to_git(path, src, len, dst, ca.working_tree_encoding, ca.attr_action, conv_flags);
if (ret && dst) {
src = dst->buf;
len = dst->len;
@@ -1455,7 +1487,7 @@ void convert_to_git_filter_fd(struct index_state *istate,
if (!apply_filter(path, NULL, 0, fd, dst, ca.drv, CAP_CLEAN, NULL, NULL))
die(_("%s: clean filter '%s' failed"), path, ca.drv->name);
- encode_to_git(path, dst->buf, dst->len, dst, ca.working_tree_encoding, conv_flags);
+ encode_to_git(path, dst->buf, dst->len, dst, ca.working_tree_encoding, ca.attr_action, conv_flags);
crlf_to_git(istate, path, dst->buf, dst->len, dst, ca.crlf_action, conv_flags);
ident_to_git(dst->buf, dst->len, dst, ca.ident);
}
@@ -1487,7 +1519,7 @@ static int convert_to_working_tree_ca_internal(const struct conv_attrs *ca,
}
}
- ret |= encode_to_worktree(path, src, len, dst, ca->working_tree_encoding);
+ ret |= encode_to_worktree(path, src, len, dst, ca->attr_action, ca->working_tree_encoding);
if (ret) {
src = dst->buf;
len = dst->len;
diff --git a/copy.c b/copy.c
index 23d84c6c1db..63546aecf81 100644
--- a/copy.c
+++ b/copy.c
@@ -14,6 +14,9 @@ int copy_fd(int ifd, int ofd)
if (write_in_full(ofd, buffer, len) < 0)
return COPY_WRITE_ERROR;
}
+#ifdef __MVS__
+ __copyfdccsid(ifd, ofd);
+#endif
return 0;
}
diff --git a/diff.c b/diff.c
index 2c602df10a3..28b96d53dbc 100644
--- a/diff.c
+++ b/diff.c
@@ -4083,6 +4083,9 @@ int diff_populate_filespec(struct repository *r,
int check_binary = options ? options->check_binary : 0;
int err = 0;
int conv_flags = global_conv_flags_eol;
+#ifdef __MVS__
+ int autocvtToASCII;
+#endif
/*
* demote FAIL to WARN to allow inspecting the situation
* instead of refusing.
@@ -4155,9 +4158,17 @@ int diff_populate_filespec(struct repository *r,
s->is_binary = 1;
return 0;
}
+#ifdef __MVS__
+ validate_codeset(r->index, s->path, &autocvtToASCII);
+#endif
fd = open(s->path, O_RDONLY);
if (fd < 0)
goto err_empty;
+
+#ifdef __MVS__
+ if (!autocvtToASCII)
+ __disableautocvt(fd);
+#endif
s->data = xmmap(NULL, s->size, PROT_READ, MAP_PRIVATE, fd, 0);
close(fd);
s->should_munmap = 1;
diff --git a/entry.c b/entry.c
index 076e97eb89c..df6feb2234b 100644
--- a/entry.c
+++ b/entry.c
@@ -126,6 +126,24 @@ int fstat_checkout_output(int fd, const struct checkout *state, struct stat *st)
return 0;
}
+#ifdef __MVS__
+void tag_file_as_working_tree_encoding(struct index_state *istate, char* path, int fd) {
+ struct conv_attrs ca;
+ convert_attrs(istate, &ca, path);
+ if (ca.attr_action != CRLF_BINARY) {
+ if (ca.working_tree_encoding)
+ __chgfdcodeset(fd, ca.working_tree_encoding);
+ else
+ __setfdtext(fd);
+ }
+ else {
+ __setfdbinary(fd);
+ }
+
+ __disableautocvt(fd);
+}
+#endif
+
static int streaming_write_entry(const struct cache_entry *ce, char *path,
struct stream_filter *filter,
const struct checkout *state, int to_tempfile,
@@ -138,6 +156,10 @@ static int streaming_write_entry(const struct cache_entry *ce, char *path,
if (fd < 0)
return -1;
+#ifdef __MVS__
+ tag_file_as_working_tree_encoding(state->istate, path, fd);
+#endif
+
result |= stream_blob_to_fd(fd, &ce->oid, filter, 1);
*fstat_done = fstat_checkout_output(fd, state, statbuf);
result |= close(fd);
@@ -374,6 +396,10 @@ static int write_entry(struct cache_entry *ce, char *path, struct conv_attrs *ca
return error_errno("unable to create file %s", path);
}
+#ifdef __MVS__
+ tag_file_as_working_tree_encoding(state->istate, path, fd);
+#endif
+
wrote = write_in_full(fd, new_blob, size);
if (!to_tempfile)
fstat_done = fstat_checkout_output(fd, state, &st);
diff --git a/environment.c b/environment.c
index bb3c2a96a33..2e4d3a1e058 100644
--- a/environment.c
+++ b/environment.c
@@ -51,6 +51,9 @@ const char *git_hooks_path;
int zlib_compression_level = Z_BEST_SPEED;
int pack_compression_level = Z_DEFAULT_COMPRESSION;
int fsync_object_files = -1;
+#ifdef __MVS__
+int ignore_file_tags = 0;
+#endif
int use_fsync = -1;
enum fsync_method fsync_method = FSYNC_METHOD_DEFAULT;
enum fsync_component fsync_components = FSYNC_COMPONENTS_DEFAULT;
diff --git a/git-compat-util.h b/git-compat-util.h
index 3e7a59b5ff1..66e0abec24b 100644
--- a/git-compat-util.h
+++ b/git-compat-util.h
@@ -223,7 +223,15 @@ struct strbuf;
#include <sys/stat.h>
#include <fcntl.h>
#include <stddef.h>
+#ifdef __MVS__
+#define release stdlib_release
+#define fetch stdlib_fetch
+#endif
#include <stdlib.h>
+#ifdef __MVS__
+#undef fetch
+#undef release
+#endif
#include <stdarg.h>
#include <string.h>
#ifdef HAVE_STRINGS_H
diff --git a/negotiator/default.c b/negotiator/default.c
index 9a5b6963272..b1f9f153372 100644
--- a/negotiator/default.c
+++ b/negotiator/default.c
@@ -174,7 +174,7 @@ static int ack(struct fetch_negotiator *n, struct commit *c)
return known_to_be_common;
}
-static void release(struct fetch_negotiator *n)
+static void release_negotiator(struct fetch_negotiator *n)
{
clear_prio_queue(&((struct negotiation_state *)n->data)->rev_list);
FREE_AND_NULL(n->data);
@@ -187,7 +187,7 @@ void default_negotiator_init(struct fetch_negotiator *negotiator)
negotiator->add_tip = add_tip;
negotiator->next = next;
negotiator->ack = ack;
- negotiator->release = release;
+ negotiator->release_negotiator = release_negotiator;
negotiator->data = CALLOC_ARRAY(ns, 1);
ns->rev_list.compare = compare_commits_by_commit_date;
diff --git a/negotiator/noop.c b/negotiator/noop.c
index de39028ab7f..82089654d8b 100644
--- a/negotiator/noop.c
+++ b/negotiator/noop.c
@@ -30,7 +30,7 @@ static int ack(struct fetch_negotiator *n UNUSED, struct commit *c UNUSED)
return 0;
}
-static void release(struct fetch_negotiator *n UNUSED)
+static void release_negotiator(struct fetch_negotiator *n UNUSED)
{
/* nothing to release */
}
@@ -41,6 +41,6 @@ void noop_negotiator_init(struct fetch_negotiator *negotiator)
negotiator->add_tip = add_tip;
negotiator->next = next;
negotiator->ack = ack;
- negotiator->release = release;
+ negotiator->release_negotiator = release_negotiator;
negotiator->data = NULL;
}
diff --git a/negotiator/skipping.c b/negotiator/skipping.c
index 5b91520430c..783b3f27e63 100644
--- a/negotiator/skipping.c
+++ b/negotiator/skipping.c
@@ -243,7 +243,7 @@ static int ack(struct fetch_negotiator *n, struct commit *c)
return known_to_be_common;
}
-static void release(struct fetch_negotiator *n)
+static void release_negotiator(struct fetch_negotiator *n)
{
clear_prio_queue(&((struct data *)n->data)->rev_list);
FREE_AND_NULL(n->data);
@@ -256,7 +256,7 @@ void skipping_negotiator_init(struct fetch_negotiator *negotiator)
negotiator->add_tip = add_tip;
negotiator->next = next;
negotiator->ack = ack;
- negotiator->release = release;
+ negotiator->release_negotiator = release_negotiator;
negotiator->data = CALLOC_ARRAY(data, 1);
data->rev_list.compare = compare;
diff --git a/object-file.c b/object-file.c
index 7c7afe57936..28e69ed1e33 100644
--- a/object-file.c
+++ b/object-file.c
@@ -43,7 +43,9 @@
#include "setup.h"
#include "submodule.h"
#include "fsck.h"
-
+#ifdef __MVS__
+#include <_Ccsid.h>
+#endif
/* The maximum size for an object header. */
#define MAX_HEADER_LEN 32
@@ -2478,6 +2480,68 @@ int index_fd(struct index_state *istate, struct object_id *oid,
return ret;
}
+#ifdef __MVS__
+void validate_codeset(struct index_state *istate, const char *path, int* autoconvertToASCII) {
+ struct conv_attrs ca;
+ struct stat st;
+ unsigned short attr_ccsid;
+ unsigned short file_ccsid;
+
+ if (ignore_file_tags)
+ return;
+
+ *autoconvertToASCII = 0;
+ convert_attrs(istate, &ca, path);
+ if (ca.attr_action == CRLF_BINARY) {
+ attr_ccsid = FT_BINARY;
+ }
+ else if (ca.working_tree_encoding) {
+ attr_ccsid = __toCcsid(ca.working_tree_encoding);
+ }
+ else
+ attr_ccsid = 819;
+
+ if (stat(path, &st) < 0)
+ return;
+
+ file_ccsid = st.st_tag.ft_ccsid;
+
+ if (file_ccsid == FT_UNTAGGED) {
+ die("File %s is untagged, set the correct file tag (using the chtag command).", path);
+ }
+
+ if (attr_ccsid != file_ccsid) {
+ if (file_ccsid == 1047 && attr_ccsid == 819) {
+ *autoconvertToASCII = 1;
+ return;
+ }
+ // Allow tag mixing of 819 and 1208
+ if ((file_ccsid == 819 || file_ccsid == 1208) && (attr_ccsid == 1208 || attr_ccsid == 819)) {
+ return;
+ }
+ // Don't check for binary files, just add them
+ if (attr_ccsid == FT_BINARY)
+ return;
+
+ char attr_csname[_XOPEN_PATH_MAX] = {0};
+ char file_csname[_XOPEN_PATH_MAX] = {0};
+ if (attr_ccsid != FT_BINARY) {
+ __toCSName(attr_ccsid, attr_csname);
+ } else {
+ snprintf(attr_csname, _XOPEN_PATH_MAX, "%s", "binary");
+ }
+ if (file_ccsid != FT_BINARY) {
+ __toCSName(file_ccsid, file_csname);
+ } else {
+ snprintf(file_csname, _XOPEN_PATH_MAX, "%s", "binary");
+ }
+ die("%s added file: file tag (%s) does not match working-tree-encoding (%s)", path, file_csname, attr_csname);
+ }
+}
+#endif
+
+
+
int index_path(struct index_state *istate, struct object_id *oid,
const char *path, struct stat *st, unsigned flags)
{
@@ -2485,11 +2549,25 @@ int index_path(struct index_state *istate, struct object_id *oid,
struct strbuf sb = STRBUF_INIT;
int rc = 0;
+#ifdef __MVS__
+ struct conv_attrs ca;
+ int autocvtToASCII;
+#endif
+
switch (st->st_mode & S_IFMT) {
case S_IFREG:
+#ifdef __MVS__
+ validate_codeset(istate, path, &autocvtToASCII);
+#endif
fd = open(path, O_RDONLY);
if (fd < 0)
return error_errno("open(\"%s\")", path);
+
+#ifdef __MVS__
+ if (!autocvtToASCII)
+ __disableautocvt(fd);
+#endif
+
if (index_fd(istate, oid, fd, st, OBJ_BLOB, path, flags) < 0)
return error(_("%s: failed to insert into database"),
path);
diff --git a/read-cache.c b/read-cache.c
index 080bd39713b..75c06121302 100644
--- a/read-cache.c
+++ b/read-cache.c
@@ -205,6 +205,9 @@ static int ce_compare_data(struct index_state *istate,
int fd = git_open_cloexec(ce->name, O_RDONLY);
if (fd >= 0) {
+#ifdef __MVS__
+ __disableautocvt(fd);
+#endif
struct object_id oid;
if (!index_fd(istate, &oid, fd, st, OBJ_BLOB, ce->name, 0))
match = !oideq(&oid, &ce->oid);
diff --git a/utf8.c b/utf8.c
index 6a0dd25b0fe..b9cb56abf14 100644
--- a/utf8.c
+++ b/utf8.c
@@ -590,6 +590,17 @@ char *reencode_string_len(const char *in, size_t insz,
#endif
}
+#ifdef __MVS__
+ //HACK: For backwards compat, ISO8859-1 really means utf-8 in the z/OS world
+ if (strcasecmp("ISO8859-1", in_encoding) == 0) {
+ in_encoding = "UTF-8";
+ out_encoding = "UTF-8";
+ }
+ if (strcasecmp("ISO8859-1", out_encoding) == 0) {
+ in_encoding = "UTF-8";
+ out_encoding = "UTF-8";
+ }
+#endif
conv = iconv_open(out_encoding, in_encoding);
if (conv == (iconv_t) -1) {
in_encoding = fallback_encoding(in_encoding);
--
gitgitgadget
^ permalink raw reply related
* [PATCH 07/13] spaces and errors fix Handled git pipeline errors
From: Haritha D via GitGitGadget @ 2023-11-13 10:24 UTC (permalink / raw)
To: git; +Cc: Haritha, Haritha D
In-Reply-To: <pull.1537.git.git.1699871056.gitgitgadget@gmail.com>
From: Haritha D <harithamma.d@ibm.com>
This PR has fixes to enable build on z/OS
Signed-off-by: Harithamma D <harithamma.d@ibm.com>
---
convert.c | 18 +++++++++++-------
1 file changed, 11 insertions(+), 7 deletions(-)
diff --git a/convert.c b/convert.c
index 7fe107710ec..c8d30011458 100644
--- a/convert.c
+++ b/convert.c
@@ -1314,9 +1314,15 @@ static int git_path_check_ident(struct attr_check_item *check)
static struct attr_check *check;
static const char* get_platform(void) {
- struct utsname uname_info = {0};
+ struct utsname uname_info;
char *result = NULL;
- if(!uname_info.sysname[0])
+
+ if (uname(&uname_info) < 0)
+ die(_("uname() failed with error '%s' (%d)\n"),
+ strerror(errno),
+ errno);
+
+ if(*uname_info.sysname != '\0')
{
int index=0;
result = (char *)malloc(strlen(uname_info.sysname)+1);
@@ -1326,15 +1332,13 @@ static const char* get_platform(void) {
++result;
++index;
}
+ *result = '\0';
}
- if (uname(&uname_info))
- die(_("uname() failed with error '%s' (%d)\n"),
- strerror(errno),
- errno);
-
+#ifdef __MVS__
if (!strcmp(uname_info.sysname, "OS/390"))
result="zos";
+#endif
return result;
}
--
gitgitgadget
^ permalink raw reply related
* [PATCH 08/13] platform_name fix Handled git pipeline errors
From: Haritha D via GitGitGadget @ 2023-11-13 10:24 UTC (permalink / raw)
To: git; +Cc: Haritha, Haritha D
In-Reply-To: <pull.1537.git.git.1699871056.gitgitgadget@gmail.com>
From: Haritha D <harithamma.d@ibm.com>
This PR has fixes to enable build on z/OS
Signed-off-by: Harithamma D <harithamma.d@ibm.com>
---
convert.c | 11 +++--------
1 file changed, 3 insertions(+), 8 deletions(-)
diff --git a/convert.c b/convert.c
index c8d30011458..9cd0c1382ac 100644
--- a/convert.c
+++ b/convert.c
@@ -1321,19 +1321,14 @@ static const char* get_platform(void) {
die(_("uname() failed with error '%s' (%d)\n"),
strerror(errno),
errno);
-
+#ifndef __MVS__
if(*uname_info.sysname != '\0')
{
int index=0;
result = (char *)malloc(strlen(uname_info.sysname)+1);
- while(index <= strlen(uname_info.sysname))
- {
- *result = uname_info.sysname[index];
- ++result;
- ++index;
- }
- *result = '\0';
+ strncpy(result, uname_info.sysname, strlen(uname_info.sysname));
}
+#endif
#ifdef __MVS__
if (!strcmp(uname_info.sysname, "OS/390"))
--
gitgitgadget
^ permalink raw reply related
* [PATCH 09/13] strncpy fix Handled git pipeline errors
From: Haritha D via GitGitGadget @ 2023-11-13 10:24 UTC (permalink / raw)
To: git; +Cc: Haritha, Haritha D
In-Reply-To: <pull.1537.git.git.1699871056.gitgitgadget@gmail.com>
From: Haritha D <harithamma.d@ibm.com>
This PR has fixes to enable build on z/OS
Signed-off-by: Harithamma D <harithamma.d@ibm.com>
---
convert.c | 8 +++++++-
1 file changed, 7 insertions(+), 1 deletion(-)
diff --git a/convert.c b/convert.c
index 9cd0c1382ac..78403de422d 100644
--- a/convert.c
+++ b/convert.c
@@ -1326,7 +1326,13 @@ static const char* get_platform(void) {
{
int index=0;
result = (char *)malloc(strlen(uname_info.sysname)+1);
- strncpy(result, uname_info.sysname, strlen(uname_info.sysname));
+ while(index <= strlen(uname_info.sysname))
+ {
+ *result = uname_info.sysname[index];
+ ++result;
+ ++index;
+ }
+ *result = '\0';
}
#endif
--
gitgitgadget
^ permalink raw reply related
* [PATCH 10/13] strncpy fix Handled git pipeline errors
From: Haritha D via GitGitGadget @ 2023-11-13 10:24 UTC (permalink / raw)
To: git; +Cc: Haritha, Haritha D
In-Reply-To: <pull.1537.git.git.1699871056.gitgitgadget@gmail.com>
From: Haritha D <harithamma.d@ibm.com>
This PR has fixes to enable build on z/OS
Signed-off-by: Harithamma D <harithamma.d@ibm.com>
---
convert.c | 10 +++-------
1 file changed, 3 insertions(+), 7 deletions(-)
diff --git a/convert.c b/convert.c
index 78403de422d..ef44e6429da 100644
--- a/convert.c
+++ b/convert.c
@@ -1326,13 +1326,10 @@ static const char* get_platform(void) {
{
int index=0;
result = (char *)malloc(strlen(uname_info.sysname)+1);
- while(index <= strlen(uname_info.sysname))
+ while(result[index] = uname_info.sysname[index])
{
- *result = uname_info.sysname[index];
- ++result;
- ++index;
+ index++;
}
- *result = '\0';
}
#endif
@@ -1340,8 +1337,7 @@ static const char* get_platform(void) {
if (!strcmp(uname_info.sysname, "OS/390"))
result="zos";
#endif
-
- return result;
+ return (char*)result;
}
--
gitgitgadget
^ permalink raw reply related
* [PATCH 11/13] strncpy fix Handled git pipeline errors
From: Haritha D via GitGitGadget @ 2023-11-13 10:24 UTC (permalink / raw)
To: git; +Cc: Haritha, Haritha D
In-Reply-To: <pull.1537.git.git.1699871056.gitgitgadget@gmail.com>
From: Haritha D <harithamma.d@ibm.com>
This PR has fixes to enable build on z/OS
Signed-off-by: Harithamma D <harithamma.d@ibm.com>
---
convert.c | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
diff --git a/convert.c b/convert.c
index ef44e6429da..16173a1caf6 100644
--- a/convert.c
+++ b/convert.c
@@ -1326,8 +1326,9 @@ static const char* get_platform(void) {
{
int index=0;
result = (char *)malloc(strlen(uname_info.sysname)+1);
- while(result[index] = uname_info.sysname[index])
+ while(index <= strlen(uname_info.sysname))
{
+ result[index] = uname_info.sysname[index];
index++;
}
}
--
gitgitgadget
^ permalink raw reply related
* [PATCH 12/13] Handled git pipeline errors - Memory leak
From: Haritha D via GitGitGadget @ 2023-11-13 10:24 UTC (permalink / raw)
To: git; +Cc: Haritha, Haritha D
In-Reply-To: <pull.1537.git.git.1699871056.gitgitgadget@gmail.com>
From: Haritha D <harithamma.d@ibm.com>
This PR has fixes to enable build on z/OS
Signed-off-by: Harithamma D <harithamma.d@ibm.com>
---
convert.c | 18 ++++++++++--------
1 file changed, 10 insertions(+), 8 deletions(-)
diff --git a/convert.c b/convert.c
index 16173a1caf6..4c034bb714c 100644
--- a/convert.c
+++ b/convert.c
@@ -1313,9 +1313,9 @@ static int git_path_check_ident(struct attr_check_item *check)
static struct attr_check *check;
-static const char* get_platform(void) {
+static void get_platform(char** result) {
struct utsname uname_info;
- char *result = NULL;
+ *result = NULL;
if (uname(&uname_info) < 0)
die(_("uname() failed with error '%s' (%d)\n"),
@@ -1325,10 +1325,10 @@ static const char* get_platform(void) {
if(*uname_info.sysname != '\0')
{
int index=0;
- result = (char *)malloc(strlen(uname_info.sysname)+1);
+ *result = (char *)malloc(strlen(uname_info.sysname)+1);
while(index <= strlen(uname_info.sysname))
{
- result[index] = uname_info.sysname[index];
+ (*result)[index] = uname_info.sysname[index];
index++;
}
}
@@ -1336,9 +1336,8 @@ static const char* get_platform(void) {
#ifdef __MVS__
if (!strcmp(uname_info.sysname, "OS/390"))
- result="zos";
+ *result = "zos";
#endif
- return (char*)result;
}
@@ -1347,8 +1346,11 @@ void convert_attrs(struct index_state *istate,
{
struct attr_check_item *ccheck = NULL;
struct strbuf platform_working_tree_encoding = STRBUF_INIT;
-
- strbuf_addf(&platform_working_tree_encoding, "%s-working-tree-encoding", get_platform());
+ char* result=NULL;
+ get_platform(&result);
+ strbuf_addf(&platform_working_tree_encoding, "%s-working-tree-encoding", result);
+ if (result != NULL)
+ free (result);
if (!check) {
check = attr_check_initl("crlf", "ident", "filter",
--
gitgitgadget
^ permalink raw reply related
* [PATCH 13/13] Handled git pipeline errors - z/OS enable
From: Haritha D via GitGitGadget @ 2023-11-13 10:24 UTC (permalink / raw)
To: git; +Cc: Haritha, Haritha D
In-Reply-To: <pull.1537.git.git.1699871056.gitgitgadget@gmail.com>
From: Haritha D <harithamma.d@ibm.com>
This PR has fixes to enable build on z/OS
Signed-off-by: Harithamma D <harithamma.d@ibm.com>
---
convert.c | 6 ++----
1 file changed, 2 insertions(+), 4 deletions(-)
diff --git a/convert.c b/convert.c
index 4c034bb714c..c66f5b3ecec 100644
--- a/convert.c
+++ b/convert.c
@@ -1315,7 +1315,6 @@ static struct attr_check *check;
static void get_platform(char** result) {
struct utsname uname_info;
- *result = NULL;
if (uname(&uname_info) < 0)
die(_("uname() failed with error '%s' (%d)\n"),
@@ -1346,11 +1345,10 @@ void convert_attrs(struct index_state *istate,
{
struct attr_check_item *ccheck = NULL;
struct strbuf platform_working_tree_encoding = STRBUF_INIT;
- char* result=NULL;
+ char* result="Unknown";
get_platform(&result);
strbuf_addf(&platform_working_tree_encoding, "%s-working-tree-encoding", result);
- if (result != NULL)
- free (result);
+ free (result);
if (!check) {
check = attr_check_initl("crlf", "ident", "filter",
--
gitgitgadget
^ permalink raw reply related
* [PATCH] remote-curl: avoid hang if curl asks for more data after eof
From: Jiří Hruška @ 2023-11-13 11:25 UTC (permalink / raw)
To: git; +Cc: Jeff King, Jonathan Tan
It has been observed that under some circumstances, libcurl can call
our `CURLOPT_READFUNCTION` callback `rpc_out()` again even after
already getting a return value of zero (EOF) back once before.
This results in `rpc_read_from_out()` trying to read more from the
child process pipe, because `rpc->flush_read_but_not_sent` is reset to
false already the first time EOF is returned, and then the whole
operation gets stuck - the child process is already trying to read
a response back and will not write anything to the output pipe anymore,
while the parent/remote process is now blocked waiting to read more too
and never even finishes sending the request.
This commit addresses the bug with the following changes:
- Move resetting of the `flush_read_but_not_sent` flag to `post_rpc()`,
just before the `rpc_out()` function would be potentially used the
next time. This makes the callback return a zero any number of times
at the end of a finished upload, thus fixing the bug.
- Rename `flush_read_but_not_sent` to `read_from_out_done`, because the
flag is not as much about reading and sending the "flush" as about
the reading/sending phase being done and that `read_from_out()` must
not be called for this particular RPC exchange anymore.
- Simplify `post_rpc()` a bit by removing nested `if` statements, extra
logic and comments. This revealed a bug in `stateless_connect()`,
where `rpc.len` is reset to 0 after each request but not `rpc.pos`,
relying on it being reset elsewhere. This is being fixed as well.
- Update all related comments and add a few new ones to help the next
person understand how the chunked large request flow works easier.
- Rename the user data/context argument of libcurl streaming callbacks
to be consistently `userdata` everywhere, just like the official
documentation calls it. The current naming is confusing especially
where it is called `buffer_` now.
- Reset also `CURLOPT_POSTFIELDSIZE_LARGE` to its default value of -1
before reusing a handle in http.c / `get_active_slot()`. It can stay
set from a previous request, e.g. to 4 from an earlier probe "0000"
packet. Seems mostly harmless in practice, but looks confusing when
looking at libcurl state while debugging problems like this one.
Signed-off-by: Jiri Hruska <jirka@fud.cz>
---
Hi all. I would like to report a bug in Git's integration of libcurl for the
HTTP transport and propose a patch to fix it.
I have seen Git get stuck while fetching a large repo sometimes over the years,
but I have never paid too much attention to it, because just trying again made
it work. But this started to happen much more frequently to me a few months back
and pushed me to investigate better.
I get to work with a large repository with lots of commits, branches and overall
traffic and when I run `git fetch --tags --force` on it, there are a lot of refs
to synchronize even after just a few days. And Git happens to hang for me, like
50% of the time. After a lot of activity and network I/O, it just gets stuck.
I have dumped the relevant processes and found that it all hangs because there
is a deadlock between a parent "git-remote-https" (curl) process and a child
process handling a "fetch-pack" command. Both sides are blocked on reading more
from _their_ side of the IPC pipe, so neither gets to do anything.
The call stacks where both sides get stuck are as follows:
Parent: git-remote-https origin https://blah@dev.azure.com/blah/_git/meh
00 ntdll!ZwReadFile
01 KERNELBASE!ReadFile
02 msvcrt!_read_nolock
03 msvcrt!_read <<<<<<<<<<<<<<<<<<<<<<<< fd = child/client.out
04 git_remote_https!xread
05 git_remote_https!read_in_full
06 git_remote_https!get_packet_data
07 git_remote_https!packet_read_with_status
08 git_remote_https!rpc_read_from_out
09 git_remote_https!rpc_out
0a libcurl_4!CURLOPT_READFUNCTION callback
0b libcurl_4!...
0e git_remote_https!run_active_slot
0f git_remote_https!run_one_slot
10 git_remote_https!run_slot
11 git_remote_https!post_rpc
12 git_remote_https!rpc_service
13 git_remote_https!fetch_git
14 git_remote_https!fetch
15 git_remote_https!parse_fetch
16 git_remote_https!cmd_main
...
Child: git fetch-pack --stateless-rpc --stdin --lock-pack --thin
--no-progress https://blah@dev.azure.com/blah/_git/meh
00 ntdll!ZwReadFile
01 KERNELBASE!ReadFile
02 msvcrt!_read_nolock
03 msvcrt!_read <<<<<<<<<<<<<<<<<<<<<<<< fd = stdin (1)
04 git!xread
05 git!read_in_full
06 git!get_packet_data
07 git!packet_read_with_status
08 git!packet_reader_read
09 git!get_ack
0a git!find_common
0b git!do_fetch_patch
0c git!fetch_pack
0d git!cmd_fetch_pack
...
That is, the child is already trying to read the _response_ of an incremental
common commits negotiation, but the parent, orchestrating the whole RPC
interaction, has not finished sending the _request_ data yet and wants more.
I looked at the packets passed between the two processes, I read through the
whole rpc_service() - post_rpc() - rpc_out() flow many times, but I could not
see how it could all get into this blocked state. It was clear that everything
worked fine as long as the request data fit into one packet, but once the "large
request" code path was taken, things broke and got stuck. But I could not see
any weirdness in the implementation on the Git side that could trigger this.
Which was a correct observation, because the weirdness happens in libcurl!
It turns out that curl's function readwrite_upload() in lib/transfer.c [1] can
call the configured CURLOPT_READFUNCTION callback multiple times even if already
at EOF, even if the user callback has previously returned a zero to signal the
end of the data to upload before. It does not happen every time, only if the
next component in curl (OS socket, http/2 library, ...) is not able to send all
the data right away (and readwrite_upload() then runs again with offset > 0).
[1]: https://github.com/curl/curl/blob/ad051e1cbec68b2456a22661bf0800d49d085294/lib/transfer.c#L810
This is harmless when using the default fread() implementation, which the whole
CURLOPT_READFUNCTION is based on. But Git's rpc_out() function is not safe in
this regard. rpc->flush_read_but_not_sent is reset to false already before
returning a zero (EOF), so the next invocation goes to rpc_read_from_out()
again, reads from the silent pipe and blocks the upload from ever finishing.
A trivial solution would be to just take the line which resets the flag
/*
* The line length either does not need to be sent at
* all or has already been completely sent. Now we can
* return 0, indicating EOF, meaning that the flush has
* been fully sent.
*/
- rpc->flush_read_but_not_sent = 0;
return 0;
from rpc_out() and reset it only in post_rpc(), before the next time a large
request is being sent out and rpc_out() will go into play again:
if (large_request) {
...
rpc->initial_buffer = 1;
+ rpc->flush_read_but_not_sent = 0;
curl_easy_setopt(slot->curl, CURLOPT_READFUNCTION, rpc_out);
This way the CURLOPT_READFUNCTION would be returning zeros at the end of the
upload as long as needed, just like fread() at the end of a real file.
Hence, the bug could be fixed with just that two-lines change.
But while trying to figure out the above, I noticed a few things that prolonged
the time I needed to understand what was going on, so I would like to propose
some more changes to make the code perhaps a bit easier to read for the next
person who comes to hack on it after me.
The description of the extra modifications is in the commit message. All of
these changes are obviously optional and naturally subjective. I think that we
can all agree on some points (less indentation = good), but naming is hard,
and so is balance between unclear and too verbose, or when to split all
non-functional changes to a separate commit. So let me know if there are things
to do differently and I will gladly obey, it is your codebase after all.
Which brings me to the next topic, testing.
Validating the fix would be trivial with a mocked libcurl, but turns out to be
much harder with the integration-level test suite of this project.
I have validated this solution against a patched version of libcurl:
diff --git a/lib/transfer.c b/lib/transfer.c
index ba0410fc5..82141b648 100644
--- a/lib/transfer.c
+++ b/lib/transfer.c
@@ -915,6 +915,9 @@ static CURLcode readwrite_upload(struct Curl_easy *data,
k->upload_fromhere += offset;
result = Curl_fillreadbuffer(data,
data->set.upload_buffer_size-offset,
&fillcount);
+ if(!result && fillcount == 0)
+ result = Curl_fillreadbuffer(data,
data->set.upload_buffer_size-offset,
+ &fillcount);
k->upload_fromhere -= offset;
if(result)
return result;
I validated this solution on Windows, in the real-life case of the repo which
triggers these problems, managing to always fetch it smoothly after the fix.
But outside of that, and as far as the bundled test suite goes, I have failed to
write a test that could validate this problem does not ever occur again.
For all I know, the special case when curl calls the callback again happens only
on some operating systems (Windows), only with some versions of libcurl, only
with some dependencies (libnghttp2, OpenSSL) only when talking to some remote
servers (Microsoft Azure DevOps repos) over some protocols (HTTP/2), and only
with some back luck (network speed, TCP/IP window size, moon phase, ...).
Actually, I found at least one report from the past (2017) on Linux [2], which
seems to show exactly the same symptoms. Hard to tell if the root cause was
really the same too, though. The Internet seems full of "git fetch/pull hangs"
stories and there are always helpful people who jump in with their proven
solution of "git config --global http.postBuffer 100000000000000"... If this
bug fix could stop some of that nonsense, that would be awesome.
[2]: https://public-inbox.org/git/xmqqefzoxeki.fsf@gitster.mtv.corp.google.com/T/
Anyway, when I enabled extra GIT_TRACE_PACKET and GIT_CURL_VERBOSE, I could not
reproduce it anymore. When I tried to play with some of the t/*http*.sh
testcases, I did not manage to reproduce it either. And even then, reproducing
it _sometimes_ is not the same as reproducing it _always_. A flaky test is
often worse than no test at all.
So, I do not know how to write a test for this. Any suggestions are welcome.
Based on the guidance from Documentation/SubmittingPatches, as a bug fix, the
patch is based on the current `maint` (2.42.1). But applies cleanly to `master`,
`next` and `seen` as of 2023-11-12 too.
I have added Jeff and Jonathan to CC based on some previous commits to similar
places in remote-curl.c. Sorry if not or no longer relevant.
Thank you for your kind consideration.
Jiri
http.c | 1 +
remote-curl.c | 93 +++++++++++++++++++++++++++++----------------------
2 files changed, 54 insertions(+), 40 deletions(-)
diff --git a/http.c b/http.c
index 8f71bf00d8..14f2fbb82e 100644
--- a/http.c
+++ b/http.c
@@ -1454,6 +1454,7 @@ struct active_request_slot *get_active_slot(void)
curl_easy_setopt(slot->curl, CURLOPT_READFUNCTION, NULL);
curl_easy_setopt(slot->curl, CURLOPT_WRITEFUNCTION, NULL);
curl_easy_setopt(slot->curl, CURLOPT_POSTFIELDS, NULL);
+ curl_easy_setopt(slot->curl, CURLOPT_POSTFIELDSIZE_LARGE, (curl_off_t)-1);
curl_easy_setopt(slot->curl, CURLOPT_UPLOAD, 0);
curl_easy_setopt(slot->curl, CURLOPT_HTTPGET, 1);
curl_easy_setopt(slot->curl, CURLOPT_FAILONERROR, 1);
diff --git a/remote-curl.c b/remote-curl.c
index ef05752ca5..c043836214 100644
--- a/remote-curl.c
+++ b/remote-curl.c
@@ -606,13 +606,14 @@ struct rpc_state {
unsigned write_line_lengths : 1;
/*
- * Used by rpc_out; initialize to 0. This is true if a flush has been
- * read, but the corresponding line length (if write_line_lengths is
- * true) and EOF have not been sent to libcurl. Since each flush marks
- * the end of a request, each flush must be completely sent before any
- * further reading occurs.
+ * Used by rpc_out; initialize to 0. This is true if a flush packet
+ * has been read from the child process, signaling the end of the
+ * current data to send. There might be still some bytes pending in
+ * 'buf' (e.g. the corresponding line length, if write_line_lengths
+ * is true), but no more reads can be performed on the 'out' pipe as
+ * part of the current RPC exchange.
*/
- unsigned flush_read_but_not_sent : 1;
+ unsigned read_from_out_done : 1;
};
#define RPC_STATE_INIT { 0 }
@@ -675,48 +676,51 @@ static int rpc_read_from_out(struct rpc_state
*rpc, int options,
return 1;
}
+/*
+ * CURLOPT_READFUNCTION callback, called by libcurl when it wants more data
+ * to send out. Used only if the request did not fit into just one buffer and
+ * data must be streamed as it comes.
+ * Has the same semantics as fread(), but reads packets from the pipe from
+ * the child process instead. A return value of 0 (EOF) finishes the upload.
+ */
static size_t rpc_out(void *ptr, size_t eltsize,
- size_t nmemb, void *buffer_)
+ size_t nmemb, void *userdata)
{
size_t max = eltsize * nmemb;
- struct rpc_state *rpc = buffer_;
+ struct rpc_state *rpc = userdata;
size_t avail = rpc->len - rpc->pos;
enum packet_read_status status;
- if (!avail) {
+ /*
+ * If there is no more data available in our buffer and the child
+ * process is not done sending yet, read the next packet.
+ */
+ if (!avail && !rpc->read_from_out_done) {
rpc->initial_buffer = 0;
rpc->len = 0;
rpc->pos = 0;
- if (!rpc->flush_read_but_not_sent) {
- if (!rpc_read_from_out(rpc, 0, &avail, &status))
- BUG("The entire rpc->buf should be larger than LARGE_PACKET_MAX");
- if (status == PACKET_READ_FLUSH)
- rpc->flush_read_but_not_sent = 1;
- }
+ if (!rpc_read_from_out(rpc, 0, &avail, &status))
+ BUG("The entire rpc->buf should be larger than LARGE_PACKET_MAX");
+
/*
- * If flush_read_but_not_sent is true, we have already read one
- * full request but have not fully sent it + EOF, which is why
- * we need to refrain from reading.
- */
- }
- if (rpc->flush_read_but_not_sent) {
- if (!avail) {
- /*
- * The line length either does not need to be sent at
- * all or has already been completely sent. Now we can
- * return 0, indicating EOF, meaning that the flush has
- * been fully sent.
- */
- rpc->flush_read_but_not_sent = 0;
- return 0;
- }
- /*
- * If avail is non-zero, the line length for the flush still
- * hasn't been fully sent. Proceed with sending the line
- * length.
+ * If a flush packet was read, it means the child process is
+ * done sending this request. The buffer might be fully empty
+ * at this point or contain a flush packet too, depending on
+ * rpc->write_line_lengths.
+ * In any case, we must refrain from reading any more, because
+ * the child process already expects to receive a response back
+ * instead. If both sides would try to read at once, they would
+ * just hang waiting for each other.
*/
+ rpc->read_from_out_done = (status == PACKET_READ_FLUSH);
}
+ /*
+ * Copy data to the provided buffer. If there is nothing more to send,
+ * nothing gets written and the return value is 0 (EOF).
+ * It is important to keep returning 0 as long as needed in that case,
+ * as libcurl invokes the callback multiple times at EOF sometimes.
+ */
if (max < avail)
avail = max;
memcpy(ptr, rpc->buf + rpc->pos, avail);
@@ -724,9 +728,16 @@ static size_t rpc_out(void *ptr, size_t eltsize,
return avail;
}
-static int rpc_seek(void *clientp, curl_off_t offset, int origin)
+/*
+ * CURLOPT_SEEKFUNCTION callback, called by libcurl when it wants to seek in
+ * the data being sent out. Used only if the request did not fit into just
+ * one buffer and data must be streamed as it comes.
+ * Has the same semantics as fseek(), but seeks in the buffered packet read
+ * from the pipe from the child process instead.
+ */
+static int rpc_seek(void *userdata, curl_off_t offset, int origin)
{
- struct rpc_state *rpc = clientp;
+ struct rpc_state *rpc = userdata;
if (origin != SEEK_SET)
BUG("rpc_seek only handles SEEK_SET, not %d", origin);
@@ -800,10 +811,10 @@ struct rpc_in_data {
* from ptr.
*/
static size_t rpc_in(char *ptr, size_t eltsize,
- size_t nmemb, void *buffer_)
+ size_t nmemb, void *userdata)
{
size_t size = eltsize * nmemb;
- struct rpc_in_data *data = buffer_;
+ struct rpc_in_data *data = userdata;
long response_code;
if (curl_easy_getinfo(data->slot->curl, CURLINFO_RESPONSE_CODE,
@@ -963,6 +974,7 @@ static int post_rpc(struct rpc_state *rpc, int
stateless_connect, int flush_rece
*/
headers = curl_slist_append(headers, "Transfer-Encoding: chunked");
rpc->initial_buffer = 1;
+ rpc->read_from_out_done = 0;
curl_easy_setopt(slot->curl, CURLOPT_READFUNCTION, rpc_out);
curl_easy_setopt(slot->curl, CURLOPT_INFILE, rpc);
curl_easy_setopt(slot->curl, CURLOPT_SEEKFUNCTION, rpc_seek);
@@ -1482,7 +1494,7 @@ static int stateless_connect(const char *service_name)
rpc.gzip_request = 1;
rpc.initial_buffer = 0;
rpc.write_line_lengths = 1;
- rpc.flush_read_but_not_sent = 0;
+ rpc.read_from_out_done = 0;
/*
* Dump the capability listing that we got from the server earlier
@@ -1505,6 +1517,7 @@ static int stateless_connect(const char *service_name)
break;
/* Reset the buffer for next request */
rpc.len = 0;
+ rpc.pos = 0;
}
free(rpc.service_url);
--
2.42.1.1.g18afbdf3b0
^ permalink raw reply related
* Re: [PATCH] checkout: add config variable checkout.autoDetach
From: Phillip Wood @ 2023-11-13 15:07 UTC (permalink / raw)
To: Junio C Hamano, Andy Koppe; +Cc: git, pclouds
In-Reply-To: <xmqqbkbzo6ba.fsf@gitster.g>
On 12/11/2023 06:04, Junio C Hamano wrote:
> Andy Koppe <andy.koppe@gmail.com> writes:
>
>> The git-checkout command without pathspecs automatically detaches HEAD
>> when switching to something other than a branch, whereas git-switch
>> requires the --detach option to do so.
>>
>> Add configuration variable checkout.autoDetach to choose the behavior
>> for both: true for automatic detaching, false for requiring --detach.
>>
>> Amend their documentation and tests accordingly.
>>
>> Signed-off-by: Andy Koppe <andy.koppe@gmail.com>
>> ---
>
> "switch" was meant to be an experimental command to sort out this
> kind of UI ideas, and I think the fact that it requires a more
> explicit "--detach", where experienced users might just say "git
> checkout that-branch^0", has established itself as a more friendly
> and good thing to help new users. I do not know how others react to
> this kind of proliferation of configuration variables, but I do not
> mind this particular variable existing.
I'm a bit wary of having a config variable that could break scripts
relying on the current behavior of "git checkout". As far as "git
switch" goes I don't particularly mind this config variable though I'm
not sure it is that hard to type "--detach" (especially with tab
completion) and I do worry that we're making the UI more complex each
time we add something like this.
Best Wishes
Phillip
^ permalink raw reply
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox