* What's cooking in git.git (Oct 2023, #05; Thu, 12)
From: Junio C Hamano @ 2023-10-12 23:21 UTC (permalink / raw)
To: git
Here are the topics that have been cooking in my tree. Commits
prefixed with '+' are in 'next' (being in 'next' is a sign that a
topic is stable enough to be used and are candidate to be in a
future release). Commits prefixed with '-' are only in 'seen', and
aren't considered "accepted" at all and may be annotated with an URL
to a message that raises issues but they are no means exhaustive. A
topic without enough support may be discarded after a long period of
no activity (of course they can be resubmit when new interests
arise).
There are too many topics marked with "Needs review" label. I can
help reviewing some of them myself, but obviously it will not scale.
Copies of the source code to Git live in many repositories, and the
following is a list of the ones I push into or their mirrors. Some
repositories have only a subset of branches.
With maint, master, next, seen, todo:
git://git.kernel.org/pub/scm/git/git.git/
git://repo.or.cz/alt-git.git/
https://kernel.googlesource.com/pub/scm/git/git/
https://github.com/git/git/
https://gitlab.com/git-vcs/git/
With all the integration branches and topics broken out:
https://github.com/gitster/git/
Even though the preformatted documentation in HTML and man format
are not sources, they are published in these repositories for
convenience (replace "htmldocs" with "manpages" for the manual
pages):
git://git.kernel.org/pub/scm/git/git-htmldocs.git/
https://github.com/gitster/git-htmldocs.git/
Release tarballs are available at:
https://www.kernel.org/pub/software/scm/git/
--------------------------------------------------
[Graduated to 'master']
* jm/git-status-submodule-states-docfix (2023-10-04) 1 commit
(merged to 'next' on 2023-10-04 at 520b7711a4)
+ git-status.txt: fix minor asciidoc format issue
Docfix.
source: <pull.1591.v3.git.1696386165616.gitgitgadget@gmail.com>
* js/ci-coverity (2023-10-05) 6 commits
(merged to 'next' on 2023-10-05 at 253788f0d1)
+ coverity: detect and report when the token or project is incorrect
+ coverity: allow running on macOS
+ coverity: support building on Windows
+ coverity: allow overriding the Coverity project
+ coverity: cache the Coverity Build Tool
+ ci: add a GitHub workflow to submit Coverity scans
GitHub CI workflow has learned to trigger Coverity check.
source: <pull.1588.v2.git.1695642662.gitgitgadget@gmail.com>
* rs/parse-opt-ctx-cleanup (2023-10-03) 1 commit
(merged to 'next' on 2023-10-04 at d5d0a2ce3b)
+ parse-options: drop unused parse_opt_ctx_t member
Code clean-up.
source: <ebcaa9e1-d306-4c93-adec-3f35d7040531@web.de>
--------------------------------------------------
[New Topics]
* jc/fail-stash-to-store-non-stash (2023-10-11) 1 commit
- stash: be careful what we store
Feeding "git stash store" with a random commit that was not created
by "git stash create" now errors out.
Will merge to 'next'?
source: <xmqqbkd4lwj0.fsf_-_@gitster.g>
* jc/doc-unit-tests-fixup (2023-10-11) 1 commit
- SQUASH???
(this branch uses js/doc-unit-tests and js/doc-unit-tests-with-cmake.)
Quick fix for jc/doc-unit-tests topic to unbreak CI running on 'seen'.
source: <xmqqwmvskf8t.fsf@gitster.g>
* bc/racy-4gb-files (2023-10-12) 2 commits
- Prevent git from rehashing 4GiB files
- t: add a test helper to truncate files
The index file has room only for lower 32-bit of the file size in
the cached stat information, which means cached stat information
will have 0 in its sd_size member for a file whose size is multiple
of 4GiB. This is mistaken for a racily clean path. Avoid it by
storing a bogus sd_size value instead for such files.
Waiting for review response.
source: <20231012160930.330618-1-sandals@crustytoothpaste.net>
* ds/mailmap-entry-update (2023-10-12) 1 commit
(merged to 'next' on 2023-10-12 at 3de300ac62)
+ mailmap: change primary address for Derrick Stolee
Update mailmap entry for Derrick.
Will merge to 'master' immediately.
source: <pull.1592.git.1697131834003.gitgitgadget@gmail.com>
* jc/grep-f-relative-to-cwd (2023-10-12) 1 commit
- grep: -f <path> is relative to $cwd
"cd sub && git grep -f patterns" tried to read "patterns" file at
the top level of the working tree; it has been corrected to read
"sub/patterns" instead.
Needs review.
source: <xmqqedhzg37z.fsf@gitster.g>
--------------------------------------------------
[Stalled]
* pw/rebase-sigint (2023-09-07) 1 commit
- rebase -i: ignore signals when forking subprocesses
If the commit log editor or other external programs (spawned via
"exec" insn in the todo list) receive internactive signal during
"git rebase -i", it caused not just the spawned program but the
"Git" process that spawned them, which is often not what the end
user intended. "git" learned to ignore SIGINT and SIGQUIT while
waiting for these subprocesses.
Expecting a reroll.
cf. <12c956ea-330d-4441-937f-7885ab519e26@gmail.com>
source: <pull.1581.git.1694080982621.gitgitgadget@gmail.com>
* tk/cherry-pick-sequence-requires-clean-worktree (2023-06-01) 1 commit
- cherry-pick: refuse cherry-pick sequence if index is dirty
"git cherry-pick A" that replays a single commit stopped before
clobbering local modification, but "git cherry-pick A..B" did not,
which has been corrected.
Expecting a reroll.
cf. <999f12b2-38d6-f446-e763-4985116ad37d@gmail.com>
source: <pull.1535.v2.git.1685264889088.gitgitgadget@gmail.com>
* jc/diff-cached-fsmonitor-fix (2023-09-15) 3 commits
- diff-lib: fix check_removed() when fsmonitor is active
- Merge branch 'jc/fake-lstat' into jc/diff-cached-fsmonitor-fix
- Merge branch 'js/diff-cached-fsmonitor-fix' into jc/diff-cached-fsmonitor-fix
(this branch uses jc/fake-lstat.)
The optimization based on fsmonitor in the "diff --cached"
codepath is resurrected with the "fake-lstat" introduced earlier.
It is unknown if the optimization is worth resurrecting, but in case...
source: <xmqqr0n0h0tw.fsf@gitster.g>
--------------------------------------------------
[Cooking]
* tb/path-filter-fix (2023-10-10) 17 commits
- bloom: introduce `deinit_bloom_filters()`
- commit-graph: reuse existing Bloom filters where possible
- object.h: fix mis-aligned flag bits table
- commit-graph: drop unnecessary `graph_read_bloom_data_context`
- commit-graph.c: unconditionally load Bloom filters
- bloom: prepare to discard incompatible Bloom filters
- bloom: annotate filters with hash version
- commit-graph: new filter ver. that fixes murmur3
- repo-settings: introduce commitgraph.changedPathsVersion
- t4216: test changed path filters with high bit paths
- t/helper/test-read-graph: implement `bloom-filters` mode
- bloom.h: make `load_bloom_filter_from_graph()` public
- t/helper/test-read-graph.c: extract `dump_graph_info()`
- gitformat-commit-graph: describe version 2 of BDAT
- commit-graph: ensure Bloom filters are read with consistent settings
- revision.c: consult Bloom filters for root commits
- t/t4216-log-bloom.sh: harden `test_bloom_filters_not_used()`
The Bloom filter used for path limited history traversal was broken
on systems whose "char" is unsigned; update the implementation and
bump the format version to 2.
Needs (hopefully final and quick) review.
source: <cover.1696969994.git.me@ttaylorr.com>
* ak/pretty-decorate-more-fix (2023-10-09) 1 commit
(merged to 'next' on 2023-10-12 at 3cbb4c2268)
+ pretty: fix ref filtering for %(decorate) formats
Unlike "git log --pretty=%D", "git log --pretty="%(decorate)" did
not auto-initialize the decoration subsystem, which has been
corrected.
Will merge to 'master'.
source: <20231008202307.1568477-1-andy.koppe@gmail.com>
* en/docfixes (2023-10-09) 25 commits
- documentation: add missing parenthesis
- documentation: add missing quotes
- documentation: add missing fullstops
- documentation: add some commas where they are helpful
- documentation: fix whitespace issues
- documentation: fix capitalization
- documentation: fix punctuation
- documentation: use clearer prepositions
- documentation: add missing hyphens
- documentation: remove unnecessary hyphens
- documentation: add missing article
- documentation: fix choice of article
- documentation: whitespace is already generally plural
- documentation: fix singular vs. plural
- documentation: fix verb vs. noun
- documentation: fix adjective vs. noun
- documentation: fix verb tense
- documentation: employ consistent verb tense for a list
- documentation: fix subject/verb agreement
- documentation: remove extraneous words
- documentation: add missing words
- documentation: fix apostrophe usage
- documentation: fix typos
- documentation: fix small error
- documentation: wording improvements
Documentation typo and grammo fixes.
Needs review.
source: <pull.1595.git.1696747527.gitgitgadget@gmail.com>
* kn/rev-list-missing-fix (2023-10-09) 3 commits
. rev-list: add commit object support in `--missing` option
. rev-list: move `show_commit()` to the bottom
. revision: rename bit to `do_not_die_on_missing_objects`
"git rev-list --missing" did not work for missing commit objects,
which has been corrected.
Needs review.
Seems to break CI job with extra environment settings.
cf. <xmqqil7etndo.fsf@gitster.g>
source: <20231009105528.17777-1-karthik.188@gmail.com>
* sn/cat-file-doc-update (2023-10-09) 1 commit
(merged to 'next' on 2023-10-10 at 6719ba7bd4)
+ doc/cat-file: make synopsis and description less confusing
"git cat-file" documentation updates.
Will merge to 'master'.
source: <20231009175604.2361700-1-stepnem@smrk.net>
* xz/commit-title-soft-limit-doc (2023-10-09) 1 commit
(merged to 'next' on 2023-10-10 at 0bf3d9ed51)
+ doc: correct the 50 characters soft limit (+)
Doc update.
Will merge to 'master'.
source: <pull.1585.v2.git.git.1696778367151.gitgitgadget@gmail.com>
* jk/chunk-bounds (2023-10-09) 20 commits
(merged to 'next' on 2023-10-10 at 21139603ce)
+ chunk-format: drop pair_chunk_unsafe()
+ commit-graph: detect out-of-order BIDX offsets
+ commit-graph: check bounds when accessing BIDX chunk
+ commit-graph: check bounds when accessing BDAT chunk
+ commit-graph: bounds-check generation overflow chunk
+ commit-graph: check size of generations chunk
+ commit-graph: bounds-check base graphs chunk
+ commit-graph: detect out-of-bounds extra-edges pointers
+ commit-graph: check size of commit data chunk
+ midx: check size of revindex chunk
+ midx: bounds-check large offset chunk
+ midx: check size of object offset chunk
+ midx: enforce chunk alignment on reading
+ midx: check size of pack names chunk
+ commit-graph: check consistency of fanout table
+ midx: check size of oid lookup chunk
+ commit-graph: check size of oid fanout chunk
+ midx: stop ignoring malformed oid fanout chunk
+ t: add library for munging chunk-format files
+ chunk-format: note that pair_chunk() is unsafe
The codepaths that read "chunk" formatted files have been corrected
to pay attention to the chunk size and notice broken files.
Will merge to 'master'.
source: <20231009205544.GA3281950@coredump.intra.peff.net>
* ps/rewritten-is-per-worktree-doc (2023-10-10) 1 commit
(merged to 'next' on 2023-10-12 at 501b960e8c)
+ doc/git-worktree: mention "refs/rewritten" as per-worktree refs
Doc update.
Will merge to 'master'.
source: <985ac850eb6e60ae76601acc8bfbcd56f99348b4.1696935657.git.ps@pks.im>
* jc/merge-ort-attr-index-fix (2023-10-09) 1 commit
(merged to 'next' on 2023-10-10 at b139b87502)
+ merge-ort: initialize repo in index state
Fix "git merge-tree" to stop segfaulting when the --attr-source
option is used.
Will merge to 'master'.
source: <pull.1583.v3.git.git.1696857660374.gitgitgadget@gmail.com>
* jk/decoration-and-other-leak-fixes (2023-10-05) 3 commits
(merged to 'next' on 2023-10-06 at 5fc05c94dc)
+ daemon: free listen_addr before returning
+ revision: clear decoration structs during release_revisions()
+ decorate: add clear_decoration() function
Leakfix.
Will merge to 'master'.
source: <20231005212802.GA982892@coredump.intra.peff.net>
* sn/typo-grammo-phraso-fixes (2023-10-05) 5 commits
- t/README: fix multi-prerequisite example
- doc/gitk: s/sticked/stuck/
- git-jump: admit to passing merge mode args to ls-files
- doc/diff-options: improve wording of the log.diffMerges mention
- doc: fix some typos, grammar and wording issues
Many typos, ungrammatical sentences and wrong phrasing have been
fixed.
Needs review.
source: <20231003082107.3002173-1-stepnem@smrk.net>
* so/diff-merges-dd (2023-10-09) 3 commits
- completion: complete '--dd'
- diff-merges: introduce '--dd' option
- diff-merges: improve --diff-merges documentation
"git log" and friends learned "--dd" that is a short-hand for
"--diff-merges=first-parent -p".
Will merge to 'next'?
source: <20231009160535.236523-1-sorganov@gmail.com>
* vd/loose-ref-iteration-optimization (2023-10-09) 4 commits
(merged to 'next' on 2023-10-12 at 99e2f83855)
+ files-backend.c: avoid stat in 'loose_fill_ref_dir'
+ dir.[ch]: add 'follow_symlink' arg to 'get_dtype'
+ dir.[ch]: expose 'get_dtype'
+ ref-cache.c: fix prefix matching in ref iteration
The code to iterate over loose references have been optimized to
reduce the number of lstat() system calls.
Will merge to 'master'.
source: <pull.1594.v2.git.1696888736.gitgitgadget@gmail.com>
* jc/update-list-references-to-lore (2023-10-06) 1 commit
- doc: update list archive reference to use lore.kernel.org
Doc update.
Needs review.
source: <xmqq7cnz741s.fsf@gitster.g>
* cc/git-replay (2023-10-10) 14 commits
- replay: stop assuming replayed branches do not diverge
- replay: add --contained to rebase contained branches
- replay: add --advance or 'cherry-pick' mode
- replay: use standard revision ranges
- replay: make it a minimal server side command
- replay: remove HEAD related sanity check
- replay: remove progress and info output
- replay: add an important FIXME comment about gpg signing
- replay: change rev walking options
- replay: introduce pick_regular_commit()
- replay: die() instead of failing assert()
- replay: start using parse_options API
- replay: introduce new builtin
- t6429: remove switching aspects of fast-rebase
Introduce "git replay", a tool meant on the server side without
working tree to recreate a history.
Needs (hopefully final and quick) review.
source: <20231010123847.2777056-1-christian.couder@gmail.com>
* jk/commit-graph-leak-fixes (2023-10-03) 10 commits
(merged to 'next' on 2023-10-06 at 5d202ef8b9)
+ commit-graph: clear oidset after finishing write
+ commit-graph: free write-context base_graph_name during cleanup
+ commit-graph: free write-context entries before overwriting
+ commit-graph: free graph struct that was not added to chain
+ commit-graph: delay base_graph assignment in add_graph_to_chain()
+ commit-graph: free all elements of graph chain
+ commit-graph: move slab-clearing to close_commit_graph()
+ merge: free result of repo_get_merge_bases()
+ commit-reach: free temporary list in get_octopus_merge_bases()
+ t6700: mark test as leak-free
Leakfix.
Will merge to 'master'.
source: <20231003202504.GA7697@coredump.intra.peff.net>
* tb/repack-max-cruft-size (2023-10-09) 5 commits
(merged to 'next' on 2023-10-09 at 38f039e880)
+ repack: free existing_cruft array after use
(merged to 'next' on 2023-10-06 at b3ca6df3b9)
+ builtin/repack.c: avoid making cruft packs preferred
+ builtin/repack.c: implement support for `--max-cruft-size`
+ builtin/repack.c: parse `--max-pack-size` with OPT_MAGNITUDE
+ t7700: split cruft-related tests to t7704
"git repack" learned "--max-cruft-size" to prevent cruft packs from
growing without bounds.
Will merge to 'master'.
source: <cover.1696293862.git.me@ttaylorr.com>
source: <20231007172031.GA1524950@coredump.intra.peff.net>
source: <035393935108d02aaf8927189b05102f4f74f340.1696370003.git.me@ttaylorr.com>
* ak/color-decorate-symbols (2023-10-03) 1 commit
- decorate: add color.decorate.symbols config option
A new config for coloring.
Needs review.
source: <20231003205442.22963-1-andy.koppe@gmail.com>
* jc/attr-tree-config (2023-10-11) 2 commits
- attr: add attr.tree for setting the treeish to read attributes from
- attr: read attributes from HEAD when bare repo
The attribute subsystem learned to honor `attr.tree` configuration
that specifies which tree to read the .gitattributes files from.
Will merge to 'next'?
source: <pull.1577.v4.git.git.1697044422.gitgitgadget@gmail.com>
* js/submodule-fix-misuse-of-path-and-name (2023-10-03) 6 commits
(merged to 'next' on 2023-10-06 at 1054b6e752)
+ t7420: test that we correctly handle renamed submodules
+ t7419: test that we correctly handle renamed submodules
+ t7419, t7420: use test_cmp_config instead of grepping .gitmodules
+ t7419: actually test the branch switching
+ submodule--helper: return error from set-url when modifying failed
+ submodule--helper: use submodule_from_path in set-{url,branch}
In .gitmodules files, submodules are keyed by their names, and the
path to the submodule whose name is $name is specified by the
submodule.$name.path variable. There were a few codepaths that
mixed the name and path up when consulting the submodule database,
which have been corrected. It took long for these bugs to be found
as the name of a submodule initially is the same as its path, and
the problem does not surface until it is moved to a different path,
which apparently happens very rarely.
Will merge to 'master'.
source: <0a0a157f88321d25fdb0be771a454b3410a449f3.camel@archlinux.org>
* ar/diff-index-merge-base-fix (2023-10-02) 1 commit
(merged to 'next' on 2023-10-06 at 0ff4dfc0e1)
+ diff: fix --merge-base with annotated tags
"git diff --merge-base X other args..." insisted that X must be a
commit and errored out when given an annotated tag that peels to a
commit, but we only need it to be a committish. This has been
corrected.
Will merge to 'master'.
source: <20231001151845.3621551-1-hi@alyssa.is>
* js/update-urls-in-doc-and-comment (2023-09-26) 4 commits
- doc: refer to internet archive
- doc: update links for andre-simon.de
- doc: update links to current pages
- doc: switch links to https
Stale URLs have been updated to their current counterparts (or
archive.org) and HTTP links are replaced with working HTTPS links.
Needs review.
source: <pull.1589.v2.git.1695553041.gitgitgadget@gmail.com>
* la/trailer-cleanups (2023-09-26) 4 commits
- trailer: only use trailer_block_* variables if trailers were found
- trailer: use offsets for trailer_start/trailer_end
- trailer: find the end of the log message
- commit: ignore_non_trailer computes number of bytes to ignore
Code clean-up.
Needs review.
source: <pull.1563.v4.git.1695709372.gitgitgadget@gmail.com>
* eb/hash-transition (2023-10-02) 30 commits
- t1016-compatObjectFormat: add tests to verify the conversion between objects
- t1006: test oid compatibility with cat-file
- t1006: rename sha1 to oid
- test-lib: compute the compatibility hash so tests may use it
- builtin/ls-tree: let the oid determine the output algorithm
- object-file: handle compat objects in check_object_signature
- tree-walk: init_tree_desc take an oid to get the hash algorithm
- builtin/cat-file: let the oid determine the output algorithm
- rev-parse: add an --output-object-format parameter
- repository: implement extensions.compatObjectFormat
- object-file: update object_info_extended to reencode objects
- object-file-convert: convert commits that embed signed tags
- object-file-convert: convert commit objects when writing
- object-file-convert: don't leak when converting tag objects
- object-file-convert: convert tag objects when writing
- object-file-convert: add a function to convert trees between algorithms
- object: factor out parse_mode out of fast-import and tree-walk into in object.h
- cache: add a function to read an OID of a specific algorithm
- tag: sign both hashes
- commit: export add_header_signature to support handling signatures on tags
- commit: convert mergetag before computing the signature of a commit
- commit: write commits for both hashes
- object-file: add a compat_oid_in parameter to write_object_file_flags
- object-file: update the loose object map when writing loose objects
- loose: compatibilty short name support
- loose: add a mapping between SHA-1 and SHA-256 for loose objects
- repository: add a compatibility hash algorithm
- object-names: support input of oids in any supported hash
- oid-array: teach oid-array to handle multiple kinds of oids
- object-file-convert: stubs for converting from one object format to another
Teach a repository to work with both SHA-1 and SHA-256 hash algorithms.
Needs review.
source: <878r8l929e.fsf@gmail.froward.int.ebiederm.org>
* jx/remote-archive-over-smart-http (2023-10-04) 4 commits
- archive: support remote archive from stateless transport
- transport-helper: call do_take_over() in connect_helper
- transport-helper: call do_take_over() in process_connect
- transport-helper: no connection restriction in connect_helper
"git archive --remote=<remote>" learned to talk over the smart
http (aka stateless) transport.
Needs review.
source: <cover.1696432593.git.zhiyou.jx@alibaba-inc.com>
* jx/sideband-chomp-newline-fix (2023-10-04) 3 commits
- pkt-line: do not chomp newlines for sideband messages
- pkt-line: memorize sideband fragment in reader
- test-pkt-line: add option parser for unpack-sideband
Sideband demultiplexer fixes.
Needs review.
source: <cover.1696425168.git.zhiyou.jx@alibaba-inc.com>
* ty/merge-tree-strategy-options (2023-10-11) 2 commits
(merged to 'next' on 2023-10-12 at 9b873601df)
+ merge: introduce {copy|clear}_merge_options()
(merged to 'next' on 2023-09-29 at aa65b54416)
+ merge-tree: add -X strategy option
"git merge-tree" learned to take strategy backend specific options
via the "-X" option, like "git merge" does.
Will merge to 'master'.
source: <pull.1565.v6.git.1695522222723.gitgitgadget@gmail.com>
* js/config-parse (2023-09-21) 5 commits
- config-parse: split library out of config.[c|h]
- config.c: accept config_parse_options in git_config_from_stdin
- config: report config parse errors using cb
- config: split do_event() into start and flush operations
- config: split out config_parse_options
The parsing routines for the configuration files have been split
into a separate file.
Needs review.
source: <cover.1695330852.git.steadmon@google.com>
* jc/fake-lstat (2023-09-15) 1 commit
- cache: add fake_lstat()
(this branch is used by jc/diff-cached-fsmonitor-fix.)
A new helper to let us pretend that we called lstat() when we know
our cache_entry is up-to-date via fsmonitor.
Needs review.
source: <xmqqcyykig1l.fsf@gitster.g>
* rs/parse-options-value-int (2023-09-18) 2 commits
- parse-options: use and require int pointer for OPT_CMDMODE
- parse-options: add int value pointer to struct option
A bit of type safety for the "value" pointer used in the
parse-options API.
Not ready yet.
cf. <4014e490-c6c1-453d-b0ed-645220e3e614@web.de>
source: <e6d8a291-03de-cfd3-3813-747fc2cad145@web.de>
* la/trailer-test-and-doc-updates (2023-09-07) 13 commits
(merged to 'next' on 2023-10-06 at 69fef35819)
+ trailer doc: <token> is a <key> or <keyAlias>, not both
+ trailer doc: separator within key suppresses default separator
+ trailer doc: emphasize the effect of configuration variables
+ trailer --unfold help: prefer "reformat" over "join"
+ trailer --parse docs: add explanation for its usefulness
+ trailer --only-input: prefer "configuration variables" over "rules"
+ trailer --parse help: expose aliased options
+ trailer --no-divider help: describe usual "---" meaning
+ trailer: trailer location is a place, not an action
+ trailer doc: narrow down scope of --where and related flags
+ trailer: add tests to check defaulting behavior with --no-* flags
+ trailer test description: this tests --where=after, not --where=before
+ trailer tests: make test cases self-contained
Test coverage for trailers has been improved.
Will merge to 'master'.
source: <pull.1564.v3.git.1694125209.gitgitgadget@gmail.com>
* js/doc-unit-tests (2023-10-09) 3 commits
- ci: run unit tests in CI
- unit tests: add TAP unit test framework
- unit tests: add a project plan document
(this branch is used by jc/doc-unit-tests-fixup and js/doc-unit-tests-with-cmake.)
Process to add some form of low-level unit tests has started.
Expecting a reroll to address CI breakage.
source: <cover.1696889529.git.steadmon@google.com>
* js/doc-unit-tests-with-cmake (2023-10-09) 7 commits
- cmake: handle also unit tests
- cmake: use test names instead of full paths
- cmake: fix typo in variable name
- artifacts-tar: when including `.dll` files, don't forget the unit-tests
- unit-tests: do show relative file paths
- unit-tests: do not mistake `.pdb` files for being executable
- cmake: also build unit tests
(this branch is used by jc/doc-unit-tests-fixup; uses js/doc-unit-tests.)
Update the base topic to work with CMake builds.
Needs review.
source: <pull.1579.v3.git.1695640836.gitgitgadget@gmail.com>
* jc/rerere-cleanup (2023-08-25) 4 commits
- rerere: modernize use of empty strbuf
- rerere: try_merge() should use LL_MERGE_ERROR when it means an error
- rerere: fix comment on handle_file() helper
- rerere: simplify check_one_conflict() helper function
Code clean-up.
Not ready to be reviewed yet.
source: <20230824205456.1231371-1-gitster@pobox.com>
* rj/status-bisect-while-rebase (2023-08-01) 1 commit
- status: fix branch shown when not only bisecting
"git status" is taught to show both the branch being bisected and
being rebased when both are in effect at the same time.
Needs review.
cf. <xmqqtttia3vn.fsf@gitster.g>
source: <48745298-f12b-8efb-4e48-90d2c22a8349@gmail.com>
--------------------------------------------------
[Discarded]
* tb/ci-coverity (2023-09-21) 1 commit
. .github/workflows: add coverity action
GitHub CI workflow has learned to trigger Coverity check.
Superseded by the js/ci-coverity topic.
source: <b23951c569660e1891a7fb3ad2c2ea1952897bd7.1695332105.git.me@ttaylorr.com>
* cw/git-std-lib (2023-09-11) 7 commits
. SQUASH???
. git-std-lib: add test file to call git-std-lib.a functions
. git-std-lib: introduce git standard library
. parse: create new library for parsing strings and env values
. config: correct bad boolean env value error message
. wrapper: remove dependency to Git-specific internal file
. hex-ll: split out functionality from hex
Another libification effort.
Superseded by the cw/prelim-cleanup topic.
cf. <xmqqy1hfrk6p.fsf@gitster.g>
cf. <20230915183927.1597414-1-jonathantanmy@google.com>
source: <20230908174134.1026823-1-calvinwan@google.com>
* so/diff-merges-d (2023-09-11) 2 commits
- diff-merges: introduce '-d' option
- diff-merges: improve --diff-merges documentation
Superseded by the so/diff-merges-dd topic.
source: <20230909125446.142715-1-sorganov@gmail.com>
* kn/rev-list-ignore-missing-links (2023-09-20) 1 commit
. revision: add `--ignore-missing-links` user option
Surface the .ignore_missing_links bit that stops the revision
traversal from stopping and dying when encountering a missing
object to a new command line option of "git rev-list", so that the
objects that are required but are missing can be enumerated.
Superseded by kn/rev-list-missing-fix
source: <20230920104507.21664-1-karthik.188@gmail.com>
^ permalink raw reply
* Re: why does git set X in LESS env var?
From: Christoph Anton Mitterer @ 2023-10-12 23:06 UTC (permalink / raw)
To: Dragan Simic; +Cc: git
In-Reply-To: <60f1922b12a6ef304ffa36c334348e34@manjaro.org>
On Fri, 2023-10-13 at 00:36 +0200, Dragan Simic wrote:
> It seems that "--redraw-on-quit" is a possible candidate for
> replacing
> "-X" in the set of default options for less(1)
*If* some changes were made to how git handles this, it might perhaps
be worth to consider not to touch LESS at all, but only add the
required settings via command line arguments (i.e. -F -R ...).
Or perhaps only remove options from it, if they're known to break the
behaviour with git (like -+R might).
I always feel configuration via env vars is a bit fragile:
- especially when one has generic names like POSIXLY_CORRECT there's
some chance that by exporting it to one program, where one wants the
effect, another program started off by that also gets it
unintentionally
- generic terms may be used by multiple programs, causing problems
Also, if one can set only one LESS var in the environment, not one for
less "alone", one for less with git, etc. - that is unless for programs
like bat/delta which have specific own env vars to set the pager.
So if I set e.g. LESS to something, than typically only to stuff from
which I believe it works as expected for any possible users.
E.g. -F might be such a case.
But if I do that, git won't touch LESS and set the required -R, so I
have to do that manually for git, e.g. either via git_config or by
defining an alias git='LESS=FRX git'.
But in both cases it would "break" again, should ever another option be
needed and added by git to the default LESS (which is however only set
when it's unset).
And in case of an alias, there would be the additional problem, that
it's typically not picked up in non-interactive shells.
Long story short, it might make sense for git, to (mostly) ignore LESS
and rather invoke less with -F -R.
The problem with that in turn would of course be that it doesn't
automatically propagate down, if e.g. git's pager is set to detla and
delta in turn runs less.
However, that's IMO litte concern, since then it's delta's duty to set
-R (if it think it needs to do so), which it actually does.
Cheers,
Chris.
^ permalink raw reply
* Re: [PATCH v2 1/2] t: add a test helper to truncate files
From: Junio C Hamano @ 2023-10-12 22:52 UTC (permalink / raw)
To: brian m. carlson; +Cc: git, Taylor Blau, Jason Hatton
In-Reply-To: <20231012160930.330618-2-sandals@crustytoothpaste.net>
"brian m. carlson" <sandals@crustytoothpaste.net> writes:
> + fd = open(argv[1], O_WRONLY | O_CREAT, 0600);
> + if (fd < 0)
> + die_errno("failed to open file %s", argv[1]);
contrib/coccinelle/xopen.cocci tells us to write this simply as
fd = xopen(argv[1], O_WRONLY | O_CREAT, 0600);
https://github.com/git/git/actions/runs/6500777388/job/17656837393#step:4:657
^ permalink raw reply
* Re: why does git set X in LESS env var?
From: Dragan Simic @ 2023-10-12 22:36 UTC (permalink / raw)
To: Christoph Anton Mitterer; +Cc: git
In-Reply-To: <c6cd3133573d5ade6d02b5da1051853a4b3885e1.camel@scientia.org>
On 2023-10-12 23:48, Christoph Anton Mitterer wrote:
> On Thu, 2023-10-12 at 23:15 +0200, Dragan Simic wrote:
>>
>> Ah, I can finally see what are you talking about... Thank you very
>> much
>> for all the testing you've performed and for supplying this screen
>> recording! I can confirm that my environment is also affected, but
>> for
>> some reason I haven't observed it this way before.
>
> Well... perhaps because it's not really "easy" to spot unless one
> carefully reads through the lines (which I guess, one does not that
> often in the terminal "history").
>
> Have a look at my ticket at less, especially:
> https://github.com/gwsw/less/issues/445#issuecomment-1758887183
Great, thanks, it's full of very useful information.
> Where it was confirmed that the issue I describe might happen (and I
> guess is non-fixable?
>
> The whole issue also contains an explanation on why scrolling doesn't
> work when -X is used (but --mouse is not) on VTE terminals (and maybe
> others, though not xterm).
> And it's basically not "fixable" but simply "by design".
It seems that "--redraw-on-quit" is a possible candidate for replacing
"-X" in the set of default options for less(1), and also in other CLI
utilities, but I still need to test it in detail.
>> Huh, that's really worrisome and I'm willing to help you with
>> debugging
>> and fixing this issue. Please, let me perform some debugging and
>> digging around, and I'll come back to you with some further insights,
>
> Well, my assumption (though I'm really not a terminal expert) would be
> that it's not fixable... because less would somehow make sure that
> everything it prints (in the alt screen buffer) is properly
> concatenated in the the regular one.
I did some work on terminal emulators, but I'm also not very much of an
expert when it comes to terminal emulators.
> less upstream made some suggestions:
> https://github.com/gwsw/less/issues/445#issuecomment-1759986293
>
> One would be to change the terminfo entry, which I guess is not really
> feasible as a general solution.
>
> The other would be less’ --redraw-on-quit option.
I agree that the latter seems like a viable solution, but as I already
wrote above, I need to test it first, together with doing some digging
through the less(1) source code.
> Maybe that would be an even better solution for git?
> AFAICS, we could have -F, the VTE mouse scrolling out of the box, plus
> (via that option) the final screen buffer of less printed when exiting.
Exactly, but still needs testing and a detailed insight.
> Would give some context (what one did in the pager, where one was) on
> the regular screen buffer, but (presumably) avoid that mess up... and
> perhaps even prevent unneeded pages of output from the pager, if one
> scrolled down a lot, which maybe aren't even needed?
Yes, it seems to tick all the boxes.
> (Also note, that less` upstream calls `-X` "a risky flag" ;-) )
In general, "-X" seems to be more of a bandaid option.
>> Those people, just as anyone else, can use $PAGER or $GIT_PAGER to
>> configure the pagination the way they like it. In the end, that's
>> also
>> what I do in my environment.
>
> Sure... all I said, that (IMO) in the case of `-X` it's not like with
> `-R` but rater like with `-S`... i.e. neither mode has more right to be
> the default than the other.
^ permalink raw reply
* Re: [PATCH v2 2/2] Prevent git from rehashing 4GiB files
From: Junio C Hamano @ 2023-10-12 22:11 UTC (permalink / raw)
To: brian m. carlson; +Cc: git, Taylor Blau, Jason Hatton
In-Reply-To: <ZShsG1SKfrefsCtu@tapette.crustytoothpaste.net>
"brian m. carlson" <sandals@crustytoothpaste.net> writes:
> nor should we support 8- or 16-bit systems. If you'd prefer "uint32_t",
> I can do that.
Going into statinfo.h and updating sd_size to uint32_t is totally
outside the scope of this fix.
> I think using 2^31 is better because it's far away from very small
> values and very large values, which means that it's a hard to modify a
> file which used to have its value as a multiple of 4 GiB and
> accidentally make Git think it was unchanged. Using 1 would make a
> simple "echo > foo" possibly think things were unchanged in some cases,
> which we should avoid.
The reason I gave the extreme "1-byte" was to gauge how much actual
"size" we are relying on the correctness of this change. As mtime
is giving the primary protection from false matching of cached stat
information, I do not think "echo >foo" would be a huge issue. IOW
my stance is 1U<<31 is as good as 1U<<0, so I do not oppose to the
former, either. But in a few years, 64-bit integers may cease to be
too odd, who knows ;-)
^ permalink raw reply
* Re: [PATCH v2 2/2] Prevent git from rehashing 4GiB files
From: brian m. carlson @ 2023-10-12 21:58 UTC (permalink / raw)
To: Junio C Hamano; +Cc: git, Taylor Blau, Jason Hatton
In-Reply-To: <xmqqpm1jn2nh.fsf@gitster.g>
[-- Attachment #1: Type: text/plain, Size: 3498 bytes --]
On 2023-10-12 at 17:58:42, Junio C Hamano wrote:
> "brian m. carlson" <sandals@crustytoothpaste.net> writes:
>
> > An example would be to have a 2^32 sized file in the index of
> > patched git. Patched git would save the file as 2^31 in the cache.
> > An unpatched git would very much see the file has changed in size
> > and force it to rehash the file, which is safe.
>
> The reason why this is "safe" is because an older Git will would
> keep rehashing whether 2^31 or 0 is stored as its sd_size, so the
> change is not making things worse? With older git, "git diff-files"
> will report that such a file is not up to date, and then the user
> will refresh the index, which will store 0 as its sd_file, so
> tentatively "git status" may give a wrong information, but we
> probalby do not care? Is that how the reasoning goes?
Correct. An older Git will rehash it either way, on every git status.
If you run git diff-files on an older Git, it will always show it as
modified (hence why I used that in the tests). The git status will
rehash it and then realize that it isn't modified, not printing any
changes, so the behaviour is not wrong, it's just extremely slow (SHA-1
DC on 4 GiB of data).
If you use a new Git with an old index, git status will still rehash it
the first time and correctly determine that it isn't changed, then write
the 0x80000000 to the index. It just won't rehash it on subsequent
calls to git status.
The only incorrect behaviour (assuming that slow is not incorrect) that
I'm aware of on older Git is that git diff-files marks it as modified
when it's not. There is no incorrect behaviour on a fixed Git.
> > +/*
> > + * Munge st_size into an unsigned int.
> > + */
> > +static unsigned int munge_st_size(off_t st_size) {
> > + unsigned int sd_size = st_size;
> > +
> > + /*
> > + * If the file is an exact multiple of 4 GiB, modify the value so it
> > + * doesn't get marked as racily clean (zero).
> > + */
> > + if (!sd_size && st_size)
> > + return 0x80000000;
> > + else
> > + return sd_size;
> > +}
>
> This assumes typeof(sd_size) aka "unsigned int" is always 32-bit,
> which does not sound reasonable. Reference to 4GiB, 2^32 and 2^31
> in the code and the proposed commit log message need to be qualified
> with "on a platform whose uint is 32-bit" or something, or better
> yet, phrased in a way that is agnostic to the integer size. At
> the very least, the hardcoded 0x80000000 needs to be rethought, I
> am afraid.
unsigned int is 32-bit on every modern 32- or 64-bit platform known to
exist today. I don't believe we support MS-DOS or systems of its era,
nor should we support 8- or 16-bit systems. If you'd prefer "uint32_t",
I can do that.
Note that the problem is reproducible on a stock Ubuntu amd64 system, so
it is definitely a problem on all modern systems.
> Other than that, the workaround for the racy-git avoidance code does
> sound good. I actually wonder if we should always use 1 regardless
> of integer size.
I think using 2^31 is better because it's far away from very small
values and very large values, which means that it's a hard to modify a
file which used to have its value as a multiple of 4 GiB and
accidentally make Git think it was unchanged. Using 1 would make a
simple "echo > foo" possibly think things were unchanged in some cases,
which we should avoid.
--
brian m. carlson (he/him or they/them)
Toronto, Ontario, CA
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 263 bytes --]
^ permalink raw reply
* Re: [PATCH 2/2] Documentation/gitformat-pack.txt: fix incorrect MIDX documentation
From: Junio C Hamano @ 2023-10-12 21:54 UTC (permalink / raw)
To: Taylor Blau; +Cc: git, Jeff King
In-Reply-To: <af2742e05dff48c4ba0a9f36d58bcbfc052dca40.1697144959.git.me@ttaylorr.com>
Taylor Blau <me@ttaylorr.com> writes:
> diff --git a/Documentation/gitformat-pack.txt b/Documentation/gitformat-pack.txt
> index d7153962d4..54000c9412 100644
> --- a/Documentation/gitformat-pack.txt
> +++ b/Documentation/gitformat-pack.txt
> @@ -392,8 +392,9 @@ CHUNK DATA:
> Packfile Names (ID: {'P', 'N', 'A', 'M'})
> Stores the packfile names as concatenated, NUL-terminated strings.
Not a problem this series (neither this or the previous step)
introduces, but I had to read the implementation of
write_midx_pack_names() to find out what "concatenated
NUL-terminated string" really means. The code has a list of
strings, writes each of them as a NUL-terminated string in sequence,
and to align the beginning of the next chunk, NULs are added to make
the whole thing multiple of MIDX_CHUNK_ALIGNMENT bytes.
A naive reader code might implement a loop like so:
while (ptr[0] != '\0') {
endp = strlen(ptr);
... ptr[0..endp] is one pathname ...
ptr = endp + 1;
}
expecting that the terminating NUL of the last entry would be
followed by a NUL, but that is buggy. The sum of the pathname
strings (with one NUL after each) may happen to be multiple of
MIDX_CHUNK_ALIGNMENT bytes, in which case no extra padding NUL bytes
will be there. So the reader also needs to pay attention to the
chunk size to notice when to stop reading. It feels somewhat
suboptimal.
> Packfiles must be listed in lexicographic order for fast lookups by
> - name. This is the only chunk not guaranteed to be a multiple of four
> - bytes in length, so should be the last chunk for alignment reasons.
> + name. Individual entries in this chunk are not guarenteed to be
> + aligned. The chunk is externally padded with zeros to align
> + remaining chunks.
I am not sure what "externally padded" means.
Before write_midx_pack_names() returns, there is a code to pad the
space after it writes those names, which does not look any external
than the bytes used to record the pathnames or NUL bytes used to
terminate these pathnames.
To me, "externally padded" is an appropriate phrase to use if this
function does not bother with the alignment after what it needs to
record, but the caller, i.e., write_chunkfile(), has code to notice
that the number of bytes written by cf->chunks[i].write_fn() it just
called is not a multiple of some required alignment and adds padding
bytes after .write_fn() returned. I do not think that is what is
going on here.
My observations.
* The packfile names chunk is used to record N pathnames; N is not
recorded anywhere explicitly. To record N pathnames, a single
chunk with N names in it is used. It is not like N chunks, each
with one name is used.
* Each of these pathname is recorded literally, followed by a NUL,
in some order, without any extra padding.
* To keep the size of the chunk multiple of alignment requirement,
there may be extra NUL bytes after the names.
This data structure does not allow you to binary search or hash
lookup without an extra table of pointers into this table of strings
at runtime, and once you build such a table, the source being sorted
does not help all that much. So I am not sure how strict the
"lexicographic" requirement needs to be, but of course, starting
strict and loosening later is much easier than starting loose, so
documenting "must be listed" and following that rule is fine.
Thanks.
^ permalink raw reply
* Re: why does git set X in LESS env var?
From: Christoph Anton Mitterer @ 2023-10-12 21:48 UTC (permalink / raw)
To: Dragan Simic; +Cc: git
In-Reply-To: <31b6f4a2b88cc3a2cfa908f82f4f2302@manjaro.org>
On Thu, 2023-10-12 at 23:15 +0200, Dragan Simic wrote:
>
> Ah, I can finally see what are you talking about... Thank you very
> much
> for all the testing you've performed and for supplying this screen
> recording! I can confirm that my environment is also affected, but
> for
> some reason I haven't observed it this way before.
Well... perhaps because it's not really "easy" to spot unless one
carefully reads through the lines (which I guess, one does not that
often in the terminal "history").
Have a look at my ticket at less, especially:
https://github.com/gwsw/less/issues/445#issuecomment-1758887183
Where it was confirmed that the issue I describe might happen (and I
guess is non-fixable?
The whole issue also contains an explanation on why scrolling doesn't
work when -X is used (but --mouse is not) on VTE terminals (and maybe
others, though not xterm).
And it's basically not "fixable" but simply "by design".
> Huh, that's really worrisome and I'm willing to help you with
> debugging
> and fixing this issue. Please, let me perform some debugging and
> digging around, and I'll come back to you with some further insights,
Well, my assumption (though I'm really not a terminal expert) would be
that it's not fixable... because less would somehow make sure that
everything it prints (in the alt screen buffer) is properly
concatenated in the the regular one.
less upstream made some suggestions:
https://github.com/gwsw/less/issues/445#issuecomment-1759986293
One would be to change the terminfo entry, which I guess is not really
feasible as a general solution.
The other would be less’ --redraw-on-quit option.
Maybe that would be an even better solution for git?
AFAICS, we could have -F, the VTE mouse scrolling out of the box, plus
(via that option) the final screen buffer of less printed when exiting.
Would give some context (what one did in the pager, where one was) on
the regular screen buffer, but (presumably) avoid that mess up... and
perhaps even prevent unneeded pages of output from the pager, if one
scrolled down a lot, which maybe aren't even needed?
(Also note, that less` upstream calls `-X` "a risky flag" ;-) )
> Those people, just as anyone else, can use $PAGER or $GIT_PAGER to
> configure the pagination the way they like it. In the end, that's
> also
> what I do in my environment.
Sure... all I said, that (IMO) in the case of `-X` it's not like with
`-R` but rater like with `-S`... i.e. neither mode has more right to be
the default than the other.
Cheers,
Chris.
^ permalink raw reply
* Re: why does git set X in LESS env var?
From: Dragan Simic @ 2023-10-12 21:15 UTC (permalink / raw)
To: Christoph Anton Mitterer; +Cc: git
In-Reply-To: <e1e187ca3d970c18e1a11d51ff93b6cb212bcbaa.camel@scientia.org>
On 2023-10-12 22:23, Christoph Anton Mitterer wrote:
> On Thu, 2023-10-12 at 07:46 +0200, Dragan Simic wrote:
>> Let me repeat that the messed up output you're experiencing isn't
>> normal
>> and has nothing to do with the arguments passed to less(1). That's a
>> separate issue of the terminal emulator(s) you're using, or in issue
>> of
>> your specific environment, and should be debugged and addressed as a
>> separate issue.
>
> As I've told you before it happens at least in gnome-terminal (and thus
> presumably and VTE based terminal), xterm, xfce4-terminal and konsole
> (all current versions of Debian unstable)... with less as of Debian
> unstable as well as 643.
>
> That affects at least on major distro, and there's a good chance that
> it affects any other distro based on Debian (*buntu, etc.).
>
> I further tried on SLES 15 with both gnome-terminal 3.42.2 and xterm
> 330 as well as less 530.
>
> Even tried with the terminal emulator started via env -i and only TERM
> set manually.
>
> *All* cases affected by the same problem I've described before.
>
> Same with the command you've used in your follow-up post, here a video
> of it in HD:
> https://youtu.be/MsxtQgrKM50
Ah, I can finally see what are you talking about... Thank you very much
for all the testing you've performed and for supplying this screen
recording! I can confirm that my environment is also affected, but for
some reason I haven't observed it this way before.
Huh, that's really worrisome and I'm willing to help you with debugging
and fixing this issue. Please, let me perform some debugging and
digging around, and I'll come back to you with some further insights,
>> To me, having inconsistent displaying of the short and long outputs
>> is simply not acceptable.
>
> Which is fine - and as I've said: I personally also tend to prefer it
> like that - but even if the above would be just some bug (which however
> seems to affect all systems I could test on a short notice, except
> yours)... one can IMO still not generally say whether on or the other
> behaviour is generally accepted to be the better one.
>
> Even if output may be just chopped of and thus ambiguously incomplete,
> some people may still prefer to have rather no output at all.
Those people, just as anyone else, can use $PAGER or $GIT_PAGER to
configure the pagination the way they like it. In the end, that's also
what I do in my environment.
>> Perhaps something is wrong with your specific environment, because
>> I see no other reason for this issue.
>
> Well may be, but seems unlikely from my PoV, given that I've now tested
> even on other distros and systems not under my control.
>
> Anyway... I think this got a bit too off-topic here :-D
Well, yes and no. This scrolling-related issue is obviously affecting
numerous git users, which makes it quite relevant for git as a project.
Of course, not directly relevant, but indirectly, yes.
^ permalink raw reply
* [PATCH 2/2] Documentation/gitformat-pack.txt: fix incorrect MIDX documentation
From: Taylor Blau @ 2023-10-12 21:09 UTC (permalink / raw)
To: git; +Cc: Junio C Hamano, Jeff King
In-Reply-To: <cover.1697144959.git.me@ttaylorr.com>
Back in 32f3c541e3 (multi-pack-index: write pack names in chunk, 2018-07-12)
the MIDX's "Packfile Names" (or "PNAM", for short) chunk was described
as containing an array of string entries. e0d1bcf825 notes that this is
the only chunk in the MIDX format's specification that is not guaranteed
to be 4-byte aligned, and so should be placed last.
This isn't quite accurate: the entries within the PNAM chunk are not
guaranteed to be aligned since they are arbitrary strings, but the
chunk itself is aligned since the ending is padded with NUL bytes.
That external padding has always been there since 32f3c541e3 via
midx.c::write_midx_pack_names(), which ended with:
i = MIDX_CHUNK_ALIGNMENT - (written % MIDX_CHUNK_ALIGNMENT)
if (i < MIDX_CHUNK_ALIGNMENT) {
unsigned char padding[MIDX_CHUNK_ALIGNMENT];
memset(padding, 0, sizeof(padding))
hashwrite(f, padding, i);
written += i;
}
In fact, 32f3c541e3's log message itself describes the chunk in its
first paragraph with:
Since filenames are not well structured, add padding to keep good
alignment in later chunks.
So these have always been externally aligned. Correct the corresponding
part of our documentation to reflect that.
Signed-off-by: Taylor Blau <me@ttaylorr.com>
---
Documentation/gitformat-pack.txt | 5 +++--
1 file changed, 3 insertions(+), 2 deletions(-)
diff --git a/Documentation/gitformat-pack.txt b/Documentation/gitformat-pack.txt
index d7153962d4..54000c9412 100644
--- a/Documentation/gitformat-pack.txt
+++ b/Documentation/gitformat-pack.txt
@@ -392,8 +392,9 @@ CHUNK DATA:
Packfile Names (ID: {'P', 'N', 'A', 'M'})
Stores the packfile names as concatenated, NUL-terminated strings.
Packfiles must be listed in lexicographic order for fast lookups by
- name. This is the only chunk not guaranteed to be a multiple of four
- bytes in length, so should be the last chunk for alignment reasons.
+ name. Individual entries in this chunk are not guarenteed to be
+ aligned. The chunk is externally padded with zeros to align
+ remaining chunks.
OID Fanout (ID: {'O', 'I', 'D', 'F'})
The ith entry, F[i], stores the number of OIDs with first
--
2.42.0.349.gf0c1128f8b.dirty
^ permalink raw reply related
* [PATCH 1/2] Documentation/gitformat-pack.txt: fix typo
From: Taylor Blau @ 2023-10-12 21:09 UTC (permalink / raw)
To: git; +Cc: Junio C Hamano, Jeff King
In-Reply-To: <cover.1697144959.git.me@ttaylorr.com>
e0d1bcf825 (multi-pack-index: add format details, 2018-07-12) describes
the MIDX's "PNAM" chunk as having entries which are "null-terminated
strings".
This is a typo, as strings are terminated with a NUL character, which is
a distinct concept from "NULL" or "null", which we typically reserve for
the void pointer to address 0.
Correct the documentation accordingly.
Signed-off-by: Taylor Blau <me@ttaylorr.com>
---
Documentation/gitformat-pack.txt | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/Documentation/gitformat-pack.txt b/Documentation/gitformat-pack.txt
index 870e00f298..d7153962d4 100644
--- a/Documentation/gitformat-pack.txt
+++ b/Documentation/gitformat-pack.txt
@@ -390,7 +390,7 @@ CHUNK LOOKUP:
CHUNK DATA:
Packfile Names (ID: {'P', 'N', 'A', 'M'})
- Stores the packfile names as concatenated, null-terminated strings.
+ Stores the packfile names as concatenated, NUL-terminated strings.
Packfiles must be listed in lexicographic order for fast lookups by
name. This is the only chunk not guaranteed to be a multiple of four
bytes in length, so should be the last chunk for alignment reasons.
--
2.42.0.349.gf0c1128f8b.dirty
^ permalink raw reply related
* [PATCH 0/2] Documentation/gitformat-pack.txt: correct a few issues/typos
From: Taylor Blau @ 2023-10-12 21:09 UTC (permalink / raw)
To: git; +Cc: Junio C Hamano, Jeff King
These short couple of patches are some fallout from a larger series[^1]
that I'm working on. I noticed a couple of minor issues in the MIDX
format's documentation, and figured that sending these patches sooner
rather than later would be worthwhile.
The first patch swaps "null" for "NUL" when describing the NUL
character. The second patch corrects a piece of the documentation which
claims the PNAM chunk is not aligned (it is externally padded and thus
guaranteed to be well-aligned).
Thanks in advance for your review.
[^1]: The broader goal of that series is to allow verbatim pack reuse
from multiple packs (instead of just the bitmapped one, or the
preferred pack of a MIDX). This is only possible in certain
circumstances, which require some tweaks to the MIDX format for
bookkeeping reasons, which is how I noticed these issues in the first
place.
Taylor Blau (2):
Documentation/gitformat-pack.txt: fix typo
Documentation/gitformat-pack.txt: fix incorrect MIDX documentation
Documentation/gitformat-pack.txt | 7 ++++---
1 file changed, 4 insertions(+), 3 deletions(-)
--
2.42.0.349.gf0c1128f8b.dirty
^ permalink raw reply
* Re: [PATCH] send-email: add --compose-cover option
From: Kristoffer Haugsbakk @ 2023-10-12 20:48 UTC (permalink / raw)
To: Junio C Hamano; +Cc: Ben Dooks, git
In-Reply-To: <xmqqlec7mvl1.fsf@gitster.g>
On Thu, Oct 12, 2023, at 22:31, Junio C Hamano wrote:
> Hmph, I am not sure why a format-patch user would bother with the
> "--compose-cover" option for the cover letter, to be honest. I am a
> format-patch user and never drive it from send-email, just like the
> workflow you use, and the norm to me is to review everything, not
> just the cover letter but also the patches, in my editor session.
> You apparently do not need "--edit-patches" option to review and
> adjust as needed before sending, even though that is what you do
> already. Why do you need a "--compose-cover" option?
The thought was to get an interactive editor session for the subject and
the message and let `format-patch` fill in the range-diff etc., but on
second thought I have to do so many edits (fix mistakes and use the cover
letter in later versions) that an interactive session would probably not
be useful for me, after all. If so I would probably need to use some tool
that makes cover-letter a first-class entity that you can edit multiple
times similar to a commit message, like `git-series` (I'm guessing that's
how it works?).
The upcoming `--description-file`[1] is probably what I'm gonna end up
using.
[1] https://lore.kernel.org/git/20230821170720.577820-1-oswald.buddenhagen@gmx.de/
--
Kristoffer Haugsbakk
^ permalink raw reply
* Re: This list is being migrated to new infrastructure (no action required)
From: Junio C Hamano @ 2023-10-12 20:46 UTC (permalink / raw)
To: Konstantin Ryabitsev; +Cc: git
In-Reply-To: <20231012-magnifier-decipher-4493ec@meerkat>
Konstantin Ryabitsev <konstantin@linuxfoundation.org> writes:
> On Thu, Oct 12, 2023 at 04:30:54PM -0400, Konstantin Ryabitsev wrote:
>> This list is being migrated to the new vger infrastructure. This should be a
>> fully transparent process and you don't need to change anything about how you
>> participate with the list or how you receive mail.
>>
>> There will be a brief delay with archives on lore.kernel.org. I will follow up
>> once the archive migration has been completed.
>
> This work is completed now. If you think something isn't working right, please
> report it to helpdesk@kernel.org.
Thanks!
^ permalink raw reply
* Re: -q option for git to suppress informational messages?
From: Emily Shaffer @ 2023-10-12 20:46 UTC (permalink / raw)
To: Michal Suchánek; +Cc: git
In-Reply-To: <20231012185308.GB6241@kitsune.suse.cz>
On Thu, Oct 12, 2023 at 11:53 AM Michal Suchánek <msuchanek@suse.de> wrote:
>
> Hello,
>
> when using git in scripts I find that many git commands insist on
> printing informational messages, and the only way to avoid this is to
> sent the output to /dev/null.
Michal, you might want to investigate further whether there is a
plumbing-specific command that meets the needs you're concerned about
instead. Those commands will not have additional human-facing output,
and have a stronger guarantee around backwards-compatibility than the
human-facing commands do.
Commands which count as "plumbing" can be located in `git help git` in
the full list of subcommands; if you're not sure what alternative to
use, I think you can feel free to describe what you're trying to do
here and get advice on which plumbing commands to use instead of
porcelain ones.
>
> While some select commands support the -q option to suppres
> informational messages many don't.
>
> Since there was recenly some discussion of problems that newcomers could
> work on I suppose this could be added to the list if not there already.
>
> Thanks
>
> Michal
^ permalink raw reply
* Re: This list is being migrated to new infrastructure (no action required)
From: Konstantin Ryabitsev @ 2023-10-12 20:40 UTC (permalink / raw)
To: git
In-Reply-To: <20231012-party-legwarmer-551ab7@meerkat>
On Thu, Oct 12, 2023 at 04:30:54PM -0400, Konstantin Ryabitsev wrote:
> This list is being migrated to the new vger infrastructure. This should be a
> fully transparent process and you don't need to change anything about how you
> participate with the list or how you receive mail.
>
> There will be a brief delay with archives on lore.kernel.org. I will follow up
> once the archive migration has been completed.
This work is completed now. If you think something isn't working right, please
report it to helpdesk@kernel.org.
-K
^ permalink raw reply
* Re: [PATCH] send-email: add --compose-cover option
From: Junio C Hamano @ 2023-10-12 20:31 UTC (permalink / raw)
To: Kristoffer Haugsbakk; +Cc: Ben Dooks, git
In-Reply-To: <277a80c5-40c9-4f1e-a68f-96673380012b@app.fastmail.com>
"Kristoffer Haugsbakk" <code@khaugsbakk.name> writes:
> 2. This would be really useful in `format-patch`. One could write the
> cover letter immediately instead of either (1) editing the generated
> one (placeholders) or (2) providing a file.
> - Personally I don't use `send-email` immediately—I use `format-patch`
> to generate everything so that I can review it before any
> sending. But there are many different workflows.
Hmph, I am not sure why a format-patch user would bother with the
"--compose-cover" option for the cover letter, to be honest. I am a
format-patch user and never drive it from send-email, just like the
workflow you use, and the norm to me is to review everything, not
just the cover letter but also the patches, in my editor session.
You apparently do not need "--edit-patches" option to review and
adjust as needed before sending, even though that is what you do
already. Why do you need a "--compose-cover" option?
^ permalink raw reply
* This list is being migrated to new infrastructure (no action required)
From: Konstantin Ryabitsev @ 2023-10-12 20:30 UTC (permalink / raw)
To: git
Hello, all:
This list is being migrated to the new vger infrastructure. This should be a
fully transparent process and you don't need to change anything about how you
participate with the list or how you receive mail.
There will be a brief delay with archives on lore.kernel.org. I will follow up
once the archive migration has been completed.
Best regards,
Konstantin
^ permalink raw reply
* Re: why does git set X in LESS env var?
From: Christoph Anton Mitterer @ 2023-10-12 20:23 UTC (permalink / raw)
To: Dragan Simic; +Cc: git
In-Reply-To: <161b9584c6c9a004c01bda98cea4f1f8@manjaro.org>
On Thu, 2023-10-12 at 07:46 +0200, Dragan Simic wrote:
> Let me repeat that the messed up output you're experiencing isn't
> normal
> and has nothing to do with the arguments passed to less(1). That's a
> separate issue of the terminal emulator(s) you're using, or in issue
> of
> your specific environment, and should be debugged and addressed as a
> separate issue.
Be it as it may...
As I've told you before it happens at least in gnome-terminal (and thus
presumably and VTE based terminal), xterm, xfce4-terminal and konsole
(all current versions of Debian unstable)... with less as of Debian
unstable as well as 643.
That affects at least on major distro, and there's a good chance that
it affects any other distro based on Debian (*buntu, etc.).
I further tried on SLES 15 with both gnome-terminal 3.42.2 and xterm
330 as well as less 530.
Even tried with the terminal emulator started via env -i and only TERM
set manually.
*All* cases affected by the same problem I've described before.
Same with the command you've used in your follow-up post, here a video
of it in HD:
https://youtu.be/MsxtQgrKM50
> To me, having inconsistent displaying of the short and long outputs
> is
> simply not acceptable.
Which is fine - and as I've said: I personally also tend to prefer it
like that - but even if the above would be just some bug (which however
seems to affect all systems I could test on a short notice, except
yours)... one can IMO still not generally say whether on or the other
behaviour is generally accepted to be the better one.
Even if output may be just chopped of and thus ambiguously incomplete,
some people may still prefer to have rather no output at all.
And in fact:
This is the default mode of less alone.
> >
> Perhaps something is wrong with your specific environment, because I
> see
> no other reason for this issue.
Well may be, but seems unlikely from my PoV, given that I've now tested
even on other distros and systems not under my control.
Anyway... I think this got a bit too off-topic here :-D
Cheers,
Chris.
^ permalink raw reply
* [PATCH] Add x64 SHA-NI implementation to sha1 and sha256 in git ?
From: Feiyang Xue @ 2023-10-12 20:04 UTC (permalink / raw)
To: git; +Cc: Feiyang Xue
Looking for comments on if this is a good idea to add SHA-NI option to git,
And if so, what'd be a good implementation.
The attached patch is more of a proof of concept, using a sha-ni example
found on internet and have it tapped into git's "hash-ll.h" when it sees make
flags "SHA1_SHANI=1" and/or "SHA256_SHANI=1" for sha1/sha256 respectively.
"git hash-object" is about 3-ish times faster for me
There are few topics that i'm uncertain about.
1. Is it good idea to have machine-specific asm codes in git. I read there was
commit fd1ec82547 which removed assembly implementation for PPC_SHA1. Does that
imply maintainers doesn't want machine-specific assembly in source?
2. If it is cool to have machine-specific codes in git, what's a good logic for
their on/off? Build time flag? Run time flag? Run time automatic detection to
corresponding instructions?
3. If i can include machine-specific codes, what'd be a good way/idea to cleanup
the asm? I suppose git maintainers do not want someone else's license header?
Regards,
Feiyang
---
Makefile | 28 ++
hash-ll.h | 8 +-
.../intel_sha_extensions_sha1_assembly.asm | 308 +++++++++++++++
sha1-x64shani/sha1_x64.c | 71 ++++
sha1-x64shani/sha1_x64.h | 21 +
.../intel_sha_extensions_sha256_assembly.asm | 371 ++++++++++++++++++
sha256-x64shani/sha256_x64.c | 82 ++++
sha256-x64shani/sha256_x64.h | 23 ++
8 files changed, 910 insertions(+), 2 deletions(-)
create mode 100644 sha1-x64shani/intel_sha_extensions_sha1_assembly.asm
create mode 100644 sha1-x64shani/sha1_x64.c
create mode 100644 sha1-x64shani/sha1_x64.h
create mode 100644 sha256-x64shani/intel_sha_extensions_sha256_assembly.asm
create mode 100644 sha256-x64shani/sha256_x64.c
create mode 100644 sha256-x64shani/sha256_x64.h
diff --git a/Makefile b/Makefile
index e440728c24..6892af2ee5 100644
--- a/Makefile
+++ b/Makefile
@@ -1917,6 +1917,11 @@ ifdef PPC_SHA1
$(error the PPC_SHA1 flag has been removed along with the PowerPC-specific SHA-1 implementation.)
endif
+ifdef SHA1_SHANI
+ LIB_OBJS += sha1-x64shani/sha1_x64.o
+ LIB_OBJS += sha1-x64shani/intel_sha_extensions_sha1_assembly.o
+ BASIC_CFLAGS += -DSHA1_SHANI
+else
ifdef OPENSSL_SHA1
EXTLIBS += $(LIB_4_CRYPTO)
BASIC_CFLAGS += -DSHA1_OPENSSL
@@ -1957,7 +1962,13 @@ endif
endif
endif
endif
+endif
+ifdef SHA256_SHANI
+ LIB_OBJS += sha256-x64shani/sha256_x64.o
+ LIB_OBJS += sha256-x64shani/intel_sha_extensions_sha256_assembly.o
+ BASIC_CFLAGS += -DSHA256_SHANI
+else
ifdef OPENSSL_SHA256
EXTLIBS += $(LIB_4_CRYPTO)
BASIC_CFLAGS += -DSHA256_OPENSSL
@@ -1975,6 +1986,9 @@ else
endif
endif
endif
+endif
+
+$(info $$LIB_OBJS is [${LIB_OBJS}])
ifdef SHA1_MAX_BLOCK_SIZE
LIB_OBJS += compat/sha1-chunked.o
@@ -2711,6 +2725,20 @@ missing_compdb_dir =
compdb_args =
endif
+$(info $$OBJECTS is [${OBJECTS}])
+
+### for SHA-NI
+YASM_SRC := $(wildcard $(OBJECTS:o=asm))
+YASM_OBJ := $(YASM_SRC:asm=o)
+OBJECTS := $(filter-out $(YASM_OBJ),$(OBJECTS))
+
+$(info $$YASM_SRC is [${YASM_SRC}])
+$(info $$YASM_OBJ is [${YASM_OBJ}])
+$(info $$OBJECTS is [${OBJECTS}])
+
+$(YASM_OBJ): %.o: %.asm $(missing_dep_dirs)
+ $(QUIET_CC)yasm $< -f elf64 -X gnu -g dwarf2 -o $@
+
$(OBJECTS): %.o: %.c GIT-CFLAGS $(missing_dep_dirs) $(missing_compdb_dir)
$(QUIET_CC)$(CC) -o $*.o -c $(dep_args) $(compdb_args) $(ALL_CFLAGS) $(EXTRA_CPPFLAGS) $<
diff --git a/hash-ll.h b/hash-ll.h
index 8050925137..31d9e91f26 100644
--- a/hash-ll.h
+++ b/hash-ll.h
@@ -1,7 +1,9 @@
#ifndef HASH_LL_H
#define HASH_LL_H
-#if defined(SHA1_APPLE)
+#if defined(SHA1_SHANI)
+#include "sha1-x64shani/sha1_x64.h"
+#elif defined(SHA1_APPLE)
#include <CommonCrypto/CommonDigest.h>
#elif defined(SHA1_OPENSSL)
#include <openssl/sha.h>
@@ -11,7 +13,9 @@
#include "block-sha1/sha1.h"
#endif
-#if defined(SHA256_NETTLE)
+#if defined(SHA256_SHANI)
+#include "sha256-x64shani/sha256_x64.h"
+#elif defined(SHA256_NETTLE)
#include "sha256/nettle.h"
#elif defined(SHA256_GCRYPT)
#define SHA256_NEEDS_CLONE_HELPER
diff --git a/sha1-x64shani/intel_sha_extensions_sha1_assembly.asm b/sha1-x64shani/intel_sha_extensions_sha1_assembly.asm
new file mode 100644
index 0000000000..ba59825966
--- /dev/null
+++ b/sha1-x64shani/intel_sha_extensions_sha1_assembly.asm
@@ -0,0 +1,308 @@
+;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
+; Copyright (c) 2013, Intel Corporation
+;
+; All rights reserved.
+;
+; Redistribution and use in source and binary forms, with or without
+; modification, are permitted provided that the following conditions are
+; met:
+;
+; * Redistributions of source code must retain the above copyright
+; notice, this list of conditions and the following disclaimer.
+;
+; * Redistributions in binary form must reproduce the above copyright
+; notice, this list of conditions and the following disclaimer in the
+; documentation and/or other materials provided with the
+; distribution.
+;
+; * Neither the name of the Intel Corporation nor the names of its
+; contributors may be used to endorse or promote products derived from
+; this software without specific prior written permission.
+;
+;
+; THIS SOFTWARE IS PROVIDED BY INTEL CORPORATION ""AS IS"" AND ANY
+; EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+; IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
+; PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL INTEL CORPORATION OR
+; CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
+; EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
+; PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
+; PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF
+; LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING
+; NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
+; SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
+;
+; Intel SHA Extensions optimized implementation of a SHA-1 update function
+;
+; The function takes a pointer to the current hash values, a pointer to the
+; input data, and a number of 64 byte blocks to process. Once all blocks have
+; been processed, the digest pointer is updated with the resulting hash value.
+; The function only processes complete blocks, there is no functionality to
+; store partial blocks. All message padding and hash value initialization must
+; be done outside the update function.
+;
+; The indented lines in the loop are instructions related to rounds processing.
+; The non-indented lines are instructions related to the message schedule.
+;
+; Author: Sean Gulley <sean.m.gulley@intel.com>
+; Date: July 2013
+;
+;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
+;
+; Example YASM command lines:
+; Windows:
+; yasm -Xvc -f x64 -rnasm -pnasm -DWINABI -o intel_sha_extensions_sha1_assembly.obj -g cv8 intel_sha_extensions_sha1_assembly.asm
+; Linux:
+; yasm -f elf64 -X gnu -g dwarf2 -o intel_sha_extensions_sha1_assembly.o intel_sha_extensions_sha1_assembly.asm
+;
+;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
+
+%ifdef WINABI
+%define DIGEST_PTR rcx ; 1st arg
+%define DATA_PTR rdx ; 2nd arg
+%define NUM_BLKS r8 ; 3rd arg
+%else
+%define DIGEST_PTR rdi ; 1st arg
+%define DATA_PTR rsi ; 2nd arg
+%define NUM_BLKS rdx ; 3rd arg
+%endif
+
+%define RSPSAVE rax
+
+struc frame
+%ifdef WINABI
+.xmm_save: resdq 2
+%endif
+.hash_save: resdq 2
+endstruc
+
+%define ABCD xmm0
+%define E0 xmm1 ; Need two E's b/c they ping pong
+%define E1 xmm2
+%define MSG0 xmm3
+%define MSG1 xmm4
+%define MSG2 xmm5
+%define MSG3 xmm6
+%define SHUF_MASK xmm7
+
+;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
+;; void sha1_update(uint32_t *digest, const void *data, uint32_t numBlocks);
+;; arg 1 : pointer to digest
+;; arg 2 : pointer to input data
+;; arg 3 : Number of blocks to process
+;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
+section .text
+global sha1_update:function
+align 32
+sha1_update:
+ mov RSPSAVE, rsp
+ sub rsp, frame_size
+ and rsp, ~0xF
+
+%ifdef WINABI
+ movdqa [rsp + frame.xmm_save + 0*16], xmm6
+ movdqa [rsp + frame.xmm_save + 1*16], xmm7
+%endif
+
+ shl NUM_BLKS, 6 ; convert to bytes
+ jz .done_hash
+ add NUM_BLKS, DATA_PTR ; pointer to end of data
+
+ ;; load initial hash values
+ pinsrd E0, [DIGEST_PTR + 1*16], 3
+ movdqu ABCD, [DIGEST_PTR + 0*16]
+ pand E0, [UPPER_WORD_MASK wrt rip]
+ pshufd ABCD, ABCD, 0x1B
+
+ movdqa SHUF_MASK, [PSHUFFLE_BYTE_FLIP_MASK wrt rip]
+
+.loop0:
+ ;; Save hash values for addition after rounds
+ movdqa [rsp + frame.hash_save + 0*16], E0
+ movdqa [rsp + frame.hash_save + 1*16], ABCD
+
+ ;; Rounds 0-3
+ movdqu MSG0, [DATA_PTR + 0*16]
+ pshufb MSG0, SHUF_MASK
+ paddd E0, MSG0
+ movdqa E1, ABCD
+ sha1rnds4 ABCD, E0, 0
+
+ ;; Rounds 4-7
+ movdqu MSG1, [DATA_PTR + 1*16]
+ pshufb MSG1, SHUF_MASK
+ sha1nexte E1, MSG1
+ movdqa E0, ABCD
+ sha1rnds4 ABCD, E1, 0
+ sha1msg1 MSG0, MSG1
+
+ ;; Rounds 8-11
+ movdqu MSG2, [DATA_PTR + 2*16]
+ pshufb MSG2, SHUF_MASK
+ sha1nexte E0, MSG2
+ movdqa E1, ABCD
+ sha1rnds4 ABCD, E0, 0
+ sha1msg1 MSG1, MSG2
+ pxor MSG0, MSG2
+
+ ;; Rounds 12-15
+ movdqu MSG3, [DATA_PTR + 3*16]
+ pshufb MSG3, SHUF_MASK
+ sha1nexte E1, MSG3
+ movdqa E0, ABCD
+ sha1msg2 MSG0, MSG3
+ sha1rnds4 ABCD, E1, 0
+ sha1msg1 MSG2, MSG3
+ pxor MSG1, MSG3
+
+ ;; Rounds 16-19
+ sha1nexte E0, MSG0
+ movdqa E1, ABCD
+ sha1msg2 MSG1, MSG0
+ sha1rnds4 ABCD, E0, 0
+ sha1msg1 MSG3, MSG0
+ pxor MSG2, MSG0
+
+ ;; Rounds 20-23
+ sha1nexte E1, MSG1
+ movdqa E0, ABCD
+ sha1msg2 MSG2, MSG1
+ sha1rnds4 ABCD, E1, 1
+ sha1msg1 MSG0, MSG1
+ pxor MSG3, MSG1
+
+ ;; Rounds 24-27
+ sha1nexte E0, MSG2
+ movdqa E1, ABCD
+ sha1msg2 MSG3, MSG2
+ sha1rnds4 ABCD, E0, 1
+ sha1msg1 MSG1, MSG2
+ pxor MSG0, MSG2
+
+ ;; Rounds 28-31
+ sha1nexte E1, MSG3
+ movdqa E0, ABCD
+ sha1msg2 MSG0, MSG3
+ sha1rnds4 ABCD, E1, 1
+ sha1msg1 MSG2, MSG3
+ pxor MSG1, MSG3
+
+ ;; Rounds 32-35
+ sha1nexte E0, MSG0
+ movdqa E1, ABCD
+ sha1msg2 MSG1, MSG0
+ sha1rnds4 ABCD, E0, 1
+ sha1msg1 MSG3, MSG0
+ pxor MSG2, MSG0
+
+ ;; Rounds 36-39
+ sha1nexte E1, MSG1
+ movdqa E0, ABCD
+ sha1msg2 MSG2, MSG1
+ sha1rnds4 ABCD, E1, 1
+ sha1msg1 MSG0, MSG1
+ pxor MSG3, MSG1
+
+ ;; Rounds 40-43
+ sha1nexte E0, MSG2
+ movdqa E1, ABCD
+ sha1msg2 MSG3, MSG2
+ sha1rnds4 ABCD, E0, 2
+ sha1msg1 MSG1, MSG2
+ pxor MSG0, MSG2
+
+ ;; Rounds 44-47
+ sha1nexte E1, MSG3
+ movdqa E0, ABCD
+ sha1msg2 MSG0, MSG3
+ sha1rnds4 ABCD, E1, 2
+ sha1msg1 MSG2, MSG3
+ pxor MSG1, MSG3
+
+ ;; Rounds 48-51
+ sha1nexte E0, MSG0
+ movdqa E1, ABCD
+ sha1msg2 MSG1, MSG0
+ sha1rnds4 ABCD, E0, 2
+ sha1msg1 MSG3, MSG0
+ pxor MSG2, MSG0
+
+ ;; Rounds 52-55
+ sha1nexte E1, MSG1
+ movdqa E0, ABCD
+ sha1msg2 MSG2, MSG1
+ sha1rnds4 ABCD, E1, 2
+ sha1msg1 MSG0, MSG1
+ pxor MSG3, MSG1
+
+ ;; Rounds 56-59
+ sha1nexte E0, MSG2
+ movdqa E1, ABCD
+ sha1msg2 MSG3, MSG2
+ sha1rnds4 ABCD, E0, 2
+ sha1msg1 MSG1, MSG2
+ pxor MSG0, MSG2
+
+ ;; Rounds 60-63
+ sha1nexte E1, MSG3
+ movdqa E0, ABCD
+ sha1msg2 MSG0, MSG3
+ sha1rnds4 ABCD, E1, 3
+ sha1msg1 MSG2, MSG3
+ pxor MSG1, MSG3
+
+ ;; Rounds 64-67
+ sha1nexte E0, MSG0
+ movdqa E1, ABCD
+ sha1msg2 MSG1, MSG0
+ sha1rnds4 ABCD, E0, 3
+ sha1msg1 MSG3, MSG0
+ pxor MSG2, MSG0
+
+ ;; Rounds 68-71
+ sha1nexte E1, MSG1
+ movdqa E0, ABCD
+ sha1msg2 MSG2, MSG1
+ sha1rnds4 ABCD, E1, 3
+ pxor MSG3, MSG1
+
+ ;; Rounds 72-75
+ sha1nexte E0, MSG2
+ movdqa E1, ABCD
+ sha1msg2 MSG3, MSG2
+ sha1rnds4 ABCD, E0, 3
+
+ ;; Rounds 76-79
+ sha1nexte E1, MSG3
+ movdqa E0, ABCD
+ sha1rnds4 ABCD, E1, 3
+
+ ;; Add current hash values with previously saved
+ sha1nexte E0, [rsp + frame.hash_save + 0*16]
+ paddd ABCD, [rsp + frame.hash_save + 1*16]
+
+ ;; Increment data pointer and loop if more to process
+ add DATA_PTR, 64
+ cmp DATA_PTR, NUM_BLKS
+ jne .loop0
+
+ ;; Write hash values back in the correct order
+ pshufd ABCD, ABCD, 0x1B
+ movdqu [DIGEST_PTR + 0*16], ABCD
+ pextrd [DIGEST_PTR + 1*16], E0, 3
+
+.done_hash:
+%ifdef WINABI
+ movdqa xmm6, [rsp + frame.xmm_save + 0*16]
+ movdqa xmm7, [rsp + frame.xmm_save + 1*16]
+%endif
+ mov rsp, RSPSAVE
+
+ ret
+
+section .data
+align 64
+PSHUFFLE_BYTE_FLIP_MASK: ddq 0x000102030405060708090a0b0c0d0e0f
+UPPER_WORD_MASK: ddq 0xFFFFFFFF000000000000000000000000
+
diff --git a/sha1-x64shani/sha1_x64.c b/sha1-x64shani/sha1_x64.c
new file mode 100644
index 0000000000..0ed0d49ed6
--- /dev/null
+++ b/sha1-x64shani/sha1_x64.c
@@ -0,0 +1,71 @@
+#include "../git-compat-util.h"
+#include "sha1_x64.h"
+#include <byteswap.h>
+
+// static inline void shani_SHA1_Init(shani_SHA_CTX *c);
+
+void shani_SHA1_Init(shani_SHA_CTX *c)
+{
+ static const uint32_t sha1InitialDigest[5] = {
+ 0x67452301, 0xefcdab89, 0x98badcfe, 0x10325476, 0xc3d2e1f0
+ };
+ memcpy(c->h, sha1InitialDigest, 20);
+ c->len = 0;
+ c->total_len = 0;
+}
+
+void sha1_update(void *c, const void *d, int l);
+void shani_SHA1_Update(shani_SHA_CTX *c, const void *data, unsigned long len)
+{
+ while (len > 0)
+ {
+ if (c->len == 0 && len >= 64)
+ {
+ sha1_update(c->h, data, len/64);
+ unsigned long bytes_hashed = (len/64)*64;
+ data += bytes_hashed;
+ len -= bytes_hashed;
+ c->total_len += bytes_hashed;
+ continue;
+ }
+#define MIN(x, y) (((x) < (y)) ? (x) : (y))
+ size_t cpy = MIN(64 - c->len, len);
+#undef MIN
+ memcpy(c->msgbuf + c->len, data, cpy);
+ data += cpy;
+ len -= cpy;
+ c->len += cpy;
+ c->total_len += cpy;
+
+ if (c->len == 64)
+ {
+ sha1_update(c->h, c->msgbuf, 1);
+ c->len = 0;
+ }
+ }
+}
+
+void shani_SHA1_Final(unsigned char *md, shani_SHA_CTX *c)
+{
+ c->msgbuf[c->len] = 0x80;
+ ++c->len;
+ memset(c->msgbuf + c->len, 0, 64 - c->len);
+ if (c->len > (64-8))
+ {
+ sha1_update(c->h, c->msgbuf, 1);
+ memset(c->msgbuf, 0, 64);
+ c->len = 0;
+ }
+
+ for (int ibyte = 0; ibyte < 8; ++ibyte)
+ c->msgbuf[63-ibyte] = (uint8_t)((c->total_len * 8ULL) >> (ibyte*8));
+
+ sha1_update(c->h, c->msgbuf, 1);
+ c->len = 0;
+
+ // flip endianness
+ int i;
+ for (i = 0; i < 20; i += 4) {
+ *(uint32_t *)(md + i) = __bswap_32(c->h[i / 4]);
+ }
+}
diff --git a/sha1-x64shani/sha1_x64.h b/sha1-x64shani/sha1_x64.h
new file mode 100644
index 0000000000..ca1ec116cf
--- /dev/null
+++ b/sha1-x64shani/sha1_x64.h
@@ -0,0 +1,21 @@
+/*
+ * SHA1 routine using x64 SHA-NI instructions
+ */
+
+typedef struct shani_SHA_CTX
+{
+ uint32_t h[5];
+ int len;
+ uint64_t total_len;
+ char msgbuf[64];
+} shani_SHA_CTX;
+
+
+void shani_SHA1_Init(shani_SHA_CTX *ctx);
+void shani_SHA1_Update(shani_SHA_CTX *ctx, const void *dataIn, size_t len);
+void shani_SHA1_Final(unsigned char hashout[20], shani_SHA_CTX *ctx);
+
+#define platform_SHA_CTX shani_SHA_CTX
+#define platform_SHA1_Init shani_SHA1_Init
+#define platform_SHA1_Update shani_SHA1_Update
+#define platform_SHA1_Final shani_SHA1_Final
diff --git a/sha256-x64shani/intel_sha_extensions_sha256_assembly.asm b/sha256-x64shani/intel_sha_extensions_sha256_assembly.asm
new file mode 100644
index 0000000000..b587e82154
--- /dev/null
+++ b/sha256-x64shani/intel_sha_extensions_sha256_assembly.asm
@@ -0,0 +1,371 @@
+;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
+; Copyright (c) 2013, Intel Corporation
+;
+; All rights reserved.
+;
+; Redistribution and use in source and binary forms, with or without
+; modification, are permitted provided that the following conditions are
+; met:
+;
+; * Redistributions of source code must retain the above copyright
+; notice, this list of conditions and the following disclaimer.
+;
+; * Redistributions in binary form must reproduce the above copyright
+; notice, this list of conditions and the following disclaimer in the
+; documentation and/or other materials provided with the
+; distribution.
+;
+; * Neither the name of the Intel Corporation nor the names of its
+; contributors may be used to endorse or promote products derived from
+; this software without specific prior written permission.
+;
+;
+; THIS SOFTWARE IS PROVIDED BY INTEL CORPORATION ""AS IS"" AND ANY
+; EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+; IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
+; PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL INTEL CORPORATION OR
+; CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
+; EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
+; PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
+; PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF
+; LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING
+; NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
+; SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
+;
+; Intel SHA Extensions optimized implementation of a SHA-256 update function
+;
+; The function takes a pointer to the current hash values, a pointer to the
+; input data, and a number of 64 byte blocks to process. Once all blocks have
+; been processed, the digest pointer is updated with the resulting hash value.
+; The function only processes complete blocks, there is no functionality to
+; store partial blocks. All message padding and hash value initialization must
+; be done outside the update function.
+;
+; The indented lines in the loop are instructions related to rounds processing.
+; The non-indented lines are instructions related to the message schedule.
+;
+; Author: Sean Gulley <sean.m.gulley@intel.com>
+; Date: July 2013
+;
+;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
+;
+; Example YASM command lines:
+; Windows: yasm -Xvc -f x64 -rnasm -pnasm -DWINABI -o intel_sha_extensions_sha256_assembly.obj -g cv8 intel_sha_extensions_sha256_assembly.asm
+; Linux: yasm -f elf64 -X gnu -g dwarf2 -o intel_sha_extensions_sha256_assembly.o intel_sha_extensions_sha256_assembly.asm
+;
+;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
+
+%ifdef WINABI
+%define DIGEST_PTR rcx ; 1st arg
+%define DATA_PTR rdx ; 2nd arg
+%define NUM_BLKS r8 ; 3rd arg
+%else
+%define DIGEST_PTR rdi ; 1st arg
+%define DATA_PTR rsi ; 2nd arg
+%define NUM_BLKS rdx ; 3rd arg
+%endif
+
+%define SHA256CONSTANTS rax
+
+%ifdef WINABI
+%define RSPSAVE r9
+
+struc frame
+.xmm_save: resdq 5
+endstruc
+%endif
+
+%define MSG xmm0
+%define STATE0 xmm1
+%define STATE1 xmm2
+%define MSGTMP0 xmm3
+%define MSGTMP1 xmm4
+%define MSGTMP2 xmm5
+%define MSGTMP3 xmm6
+%define MSGTMP4 xmm7
+
+%define SHUF_MASK xmm8
+
+%define ABEF_SAVE xmm9
+%define CDGH_SAVE xmm10
+
+;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
+;; void sha256_update(uint32_t *digest, const void *data, uint32_t numBlocks);
+;; arg 1 : pointer to digest
+;; arg 2 : pointer to input data
+;; arg 3 : Number of blocks to process
+;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
+section .text
+global sha256_update:function
+align 32
+sha256_update:
+%ifdef WINABI
+ mov RSPSAVE, rsp
+ sub rsp, frame_size
+ and rsp, ~0xF
+
+ movdqa [rsp + frame.xmm_save + 0*16], xmm6
+ movdqa [rsp + frame.xmm_save + 1*16], xmm7
+ movdqa [rsp + frame.xmm_save + 2*16], xmm8
+ movdqa [rsp + frame.xmm_save + 3*16], xmm9
+ movdqa [rsp + frame.xmm_save + 4*16], xmm10
+%endif
+
+ shl NUM_BLKS, 6 ; convert to bytes
+ jz .done_hash
+ add NUM_BLKS, DATA_PTR ; pointer to end of data
+
+ ;; load initial hash values
+ ;; Need to reorder these appropriately
+ ;; DCBA, HGFE -> ABEF, CDGH
+ movdqu STATE0, [DIGEST_PTR + 0*16]
+ movdqu STATE1, [DIGEST_PTR + 1*16]
+
+ pshufd STATE0, STATE0, 0xB1 ; CDAB
+ pshufd STATE1, STATE1, 0x1B ; EFGH
+ movdqa MSGTMP4, STATE0
+ palignr STATE0, STATE1, 8 ; ABEF
+ pblendw STATE1, MSGTMP4, 0xF0 ; CDGH
+
+ movdqa SHUF_MASK, [PSHUFFLE_BYTE_FLIP_MASK wrt rip]
+ lea SHA256CONSTANTS,[K256 wrt rip]
+
+.loop0:
+ ;; Save hash values for addition after rounds
+ movdqa ABEF_SAVE, STATE0
+ movdqa CDGH_SAVE, STATE1
+
+ ;; Rounds 0-3
+ movdqu MSG, [DATA_PTR + 0*16]
+ pshufb MSG, SHUF_MASK
+ movdqa MSGTMP0, MSG
+ paddd MSG, [SHA256CONSTANTS + 0*16]
+ sha256rnds2 STATE1, STATE0
+ pshufd MSG, MSG, 0x0E
+ sha256rnds2 STATE0, STATE1
+
+ ;; Rounds 4-7
+ movdqu MSG, [DATA_PTR + 1*16]
+ pshufb MSG, SHUF_MASK
+ movdqa MSGTMP1, MSG
+ paddd MSG, [SHA256CONSTANTS + 1*16]
+ sha256rnds2 STATE1, STATE0
+ pshufd MSG, MSG, 0x0E
+ sha256rnds2 STATE0, STATE1
+ sha256msg1 MSGTMP0, MSGTMP1
+
+ ;; Rounds 8-11
+ movdqu MSG, [DATA_PTR + 2*16]
+ pshufb MSG, SHUF_MASK
+ movdqa MSGTMP2, MSG
+ paddd MSG, [SHA256CONSTANTS + 2*16]
+ sha256rnds2 STATE1, STATE0
+ pshufd MSG, MSG, 0x0E
+ sha256rnds2 STATE0, STATE1
+ sha256msg1 MSGTMP1, MSGTMP2
+
+ ;; Rounds 12-15
+ movdqu MSG, [DATA_PTR + 3*16]
+ pshufb MSG, SHUF_MASK
+ movdqa MSGTMP3, MSG
+ paddd MSG, [SHA256CONSTANTS + 3*16]
+ sha256rnds2 STATE1, STATE0
+ movdqa MSGTMP4, MSGTMP3
+ palignr MSGTMP4, MSGTMP2, 4
+ paddd MSGTMP0, MSGTMP4
+ sha256msg2 MSGTMP0, MSGTMP3
+ pshufd MSG, MSG, 0x0E
+ sha256rnds2 STATE0, STATE1
+ sha256msg1 MSGTMP2, MSGTMP3
+
+ ;; Rounds 16-19
+ movdqa MSG, MSGTMP0
+ paddd MSG, [SHA256CONSTANTS + 4*16]
+ sha256rnds2 STATE1, STATE0
+ movdqa MSGTMP4, MSGTMP0
+ palignr MSGTMP4, MSGTMP3, 4
+ paddd MSGTMP1, MSGTMP4
+ sha256msg2 MSGTMP1, MSGTMP0
+ pshufd MSG, MSG, 0x0E
+ sha256rnds2 STATE0, STATE1
+ sha256msg1 MSGTMP3, MSGTMP0
+
+ ;; Rounds 20-23
+ movdqa MSG, MSGTMP1
+ paddd MSG, [SHA256CONSTANTS + 5*16]
+ sha256rnds2 STATE1, STATE0
+ movdqa MSGTMP4, MSGTMP1
+ palignr MSGTMP4, MSGTMP0, 4
+ paddd MSGTMP2, MSGTMP4
+ sha256msg2 MSGTMP2, MSGTMP1
+ pshufd MSG, MSG, 0x0E
+ sha256rnds2 STATE0, STATE1
+ sha256msg1 MSGTMP0, MSGTMP1
+
+ ;; Rounds 24-27
+ movdqa MSG, MSGTMP2
+ paddd MSG, [SHA256CONSTANTS + 6*16]
+ sha256rnds2 STATE1, STATE0
+ movdqa MSGTMP4, MSGTMP2
+ palignr MSGTMP4, MSGTMP1, 4
+ paddd MSGTMP3, MSGTMP4
+ sha256msg2 MSGTMP3, MSGTMP2
+ pshufd MSG, MSG, 0x0E
+ sha256rnds2 STATE0, STATE1
+ sha256msg1 MSGTMP1, MSGTMP2
+
+ ;; Rounds 28-31
+ movdqa MSG, MSGTMP3
+ paddd MSG, [SHA256CONSTANTS + 7*16]
+ sha256rnds2 STATE1, STATE0
+ movdqa MSGTMP4, MSGTMP3
+ palignr MSGTMP4, MSGTMP2, 4
+ paddd MSGTMP0, MSGTMP4
+ sha256msg2 MSGTMP0, MSGTMP3
+ pshufd MSG, MSG, 0x0E
+ sha256rnds2 STATE0, STATE1
+ sha256msg1 MSGTMP2, MSGTMP3
+
+ ;; Rounds 32-35
+ movdqa MSG, MSGTMP0
+ paddd MSG, [SHA256CONSTANTS + 8*16]
+ sha256rnds2 STATE1, STATE0
+ movdqa MSGTMP4, MSGTMP0
+ palignr MSGTMP4, MSGTMP3, 4
+ paddd MSGTMP1, MSGTMP4
+ sha256msg2 MSGTMP1, MSGTMP0
+ pshufd MSG, MSG, 0x0E
+ sha256rnds2 STATE0, STATE1
+ sha256msg1 MSGTMP3, MSGTMP0
+
+ ;; Rounds 36-39
+ movdqa MSG, MSGTMP1
+ paddd MSG, [SHA256CONSTANTS + 9*16]
+ sha256rnds2 STATE1, STATE0
+ movdqa MSGTMP4, MSGTMP1
+ palignr MSGTMP4, MSGTMP0, 4
+ paddd MSGTMP2, MSGTMP4
+ sha256msg2 MSGTMP2, MSGTMP1
+ pshufd MSG, MSG, 0x0E
+ sha256rnds2 STATE0, STATE1
+ sha256msg1 MSGTMP0, MSGTMP1
+
+ ;; Rounds 40-43
+ movdqa MSG, MSGTMP2
+ paddd MSG, [SHA256CONSTANTS + 10*16]
+ sha256rnds2 STATE1, STATE0
+ movdqa MSGTMP4, MSGTMP2
+ palignr MSGTMP4, MSGTMP1, 4
+ paddd MSGTMP3, MSGTMP4
+ sha256msg2 MSGTMP3, MSGTMP2
+ pshufd MSG, MSG, 0x0E
+ sha256rnds2 STATE0, STATE1
+ sha256msg1 MSGTMP1, MSGTMP2
+
+ ;; Rounds 44-47
+ movdqa MSG, MSGTMP3
+ paddd MSG, [SHA256CONSTANTS + 11*16]
+ sha256rnds2 STATE1, STATE0
+ movdqa MSGTMP4, MSGTMP3
+ palignr MSGTMP4, MSGTMP2, 4
+ paddd MSGTMP0, MSGTMP4
+ sha256msg2 MSGTMP0, MSGTMP3
+ pshufd MSG, MSG, 0x0E
+ sha256rnds2 STATE0, STATE1
+ sha256msg1 MSGTMP2, MSGTMP3
+
+ ;; Rounds 48-51
+ movdqa MSG, MSGTMP0
+ paddd MSG, [SHA256CONSTANTS + 12*16]
+ sha256rnds2 STATE1, STATE0
+ movdqa MSGTMP4, MSGTMP0
+ palignr MSGTMP4, MSGTMP3, 4
+ paddd MSGTMP1, MSGTMP4
+ sha256msg2 MSGTMP1, MSGTMP0
+ pshufd MSG, MSG, 0x0E
+ sha256rnds2 STATE0, STATE1
+ sha256msg1 MSGTMP3, MSGTMP0
+
+ ;; Rounds 52-55
+ movdqa MSG, MSGTMP1
+ paddd MSG, [SHA256CONSTANTS + 13*16]
+ sha256rnds2 STATE1, STATE0
+ movdqa MSGTMP4, MSGTMP1
+ palignr MSGTMP4, MSGTMP0, 4
+ paddd MSGTMP2, MSGTMP4
+ sha256msg2 MSGTMP2, MSGTMP1
+ pshufd MSG, MSG, 0x0E
+ sha256rnds2 STATE0, STATE1
+
+ ;; Rounds 56-59
+ movdqa MSG, MSGTMP2
+ paddd MSG, [SHA256CONSTANTS + 14*16]
+ sha256rnds2 STATE1, STATE0
+ movdqa MSGTMP4, MSGTMP2
+ palignr MSGTMP4, MSGTMP1, 4
+ paddd MSGTMP3, MSGTMP4
+ sha256msg2 MSGTMP3, MSGTMP2
+ pshufd MSG, MSG, 0x0E
+ sha256rnds2 STATE0, STATE1
+
+ ;; Rounds 60-63
+ movdqa MSG, MSGTMP3
+ paddd MSG, [SHA256CONSTANTS + 15*16]
+ sha256rnds2 STATE1, STATE0
+ pshufd MSG, MSG, 0x0E
+ sha256rnds2 STATE0, STATE1
+
+ ;; Add current hash values with previously saved
+ paddd STATE0, ABEF_SAVE
+ paddd STATE1, CDGH_SAVE
+
+ ;; Increment data pointer and loop if more to process
+ add DATA_PTR, 64
+ cmp DATA_PTR, NUM_BLKS
+ jne .loop0
+
+ ;; Write hash values back in the correct order
+ pshufd STATE0, STATE0, 0x1B ; FEBA
+ pshufd STATE1, STATE1, 0xB1 ; DCHG
+ movdqa MSGTMP4, STATE0
+ pblendw STATE0, STATE1, 0xF0 ; DCBA
+ palignr STATE1, MSGTMP4, 8 ; HGFE
+
+ movdqu [DIGEST_PTR + 0*16], STATE0
+ movdqu [DIGEST_PTR + 1*16], STATE1
+
+.done_hash:
+%ifdef WINABI
+ movdqa xmm6, [rsp + frame.xmm_save + 0*16]
+ movdqa xmm7, [rsp + frame.xmm_save + 1*16]
+ movdqa xmm8, [rsp + frame.xmm_save + 2*16]
+ movdqa xmm9, [rsp + frame.xmm_save + 3*16]
+ movdqa xmm10, [rsp + frame.xmm_save + 4*16]
+ mov rsp, RSPSAVE
+%endif
+
+ ret
+
+section .data
+align 64
+K256:
+ dd 0x428a2f98,0x71374491,0xb5c0fbcf,0xe9b5dba5
+ dd 0x3956c25b,0x59f111f1,0x923f82a4,0xab1c5ed5
+ dd 0xd807aa98,0x12835b01,0x243185be,0x550c7dc3
+ dd 0x72be5d74,0x80deb1fe,0x9bdc06a7,0xc19bf174
+ dd 0xe49b69c1,0xefbe4786,0x0fc19dc6,0x240ca1cc
+ dd 0x2de92c6f,0x4a7484aa,0x5cb0a9dc,0x76f988da
+ dd 0x983e5152,0xa831c66d,0xb00327c8,0xbf597fc7
+ dd 0xc6e00bf3,0xd5a79147,0x06ca6351,0x14292967
+ dd 0x27b70a85,0x2e1b2138,0x4d2c6dfc,0x53380d13
+ dd 0x650a7354,0x766a0abb,0x81c2c92e,0x92722c85
+ dd 0xa2bfe8a1,0xa81a664b,0xc24b8b70,0xc76c51a3
+ dd 0xd192e819,0xd6990624,0xf40e3585,0x106aa070
+ dd 0x19a4c116,0x1e376c08,0x2748774c,0x34b0bcb5
+ dd 0x391c0cb3,0x4ed8aa4a,0x5b9cca4f,0x682e6ff3
+ dd 0x748f82ee,0x78a5636f,0x84c87814,0x8cc70208
+ dd 0x90befffa,0xa4506ceb,0xbef9a3f7,0xc67178f2
+
+PSHUFFLE_BYTE_FLIP_MASK: ddq 0x0c0d0e0f08090a0b0405060700010203
+
diff --git a/sha256-x64shani/sha256_x64.c b/sha256-x64shani/sha256_x64.c
new file mode 100644
index 0000000000..01f3417719
--- /dev/null
+++ b/sha256-x64shani/sha256_x64.c
@@ -0,0 +1,82 @@
+#include "../git-compat-util.h"
+#include "sha256_x64.h"
+#include <byteswap.h>
+
+static int shani = -1;
+
+void shani_SHA256_Init(shani_SHA256_CTX *ctx)
+{
+ ctx->total_len = 0;
+ ctx->len = 0;
+ ctx->h[0] = 0x6a09e667ul;
+ ctx->h[1] = 0xbb67ae85ul;
+ ctx->h[2] = 0x3c6ef372ul;
+ ctx->h[3] = 0xa54ff53aul;
+ ctx->h[4] = 0x510e527ful;
+ ctx->h[5] = 0x9b05688cul;
+ ctx->h[6] = 0x1f83d9abul;
+ ctx->h[7] = 0x5be0cd19ul;
+
+ if (shani != 1) {
+ shani = 1;
+ printf("set shani to 1\n");
+ } else {
+ printf("shani is 1 already\n");
+ }
+}
+
+void sha256_update(uint32_t *digest, const void *data, uint32_t numBlocks);
+void shani_SHA256_Update(shani_SHA256_CTX *c, const void *data, size_t len)
+{
+ while (len > 0)
+ {
+ if (c->len == 0 && len >= 64)
+ {
+ sha256_update(c->h, data, len/64);
+ unsigned long bytes_hashed = (len/64)*64;
+ data += bytes_hashed;
+ len -= bytes_hashed;
+ c->total_len += bytes_hashed;
+ continue;
+ }
+#define MIN(x, y) (((x) < (y)) ? (x) : (y))
+ size_t cpy = MIN(64 - c->len, len);
+#undef MIN
+ memcpy(c->msgbuf + c->len, data, cpy);
+ data += cpy;
+ len -= cpy;
+ c->len += cpy;
+ c->total_len += cpy;
+
+ if (c->len == 64)
+ {
+ sha256_update(c->h, c->msgbuf, 1);
+ c->len = 0;
+ }
+ }
+}
+
+void shani_SHA256_Final(unsigned char *digest, shani_SHA256_CTX *c)
+{
+ c->msgbuf[c->len] = 0x80;
+ ++c->len;
+ memset(c->msgbuf + c->len, 0, 64 - c->len);
+ if (c->len > (64-8))
+ {
+ sha256_update(c->h, c->msgbuf, 1);
+ memset(c->msgbuf, 0, 64);
+ c->len = 0;
+ }
+
+ for (int ibyte = 0; ibyte < 8; ++ibyte)
+ c->msgbuf[63-ibyte] = (uint8_t)((c->total_len * 8ULL) >> (ibyte*8));
+
+ sha256_update(c->h, c->msgbuf, 1);
+ c->len = 0;
+
+ // flip endianness
+ int i;
+ for (i = 0; i < 32; i += 4) {
+ *(uint32_t *)(digest + i) = __bswap_32(c->h[i / 4]);
+ }
+}
\ No newline at end of file
diff --git a/sha256-x64shani/sha256_x64.h b/sha256-x64shani/sha256_x64.h
new file mode 100644
index 0000000000..25f1ff67ba
--- /dev/null
+++ b/sha256-x64shani/sha256_x64.h
@@ -0,0 +1,23 @@
+/*
+ * SHA256 routine using x64 SHA-NI instructions
+ */
+
+#define SHANI_SHA256_BLKSIZE 64
+
+struct shani_SHA256_CTX {
+ uint32_t h[8];
+ int len;
+ uint64_t total_len;
+ char msgbuf[SHANI_SHA256_BLKSIZE];
+};
+
+typedef struct shani_SHA256_CTX shani_SHA256_CTX;
+
+void shani_SHA256_Init(shani_SHA256_CTX *ctx);
+void shani_SHA256_Update(shani_SHA256_CTX *c, const void *data, size_t len);
+void shani_SHA256_Final(unsigned char *digest, shani_SHA256_CTX *c);
+
+#define platform_SHA256_CTX shani_SHA256_CTX
+#define platform_SHA256_Init shani_SHA256_Init
+#define platform_SHA256_Update shani_SHA256_Update
+#define platform_SHA256_Final shani_SHA256_Final
\ No newline at end of file
--
2.41.0
^ permalink raw reply related
* Re: [PATCH] send-email: add --compose-cover option
From: Kristoffer Haugsbakk @ 2023-10-12 20:03 UTC (permalink / raw)
To: Ben Dooks; +Cc: git, Junio C Hamano
In-Reply-To: <20231012112743.2756259-1-ben.dooks@codethink.co.uk>
Hi Ben
On Thu, Oct 12, 2023, at 13:27, Ben Dooks wrote:
> Adding an option to automatically compose a cover letter would be
> helpful to put the whole process of sending an series with a cover
> into one command.
I didn't manage to apply the patch so I haven't tried it out. Some
questions:
1. There is already `--compose` which is clearly different. What's the
difference between a cover letter an an introduction? That's a more
immediate question now with two similar-looking options.
2. This would be really useful in `format-patch`. One could write the
cover letter immediately instead of either (1) editing the generated
one (placeholders) or (2) providing a file.
- Personally I don't use `send-email` immediately—I use `format-patch`
to generate everything so that I can review it before any
sending. But there are many different workflows.
3. Does this editor only deal with the subject and the message part before
the diffstat and all the auto-generated stuff? And then it applies it
after the user closes the editor (to make the complete cover letter)?
--
Kristoffer Haugsbakk
^ permalink raw reply
* -q option for git to suppress informational messages?
From: Michal Suchánek @ 2023-10-12 18:53 UTC (permalink / raw)
To: git
Hello,
when using git in scripts I find that many git commands insist on
printing informational messages, and the only way to avoid this is to
sent the output to /dev/null.
While some select commands support the -q option to suppres
informational messages many don't.
Since there was recenly some discussion of problems that newcomers could
work on I suppose this could be added to the list if not there already.
Thanks
Michal
^ permalink raw reply
* git and multiple processes
From: Federico Kircheis @ 2023-10-12 18:28 UTC (permalink / raw)
To: git
Hello,
I wanted to know if concurrent command can "break" a repository, ie put
it in an inconsistent state.
I suppose that as long all commands are only querying the status (log,
reflog, status, ....) then such a thing will not happen.
I guess that if only one command changes the status (add, commit, pull,
fetch, push, ...) then the output might be "wrong", but the command that
queries the status does not interfere with the others.
But what if two commands that change the status?
For example a git fetch and git add or git commit at the same time?
A use-case I had recently was me doing "git commit -p" from one console,
and noticing a change I wanted to undo.
I skipped the change, and while "git commit -p" showed me the next diff,
I opened another console.
From there, I did a "git checkout", and reverted the change I just skipped.
Is such use-case supported?
Everything worked as I expected, but it would be nice to know that I did
not "risk" anything.
The use-cases I am most interested in is when working with worktrees.
What if I add a worktree, and "work concurrently" in two worktrees?
For example while git is switching branches in one worktree, doing a
commit in another worktree (assuming I'm not switching to the branch
where I am committing)
What if I am doing an automated "git bisect" in a worktree.
Can I work in another one (doing commits, fetch, rebase, reflog, stash,
...)?
What if I have a cron job that executes "git fetch" in one worktree.
Is it problematic if I happen to execute git fetch at the same time from
another worktree?
If those use-cases are not problematic, as all repositories are linked
together, what are possible issues (ignoring bugs).
Common sense tells me that those operations are supported, but I could
not find any reference, or if there are some guarantees that git wants
to hold, no matter what the current status is.
Best
Federico
^ permalink raw reply
* Re: [PATCH v2 2/2] Prevent git from rehashing 4GiB files
From: Junio C Hamano @ 2023-10-12 17:58 UTC (permalink / raw)
To: brian m. carlson; +Cc: git, Taylor Blau, Jason Hatton
In-Reply-To: <20231012160930.330618-3-sandals@crustytoothpaste.net>
"brian m. carlson" <sandals@crustytoothpaste.net> writes:
> An example would be to have a 2^32 sized file in the index of
> patched git. Patched git would save the file as 2^31 in the cache.
> An unpatched git would very much see the file has changed in size
> and force it to rehash the file, which is safe.
The reason why this is "safe" is because an older Git will would
keep rehashing whether 2^31 or 0 is stored as its sd_size, so the
change is not making things worse? With older git, "git diff-files"
will report that such a file is not up to date, and then the user
will refresh the index, which will store 0 as its sd_file, so
tentatively "git status" may give a wrong information, but we
probalby do not care? Is that how the reasoning goes?
> +/*
> + * Munge st_size into an unsigned int.
> + */
> +static unsigned int munge_st_size(off_t st_size) {
> + unsigned int sd_size = st_size;
> +
> + /*
> + * If the file is an exact multiple of 4 GiB, modify the value so it
> + * doesn't get marked as racily clean (zero).
> + */
> + if (!sd_size && st_size)
> + return 0x80000000;
> + else
> + return sd_size;
> +}
This assumes typeof(sd_size) aka "unsigned int" is always 32-bit,
which does not sound reasonable. Reference to 4GiB, 2^32 and 2^31
in the code and the proposed commit log message need to be qualified
with "on a platform whose uint is 32-bit" or something, or better
yet, phrased in a way that is agnostic to the integer size. At
the very least, the hardcoded 0x80000000 needs to be rethought, I
am afraid.
Other than that, the workaround for the racy-git avoidance code does
sound good. I actually wonder if we should always use 1 regardless
of integer size.
^ permalink raw reply
* Re: [PATCH v2 1/2] t: add a test helper to truncate files
From: Eric Sunshine @ 2023-10-12 17:49 UTC (permalink / raw)
To: brian m. carlson; +Cc: git, Junio C Hamano, Taylor Blau, Jason Hatton
In-Reply-To: <20231012160930.330618-2-sandals@crustytoothpaste.net>
On Thu, Oct 12, 2023 at 12:10 PM brian m. carlson
<sandals@crustytoothpaste.net> wrote:
> In a future commit, we're going to work with some large files which will
> be at least 4 GiB in size. To take advantage of the sparseness
> functionality on most Unix systems and avoid running the system out of
> disk, it would be convenient to use truncate(2) to simply create a
> sparse file of sufficient size.
>
> However, the GNU truncate(1) utility isn't portable, so let's write a
> tiny test helper that does the work for us.
>
> Signed-off-by: brian m. carlson <bk2204@github.com>
> ---
> diff --git a/t/helper/test-truncate.c b/t/helper/test-truncate.c
> @@ -0,0 +1,27 @@
> +int cmd__truncate(int argc, const char **argv)
> +{
> + char *p = NULL;
> + uintmax_t sz = 0;
> + int fd = -1;
> +
> + if (argc != 3)
> + die("expected filename and size");
> +
> + sz = strtoumax(argv[2], &p, 0);
> + if (*p)
> + die("invalid size");
Do you want to check 'errno' here, as well (probably before the '*p' check)?
Or is that being too defensive for a 'test-tool' command?
> + fd = open(argv[1], O_WRONLY | O_CREAT, 0600);
> + if (fd < 0)
> + die_errno("failed to open file %s", argv[1]);
> +
> + if (ftruncate(fd, (off_t) sz) < 0)
> + die_errno("failed to truncate file");
> + return 0;
> +}
^ permalink raw reply
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox