What's cooking in git.git (Oct 2013, #07; Mon, 28)

What's cooking in git.git (Oct 2013, #07; Mon, 28)

[Junio C Hamano]

Here are the topics that have been cooking.  Commits prefixed with
'-' are only in 'pu' (proposed updates) while commits prefixed with
'+' are in 'next'.

It is already 10th week of this cycle, but somehow I completely
forgot where in the cycle we were.  Sorry about that.

I'll tag 1.8.5-rc0 in a few days by the end of this month, and then
hopefully we will have two to three -rc weeks after that, aiming for
the final 1.8.5 release sometime late November (tentative schedule
at http://tinyurl.com/gitCal).

As promised/requested, the final steps for 2.0 are in 'next'; they,
together with a handful topics that have been merged to 'next'
fairly recently, will _not_ be part of the upcoming 1.8.5 release,
but will be carried over in 'next' to the next cycle.

Also there is 1.8.4.2 maintenance release out.

You can find the changes described here in the integration branches
of the repositories listed at

    http://git-blame.blogspot.com/p/git-public-repositories.html

--------------------------------------------------
[Graduated to "master"]

* ew/keepalive (2013-10-16) 2 commits
  (merged to 'next' on 2013-10-16 at 56fd9f3)
 + http: use curl's tcp keepalive if available
  (merged to 'next' on 2013-10-14 at 24d786f)
 + http: enable keepalive on TCP sockets

 The HTTP transport will try to use TCP keepalive when able.


* jc/revision-range-unpeel (2013-10-15) 1 commit
  (merged to 'next' on 2013-10-16 at d04ddfe)
 + revision: do not peel tags used in range notation

 "git rev-list --objects ^v1.0^ v1.0" gave v1.0 tag itself in the
 output, but "git rev-list --objects v1.0^..v1.0" did not.


* jk/remote-literal-string-leakfix (2013-10-15) 1 commit
  (merged to 'next' on 2013-10-18 at 6abddac)
 + remote: do not copy "origin" string literal


* jk/split-broken-ident (2013-10-15) 1 commit
  (merged to 'next' on 2013-10-18 at 8f4b8b7)
 + split_ident: parse timestamp from end of line

 Make the fall-back parsing of commit objects with broken author or
 committer lines more robust to pick up the timestamps.


* jx/relative-path-regression-fix (2013-10-14) 3 commits
  (merged to 'next' on 2013-10-18 at b4af45f)
 + Use simpler relative_path when set_git_dir
  (merged to 'next' on 2013-10-14 at 704b9ee)
 + relative_path should honor dos-drive-prefix
 + test: use unambigous leading path (/foo) for MSYS

 Will merge to 'master' and later to 'maint'.


* sb/repack-in-c (2013-10-22) 1 commit
  (merged to 'next' on 2013-10-23 at 5d7ac72)
 + Reword repack documentation to no longer state it's a script

 Finishing touches to update documentation.


* sg/prompt-svn-remote-fix (2013-10-15) 1 commit
  (merged to 'next' on 2013-10-18 at 20b47eb)
 + bash prompt: don't use '+=' operator in show upstream code path

 Bash portability fix.

--------------------------------------------------
[New Topics]

* bw/solaris-sed-tr-test-portability (2013-10-28) 2 commits
 - Avoid difference in tr semantics between System V and BSD
 - Change sed i\ usage to something Solaris' sed can handle

 Needs a bit of reroll.


* fc/transport-helper-fixes (2013-10-28) 13 commits
 - test: remote-helper: add test for force pushes
 - git-remote-testgit: support the new 'force' option
 - fixup! transport-helper: add 'force' to 'export' helpers
 - transport-helper: don't update refs in dry-run
 - transport-helper: add support to delete branches
 - fast-export: add support to delete refs
 - fast-import: add support to delete refs
 - transport-helper: add support for old:new refspec
 - fast-export: add new --refspec option
 - fast-export: improve argument parsing
 - transport-helper: check for 'forced update' message
 - transport-helper: fix extra lines
 - transport-helper: add 'force' to 'export' helpers



* jh/loose-object-dirs-creation-race (2013-10-28) 1 commit
 - sha1_file.c:create_tmpfile(): Fix race when creating loose object dirs


* js/test-help-format-windows-port-fix (2013-10-28) 1 commit
 - PATCH] t3200: do not open a HTML manual page when DEFAULT_MAN_FORMAT is html

 Will merge to 'next' after amending the title.


* js/tests-windows-port-fix (2013-10-28) 3 commits
 - tests: undo special treatment of CRLF for Windows
 - Windows: a test_cmp that is agnostic to random LF <> CRLF conversions
 - t5300-pack-object: do not compare binary data using test_cmp

 Will merge to 'next'.


* nd/liteal-pathspecs (2013-10-28) 1 commit
 - pathspec: stop --*-pathspecs impact on internal parse_pathspec() uses


* rs/web-browse-xdg-open (2013-10-28) 1 commit
 - web--browse: Add support for xdg-open.

 Will merge to 'next'.


* sb/refs-code-cleanup (2013-10-28) 2 commits
 - cache: remove unused function 'have_git_dir'
 - refs: remove unused function invalidate_ref_cache

 Will merge to 'next'.


* th/reflog-annotated-tag (2013-10-28) 1 commit
 - reflog: handle lightweight and annotated tags equally

--------------------------------------------------
[Stalled]

* np/pack-v4 (2013-09-18) 90 commits
 . packv4-parse.c: add tree offset caching
 . t1050: replace one instance of show-index with verify-pack
 . index-pack, pack-objects: allow creating .idx v2 with .pack v4
 . unpack-objects: decode v4 trees
 . unpack-objects: allow to save processed bytes to a buffer
 - ...

 Nico and Duy advancing the eternal vaporware pack-v4.  This is here
 primarily for wider distribution of the preview edition.

 Temporarily ejected from 'pu', to try out jk/pack-bitmap, which
 this topic conflicts with.


* sc/doc-howto-dumb-http (2013-10-16) 1 commit
 . doc/howto: warn about (dumb)http server document being too old

 The new text needs to go somewhere in the body of the document,
 not before the title line.


* tg/perf-lib-test-perf-cleanup (2013-09-19) 2 commits
 - perf-lib: add test_perf_cleanup target
 - perf-lib: split starting the test from the execution

 Add test_perf_cleanup shell function to the perf suite, that allows
 the script writers to define a test with a clean-up action.

 Holding until needed.


* yt/shortened-rename (2013-10-18) 2 commits
 - SQUASH??? style fixes and s/omit/shorten/ where appropriate
 - diff.c: keep arrow(=>) on show_stats()'s shortened filename part to make rename visible

 Attempts to give more weight on the fact that a filepair represents
 a rename than showing substring of the actual path when diffstat
 lines are not wide enough.

 I am not sure if that is solving a right problem, though.


* jc/ref-excludes (2013-09-03) 2 commits
 - document --exclude option
 - revision: introduce --exclude=<glob> to tame wildcards

 People often wished a way to tell "git log --branches" (and "git
 log --remotes --not --branches") to exclude some local branches
 from the expansion of "--branches" (similarly for "--tags", "--all"
 and "--glob=<pattern>").  Now they have one.

 Needs a matching change to rev-parse.


* rv/send-email-cache-generated-mid (2013-08-21) 2 commits
 - git-send-email: Cache generated message-ids, use them when prompting
 - git-send-email: add optional 'choices' parameter to the ask sub


* rj/read-default-config-in-show-ref-pack-refs (2013-06-17) 3 commits
 - ### DONTMERGE: needs better explanation on what config they need
 - pack-refs.c: Add missing call to git_config()
 - show-ref.c: Add missing call to git_config()

 The changes themselves are probably good, but it is unclear what
 basic setting needs to be read for which exact operation.

 Waiting for clarification.
 $gmane/228294


* jc/format-patch (2013-04-22) 2 commits
 - format-patch: --inline-single
 - format-patch: rename "no_inline" field

 A new option to send a single patch to the standard output to be
 appended at the bottom of a message.  I personally have no need for
 this, but it was easy enough to cobble together.  Tests, docs and
 stripping out more MIMEy stuff are left as exercises to interested
 parties.


* jk/gitweb-utf8 (2013-04-08) 4 commits
 - gitweb: Fix broken blob action parameters on blob/commitdiff pages
 - gitweb: Don't append ';js=(0|1)' to external links
 - gitweb: Make feed title valid utf8
 - gitweb: Fix utf8 encoding for blob_plain, blobdiff_plain, commitdiff_plain, and patch

 Various fixes to gitweb.

 Drew Northup volunteered to take a look into this.
 $gmane/226216


* jc/show-branch (2013-06-07) 5 commits
 - show-branch: use commit slab to represent bitflags of arbitrary width
 - show-branch.c: remove "all_mask"
 - show-branch.c: abstract out "flags" operation
 - show-branch.c: lift all_mask/all_revs to a global static
 - show-branch.c: update comment style

 Waiting for the final step to lift the hard-limit before sending it out.

--------------------------------------------------
[Cooking]

* tr/merge-recursive-index-only (2013-10-28) 3 commits
 - merge-recursive: -Xindex-only to leave worktree unchanged
 - merge-recursive: internal flag to avoid touching the worktree
 - merge-recursive: remove dead conditional in update_stages()

 Holding until there is a caller to learn from.


* bc/http-100-continue (2013-10-28) 1 commit
 - remote-curl: fix large pushes with GSSAPI

 Conditionally allow "100 Continue" responses to help use of
 GSS-Negotiate authentication scheme over HTTP transport.

 Rerolled. Is everybody happy with this version without
 configuration?


* jc/merge-base-reflog (2013-10-28) 2 commits
 - merge-base: teach "--fork-point" mode
 - merge-base: use OPT_CMDMODE and clarify the command line parsing

 Code the logic in "pull --rebase" that figures out a fork point
 from reflog entries in C.

 Rerolled.


* jk/date-c-double-semicolon (2013-10-24) 1 commit
  (merged to 'next' on 2013-10-28 at 00ce440)
 + drop redundant semicolon in empty while

 Will merge to 'master'.


* jk/for-each-ref-skip-parsing (2013-10-24) 1 commit
 - for-each-ref: avoid loading objects to print %(objectname)

 Will merge to 'next' and then to 'master'.


* jk/pack-bitmap (2013-10-28) 20 commits
 - count-objects: consider .bitmap without .pack/.idx pair garbage
 - pack-bitmap: implement optional name_hash cache
 - t: add basic bitmap functionality tests
 - repack: consider bitmaps when performing repacks
 - repack: handle optional files created by pack-objects
 - repack: turn exts array into array-of-struct
 - repack: stop using magic number for ARRAY_SIZE(exts)
 - pack-objects: implement bitmap writing
 - rev-list: add bitmap mode to speed up object lists
 - pack-objects: use bitmaps when packing objects
 - pack-bitmap: add support for bitmap indexes
 - documentation: add documentation for the bitmap format
 - ewah: compressed bitmap implementation
 - compat: add endianness helpers
 - sha1_file: export `git_open_noatime`
 - revision: allow setting custom limiter function
 - pack-objects: factor out name_hash
 - pack-objects: refactor the packing list
 - revindex: export new APIs
 - sha1write: make buffer const-correct

 Borrows the bitmap index into packfiles from JGit to speed up
 enumeration of objects involved in a commit range without having to
 fully traverse the history.


* jk/refs-c-squelch-gcc (2013-10-24) 1 commit
  (merged to 'next' on 2013-10-28 at d15f7c2)
 + silence gcc array-bounds warning

 Will merge to 'master'.


* jk/robustify-parse-commit (2013-10-24) 6 commits
 - checkout: do not die when leaving broken detached HEAD
 - use parse_commit_or_die instead of custom message
 - use parse_commit_or_die instead of segfaulting
 - assume parse_commit checks for NULL commit
 - assume parse_commit checks commit->object.parsed
 - log_tree_diff: die when we fail to parse a commit

 Will merge to 'next' after taking another look.


* mh/fetch-tags-in-addition-to-normal-refs (2013-10-24) 16 commits
 - fetch, remote: properly convey --no-prune options to subprocesses
 - builtin/remote.c:update(): use struct argv_array
 - builtin/remote.c: reorder function definitions
 - query_refspecs(): move some constants out of the loop
 - fetch --prune: prune only based on explicit refspecs
 - SQUASH??? --tags is no longer a short-hand
 - fetch --tags: fetch tags *in addition to* other stuff
 - builtin/fetch.c: reorder function definitions
 - ref_remove_duplicates(): improve documentation comment
 - ref_remove_duplicates(): simplify function
 - ref_remove_duplicates(): avoid redundant bisection
 - get_ref_map(): rename local variables
 - api-remote.txt: correct section "struct refspec"
 - t5510: check that "git fetch --prune --tags" does not prune branches
 - t5510: prepare test refs more straightforwardly
 - t5510: use the correct tag name in test

 Some questionable paragraphs in the doc updates, but other than
 that looks reasonably solid.

 Expecting a reroll.


* nd/lift-path-max (2013-10-24) 2 commits
  (merged to 'next' on 2013-10-28 at 07698af)
 + checkout_entry(): clarify the use of topath[] parameter
 + entry.c: convert checkout_entry to use strbuf

 Will merge to 'master'.


* jk/pack-corruption-post-mortem (2013-10-25) 1 commit
 - howto: add article on recovering a corrupted object

 Will merge to 'next' and then to 'master'.


* jk/reset-p-current-head-fix (2013-10-25) 2 commits
 - reset: pass real rev name to add--interactive
 - add-interactive: handle unborn branch in patch mode

 "git reset -p HEAD" has codepath to special case it from resetting
 to contents of other commits, but recent change broke it.

 Will merge to 'next' and then to 'master'.


* mf/graph-show-root (2013-10-25) 1 commit
 - graph.c: mark root commit differently

 In a repository with multiple-roots, "log --graph", especially with
 "--oneline", does not give the reader enough visual cue to see
 where one line of history ended and a separate history began.

 This is the version that marks the roots 'x' when they would have
 been marked as '*'; Keshav Kini suggested an alternative of giving
 an extra blank line after every root, which I tend to think is a
 better approach to the problem.


* nv/parseopt-opt-arg (2013-10-25) 1 commit
 - rev-parse --parseopt: add the --sticked-long mode

 Enhance "rev-parse --parseopt" mode to help parsing options with
 an optional parameter.

 Option name bikeshedding aside ("Is it sticked or stuck?"), the
 change seems to be competently done.


* ap/remote-hg-unquote-cquote (2013-10-23) 1 commit
  (merged to 'next' on 2013-10-28 at 6b99fd5)
 + remote-hg: unquote C-style paths when exporting

 A fast-import stream expresses a pathname with funny characters by
 quoting them in C style; remote-hg remote helper forgot to unquote
 such a path.

 Will merge to 'next'.


* jl/pack-transfer-avoid-double-close (2013-10-23) 1 commit
  (merged to 'next' on 2013-10-28 at 4a55bba)
 + Clear fd after closing to avoid double-close error

 The codepath that send_pack() calls pack_objects() mistakenly
 closed the same file descriptor twice, leading to potentially
 closing a wrong file descriptor that was opened in the meantime.

 Will merge to 'master' and later to 'maint'.


* nd/magic-pathspec (2013-10-22) 1 commit
  (merged to 'next' on 2013-10-28 at 50eda68)
 + Fix calling parse_pathspec with no paths nor PATHSPEC_PREFER_* flags

 All callers to parse_pathspec() must choose between getting no
 pathspec or one path that is limited to the current directory
 when there is no paths given on the command line, but there were
 two callers that violated this rule, triggering a BUG().

 Will merge to 'master'.


* sb/git-svn-docs-indent-with-ht (2013-10-22) 1 commit
  (merged to 'next' on 2013-10-28 at 8a952d1)
 + git-svn docs: Use tabs consistently within the ascii doc

 Will merge to 'master'.


* tr/gitk-doc-update (2013-10-22) 1 commit
  (merged to 'next' on 2013-10-28 at f4158b8)
 + Documentation: revamp gitk(1)

 Will merge to 'master'.


* tr/valgrind-test-fix (2013-10-22) 2 commits
  (merged to 'next' on 2013-10-28 at 4d3f31a)
 + Revert "test-lib: allow prefixing a custom string before "ok N" etc."
 + Revert "test-lib: support running tests under valgrind in parallel"

 Will merge to 'master'.


* mm/checkout-auto-track-fix (2013-10-18) 2 commits
  (merged to 'next' on 2013-10-28 at f4594ba)
 + checkout: proper error message on 'git checkout foo bar --'
 + checkout: allow dwim for branch creation for "git checkout $branch --"

 "git checkout topic", when there is not yet a local "topic" branch
 but there is a unique remote-tracking branch for a remote "topic"
 branch, pretended as if "git checkout -t -b topic remote/$r/topic"
 (for that unique remote $r) was run. This hack however was not
 implemented for "git checkout topic --".

 Will merge to 'master'.


* hn/log-graph-color-octopus (2013-10-18) 1 commit
  (merged to 'next' on 2013-10-28 at e103175)
 + graph: fix coloring around octopus merges

 Will merge to 'master'.


* nd/gc-lock-against-each-other (2013-10-18) 1 commit
  (merged to 'next' on 2013-10-28 at 14bd458)
 + gc: remove gc.pid file at end of execution

 Will merge to 'master'.


* fc/styles (2013-10-16) 7 commits
  (merged to 'next' on 2013-10-28 at cf592ed)
 + block-sha1/sha1.c: have SP around arithmetic operators
 + base85.c: have SP around arithmetic operators
 + archive.c: have SP around arithmetic operators
 + alloc.c: have SP around arithmetic operators
 + abspath.c: have SP around arithmetic operators
 + alias: have SP around arithmetic operators
 + C: have space around && and || operators

 C coding style fixes.

 Will merge to 'master'.


* sg/t3600-nul-sha1-fix (2013-10-16) 1 commit
  (merged to 'next' on 2013-10-28 at ac4b703)
 + t3600: fix broken "choking git rm" test

 Will merge to 'master'.


* ak/submodule-foreach-quoting (2013-09-27) 1 commit
  (merged to 'next' on 2013-10-14 at d77c5f1)
 + submodule foreach: skip eval for more than one argument

 A behavior change, but a worthwhile one: "git submodule foreach"
 was treating its arguments as part of a single command to be
 concatenated and passed to a shell, making writing buggy
 scripts too easy.

 This patch preserves the old "just pass it to the shell" behavior
 when a single argument is passed to 'git submodule foreach' and
 moves to a new "skip the shell and use the arguments passed
 unmolested" behavior when more than one argument is passed.

 The old behavior (always concatenating and passing to the shell)
 was similar to the 'ssh' command, while the new behavior (switching
 on the number of arguments) is what 'xterm -e' does.

 May need more thought to make sure this change is advertised well
 so that scripts that used multiple arguments but added their own
 extra layer of quoting are not broken.

 Will cook in 'next' for the rest of this cycle.


* jk/http-auth-redirects (2013-10-24) 10 commits
  (merged to 'next' on 2013-10-24 at 4bebb66)
 + http.c: Spell the null pointer as NULL
 + remote-curl: rewrite base url from info/refs redirects
 + remote-curl: store url as a strbuf
 + remote-curl: make refs_url a strbuf
 + http: update base URLs when we see redirects
 + http: provide effective url to callers
 + http: hoist credential request out of handle_curl_result
  (merged to 'next' on 2013-10-14 at a0642be)
 + http: refactor options to http_get_*
 + http_request: factor out curlinfo_strbuf
 + http_get_file: style fixes

 Handle the case where http transport gets redirected during the
 authorization request better.

 Will merge to 'master'.


* jl/submodule-mv (2013-10-13) 1 commit
  (merged to 'next' on 2013-10-28 at 8dc9b31)
 + mv: Fix spurious warning when moving a file in presence of submodules

 Moving a regular file in a repository with a .gitmodules file was
 producing a warning 'Could not find section in .gitmodules where
 path=<filename>'.

 Will merge to 'master'.


* kb/fast-hashmap (2013-10-22) 12 commits
 - remove old hash.[ch] implementation
 - read-cache.c: fix memory leaks caused by removed cache entries
 - name-hash.c: remove cache entries instead of marking them CE_UNHASHED
 - name-hash.c: use new hash map implementation for cache entries
 - name-hash.c: remove unreferenced directory entries
 - name-hash.c: use new hash map implementation for directories
 - diffcore-rename.c: use new hash map implementation
 - diffcore-rename.c: simplify finding exact renames
 - diffcore-rename.c: move code around to prepare for the next patch
 - buitin/describe.c: use new hash map implementation
 - add a hashtable implementation that supports O(1) removal
 - submodule: don't access the .gitmodules cache entry after removing it

 Improvements to our hash table to get it to meet the needs of the
 msysgit fscache project, with some nice performance improvements.

 The preparatory clean-up to submodule from Jens is at the bottom. I
 also squashed in a fix-up by Karsten found at $gmane/236468 (please
 double-check the result).

 Will merge to 'next'.


* jc/upload-pack-send-symref (2013-10-22) 10 commits
  (merged to 'next' on 2013-10-23 at 8ef5660)
 + t5570: Update for clone-progress-to-stderr branch
 + Merge branch 'jk/clone-progress-to-stderr' into jc/upload-pack-send-symref
 + t5570: Update for symref capability
  (merged to 'next' on 2013-10-16 at eb1ae25)
 + clone: test the new HEAD detection logic
 + connect: annotate refs with their symref information in get_remote_head()
 + connect.c: make parse_feature_value() static
 + upload-pack: send non-HEAD symbolic refs
 + upload-pack: send symbolic ref information as capability
 + upload-pack.c: do not pass confusing cb_data to mark_our_ref()
 + t5505: fix "set-head --auto with ambiguous HEAD" test

 One long-standing flaw in the pack transfer protocol used by "git
 clone" was that there was no way to tell the other end which branch
 "HEAD" points at, and the receiving end needed to guess.  A new
 capability has been defined in the pack protocol to convey this
 information so that cloning from a repository with more than one
 branches pointing at the same commit where the HEAD is at now
 reliably sets the initial branch in the resulting repository.

 Will merge to 'master'.


* jn/add-2.0-u-A-sans-pathspec (2013-04-26) 1 commit
  (merged to 'next' on 2013-10-28 at d8cdf30)
 + git add: -u/-A now affects the entire working tree

 Will cook in 'next' until Git 2.0.


* jc/core-checkstat-2.0 (2013-05-06) 1 commit
  (merged to 'next' on 2013-10-28 at f1bec96)
 + core.statinfo: remove as promised in Git 2.0

 Will cook in 'next' until Git 2.0.


* jc/push-2.0-default-to-simple (2013-06-18) 1 commit
  (merged to 'next' on 2013-10-28 at 3153a9e)
 + push: switch default from "matching" to "simple"

 Will cook in 'next' until Git 2.0.


* jc/add-2.0-ignore-removal (2013-04-22) 1 commit
  (merged to 'next' on 2013-10-28 at 5fd76ec)
 + git add <pathspec>... defaults to "-A"

 Updated endgame for "git add <pathspec>" that defaults to "--all"
 aka "--no-ignore-removal".

 Will cook in 'next' until Git 2.0.


* jc/hold-diff-remove-q-synonym-for-no-deletion (2013-07-19) 1 commit
 - diff: remove "diff-files -q" in a version of Git in a distant future

 Will merge to and cook in 'next' until a distant future.

--------------------------------------------------
[Discarded]

* jh/shorten-refname (2013-05-07) 4 commits
 . t1514: refname shortening is done after dereferencing symbolic refs
 . shorten_unambiguous_ref(): Fix shortening refs/remotes/origin/HEAD to origin
 . t1514: Demonstrate failure to correctly shorten "refs/remotes/origin/HEAD"
 . t1514: Add tests of shortening refnames in strict/loose mode

 When remotes/origin/HEAD is not a symbolic ref, "rev-parse
 --abbrev-ref remotes/origin/HEAD" ought to show "origin", not
 "origin/HEAD", which is fixed with this series (if it is a symbolic
 ref that points at remotes/origin/something, then it should show
 "origin/something" and it already does).

 Has been expecting a reroll, as an early part of a larger series.
 $gmane/225137

 Discarded due to inactivity, without prejudice.

Re: What's cooking in git.git (Oct 2013, #07; Mon, 28)

[Thomas Rast]

Hi Karsten

Junio C Hamano <gitster@pobox.com> writes:

> * kb/fast-hashmap (2013-10-22) 12 commits
>  - remove old hash.[ch] implementation
>  - read-cache.c: fix memory leaks caused by removed cache entries

I found more valgrind breakage related to this commit, in t2101.[3567]
(sorry for only reporting them so late, I probably missed them in the
last run).  E.g. I get this:

  $ ./t2101-update-index-reupdate.sh --valgrind-only=3
  ok 1 - update-index --add
  ok 2 - update-index --again

  expecting success: git update-index --remove --again &&
           git ls-files -s >current &&
           cmp current expected
  ==21665== Invalid read of size 1
  ==21665==    at 0x4C2C762: __GI_strlen (mc_replace_strmem.c:405)
  ==21665==    by 0x484B0E: update_one (update-index.c:305)
  ==21665==    by 0x485466: do_reupdate (update-index.c:582)
  ==21665==    by 0x4858FB: reupdate_callback (update-index.c:696)
  ==21665==    by 0x4EB5E7: get_value (parse-options.c:96)
  ==21665==    by 0x4EBEC5: parse_long_opt (parse-options.c:302)
  ==21665==    by 0x4EC5CD: parse_options_step (parse-options.c:474)
  ==21665==    by 0x486115: cmd_update_index (update-index.c:824)
  ==21665==    by 0x405999: run_builtin (git.c:314)
  ==21665==    by 0x405B2C: handle_internal_command (git.c:477)
  ==21665==    by 0x405C46: run_argv (git.c:523)
  ==21665==    by 0x405DE2: main (git.c:606)
  ==21665==  Address 0x5bee774 is 84 bytes inside a block of size 90 free'd
  ==21665==    at 0x4C2ACDA: free (vg_replace_malloc.c:468)
  ==21665==    by 0x4F9360: remove_index_entry_at (read-cache.c:482)
  ==21665==    by 0x4F9536: remove_file_from_index (read-cache.c:522)
  ==21665==    by 0x4841DF: remove_one_path (update-index.c:68)
  ==21665==    by 0x48422E: process_lstat_error (update-index.c:83)
  ==21665==    by 0x4846BB: process_path (update-index.c:211)
  ==21665==    by 0x484AC2: update_one (update-index.c:301)
  ==21665==    by 0x485466: do_reupdate (update-index.c:582)
  ==21665==    by 0x4858FB: reupdate_callback (update-index.c:696)
  ==21665==    by 0x4EB5E7: get_value (parse-options.c:96)
  ==21665==    by 0x4EBEC5: parse_long_opt (parse-options.c:302)
  ==21665==    by 0x4EC5CD: parse_options_step (parse-options.c:474)
  [...]
  not ok 3 - update-index --remove --again
  #       git update-index --remove --again &&
  #                git ls-files -s >current &&
  #                cmp current expected

  ok 4 - first commit
  ok 5 - update-index again
  ok 6 - update-index --update from subdir
  ok 7 - update-index --update with pathspec
  # failed 1 among 7 test(s)
  1..7

The errors for tests 5-7 look like they're the same piece of code
breaking.

-- 
Thomas Rast
tr@thomasrast.ch

Re: What's cooking in git.git (Oct 2013, #07; Mon, 28)

[Torsten Bögershausen]

On 2013-10-28 20.28, Junio C Hamano wrote:
> * jk/pack-bitmap (2013-10-28) 20 commits
There is a name clash under cygwin 1.7 (1.5 is OK)
The following "first aid hot fix" works for me:
/Torsten

$ git diff
diff --git a/compat/bswap.h b/compat/bswap.h
index ea1a9ed..8dc39be 100644
--- a/compat/bswap.h
+++ b/compat/bswap.h
@@ -64,7 +64,7 @@ static inline uint32_t git_bswap32(uint32_t x)
 #      if defined(__GNUC__) && defined(__GLIBC__)
 #              include <byteswap.h>
 #      else /* GNUC & GLIBC */
-static inline uint64_t bswap_64(uint64_t val)
+static inline uint64_t git_bswap_64(uint64_t val)
 {
        return ((val & (uint64_t)0x00000000000000ffULL) << 56)
                | ((val & (uint64_t)0x000000000000ff00ULL) << 40)
@@ -76,8 +76,8 @@ static inline uint64_t bswap_64(uint64_t val)
                | ((val & (uint64_t)0xff00000000000000ULL) >> 56);
 }
 #      endif /* GNUC & GLIBC */
-#      define ntohll(n) bswap_64(n)
-#      define htonll(n) bswap_64(n)
+#      define ntohll(n) git_bswap_64(n)
+#      define htonll(n) git_bswap_64(n)
 #else /* __BYTE_ORDER */
 #      error "Can't define htonll or ntohll!"
 #endif

Re: What's cooking in git.git (Oct 2013, #07; Mon, 28)

[Vicent Martí]

On Wed, Oct 30, 2013 at 5:51 PM, Torsten Bögershausen <tboegi@web.de> wrote:
> There is a name clash under cygwin 1.7 (1.5 is OK)
> The following "first aid hot fix" works for me:
> /Torsten

If Cygwin declares its own bswap_64, wouldn't it be better to use it
instead of overwriting it with our own?

Re: What's cooking in git.git (Oct 2013, #07; Mon, 28)

[Torsten Bögershausen]

On 2013-10-30 18.01, Vicent Martí wrote:
> On Wed, Oct 30, 2013 at 5:51 PM, Torsten Bögershausen <tboegi@web.de> wrote:
>> There is a name clash under cygwin 1.7 (1.5 is OK)
>> The following "first aid hot fix" works for me:
>> /Torsten
> 
> If Cygwin declares its own bswap_64, wouldn't it be better to use it
> instead of overwriting it with our own?
Yes,
this will be part of a longer patch.
I found that some systems have something like this:

#define htobe64(x) bswap_64(x)
And bswap_64 is a function, so we can not detect it by "asking"
#ifdef bswap_64
..
#endif


But we can use
#ifdef htobe64
...
#endif
and this will be part of a bigger patch.

And, in general, we should avoid to introduce functions which may have a
name clash.
Using the git_ prefix for function names is a good practice.
So in order to unbrake the compilation error under cygwin 17,
the "hotfix" can be used.
/Torsten

Re: What's cooking in git.git (Oct 2013, #07; Mon, 28)

[Torsten Bögershausen]

On 2013-10-30 18.14, Torsten Bögershausen wrote:
> On 2013-10-30 18.01, Vicent Martí wrote:
>> On Wed, Oct 30, 2013 at 5:51 PM, Torsten Bögershausen <tboegi@web.de> wrote:
>>> There is a name clash under cygwin 1.7 (1.5 is OK)
>>> The following "first aid hot fix" works for me:
>>> /Torsten
>>
>> If Cygwin declares its own bswap_64, wouldn't it be better to use it
>> instead of overwriting it with our own?
> Yes,
> this will be part of a longer patch.
> I found that some systems have something like this:
> 
> #define htobe64(x) bswap_64(x)
> And bswap_64 is a function, so we can not detect it by "asking"
> #ifdef bswap_64
> ..
> #endif
> 
> 
> But we can use
> #ifdef htobe64
> ...
> #endif
> and this will be part of a bigger patch.
> 
> And, in general, we should avoid to introduce functions which may have a
> name clash.
> Using the git_ prefix for function names is a good practice.
> So in order to unbrake the compilation error under cygwin 17,
> the "hotfix" can be used.
> /Torsten
I just realized that there seem to problems to compile pu under msysgit.
More investigation needed here.

Re: What's cooking in git.git (Oct 2013, #07; Mon, 28)

[Ramsay Jones]

On 30/10/13 17:14, Torsten Bögershausen wrote:
> On 2013-10-30 18.01, Vicent Martí wrote:
>> On Wed, Oct 30, 2013 at 5:51 PM, Torsten Bögershausen <tboegi@web.de> wrote:
>>> There is a name clash under cygwin 1.7 (1.5 is OK)
>>> The following "first aid hot fix" works for me:
>>> /Torsten
>>
>> If Cygwin declares its own bswap_64, wouldn't it be better to use it
>> instead of overwriting it with our own?
> Yes,
> this will be part of a longer patch.
> I found that some systems have something like this:
> 
> #define htobe64(x) bswap_64(x)
> And bswap_64 is a function, so we can not detect it by "asking"
> #ifdef bswap_64
> ..
> #endif
> 
> 
> But we can use
> #ifdef htobe64
> ...
> #endif
> and this will be part of a bigger patch.
> 
> And, in general, we should avoid to introduce functions which may have a
> name clash.
> Using the git_ prefix for function names is a good practice.
> So in order to unbrake the compilation error under cygwin 17,
> the "hotfix" can be used.

heh, my patch (given below) took a different approach, but ....

ATB,
Ramsay Jones

-- >8 --
Subject: [PATCH] compat/bswap.h: Fix redefinition of bswap_64 error on cygwin
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Since commit 452e0f20 ("compat: add endianness helpers", 24-10-2013)
the cygwin build has failed like so:

    GIT_VERSION = 1.8.4.1.804.g1f3748b
        * new build flags
        CC credential-store.o
    In file included from git-compat-util.h:305:0,
                     from cache.h:4,
                     from credential-store.c:1:
    compat/bswap.h:67:24: error: redefinition of 'bswap_64'
    In file included from /usr/include/endian.h:32:0,
                     from /usr/include/cygwin/types.h:21,
                     from /usr/include/sys/types.h:473,
                     from /usr/include/sys/unistd.h:9,
                     from /usr/include/unistd.h:4,
                     from git-compat-util.h:98,
                     from cache.h:4,
                     from credential-store.c:1:
    /usr/include/byteswap.h:31:1: note: previous definition of \
	‘bswap_64’ was here
    Makefile:1985: recipe for target 'credential-store.o' failed
    make: *** [credential-store.o] Error 1

Note that cygwin has a defintion of 'bswap_64' in the <byteswap.h>
header file (which had already been included by git-compat-util.h).
In order to suppress the error, ensure that the <byteswap.h> header
is included, just like the __GNUC__/__GLIBC__ case, rather than
attempting to define a fallback implementation.

Signed-off-by: Ramsay Jones <ramsay@ramsay1.demon.co.uk>
---
 compat/bswap.h | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/compat/bswap.h b/compat/bswap.h
index ea1a9ed..b864abd 100644
--- a/compat/bswap.h
+++ b/compat/bswap.h
@@ -61,7 +61,7 @@ static inline uint32_t git_bswap32(uint32_t x)
 # define ntohll(n) (n)
 # define htonll(n) (n)
 #elif __BYTE_ORDER == __LITTLE_ENDIAN
-#	if defined(__GNUC__) && defined(__GLIBC__)
+#	if defined(__GNUC__) && (defined(__GLIBC__) || defined(__CYGWIN__))
 #		include <byteswap.h>
 #	else /* GNUC & GLIBC */
 static inline uint64_t bswap_64(uint64_t val)
-- 
1.8.4

Re: What's cooking in git.git (Oct 2013, #07; Mon, 28)

[Ramsay Jones]

On 30/10/13 17:39, Torsten Bögershausen wrote:
> On 2013-10-30 18.14, Torsten Bögershausen wrote:
>> On 2013-10-30 18.01, Vicent Martí wrote:
>>> On Wed, Oct 30, 2013 at 5:51 PM, Torsten Bögershausen <tboegi@web.de> wrote:
>>>> There is a name clash under cygwin 1.7 (1.5 is OK)
>>>> The following "first aid hot fix" works for me:
>>>> /Torsten
>>>
>>> If Cygwin declares its own bswap_64, wouldn't it be better to use it
>>> instead of overwriting it with our own?
>> Yes,
>> this will be part of a longer patch.
>> I found that some systems have something like this:
>>
>> #define htobe64(x) bswap_64(x)
>> And bswap_64 is a function, so we can not detect it by "asking"
>> #ifdef bswap_64
>> ..
>> #endif
>>
>>
>> But we can use
>> #ifdef htobe64
>> ...
>> #endif
>> and this will be part of a bigger patch.
>>
>> And, in general, we should avoid to introduce functions which may have a
>> name clash.
>> Using the git_ prefix for function names is a good practice.
>> So in order to unbrake the compilation error under cygwin 17,
>> the "hotfix" can be used.
>> /Torsten
> I just realized that there seem to problems to compile pu under msysgit.
> More investigation needed here.

... I noticed this too, and my patch is given below (I have another
patch for mingw which fixes some printf format warnings too) ...

However, you would not be surprised to hear that this breaks on msvc
too, so I too was planning a larger re-write ... :-D

ATB,
Ramsay Jones

-- >8 --
Subject: [PATCH] compat/bswap.h: Fix failure to determine endianness on MinGW

Since commit 452e0f20 ("compat: add endianness helpers", 24-10-2013)
added the 'ntohll' and 'htonll' helpers, the MinGW build has failed
like so:

    GIT_VERSION = 1.8.4.1.804.g1f3748b
        * new build flags
        CC credential-store.o
    In file included from git-compat-util.h:305,
                     from cache.h:4,
                     from credential-store.c:1:
    compat/bswap.h:56:4: error: #error "Cannot determine endianness"
    make: *** [credential-store.o] Error 1

The #error is triggered because the 'endian macros' BYTE_ORDER,
LITTLE_ENDIAN and BIG_ENDIAN not being defined. On MinGW, these macros
are defined in the <sys/param.h> header file. In order to suppress the
error, set the build variable NEEDS_SYS_PARAM_H, which will cause the
"git-compat-util.h" header file to include <sys/param.h>.

Signed-off-by: Ramsay Jones <ramsay@ramsay1.demon.co.uk>
---
 config.mak.uname | 1 +
 1 file changed, 1 insertion(+)

diff --git a/config.mak.uname b/config.mak.uname
index 82d549e..c03ea1e 100644
--- a/config.mak.uname
+++ b/config.mak.uname
@@ -469,6 +469,7 @@ ifneq (,$(findstring MINGW,$(uname_S)))
 	pathsep = ;
 	NO_PREAD = YesPlease
 	NEEDS_CRYPTO_WITH_SSL = YesPlease
+	NEEDS_SYS_PARAM_H = YesPlease
 	NO_LIBGEN_H = YesPlease
 	NO_POLL = YesPlease
 	NO_SYMLINK_HEAD = YesPlease
-- 
1.8.4

Re: What's cooking in git.git (Oct 2013, #07; Mon, 28)

[Torsten Bögershausen]

On 2013-10-30 20.06, Ramsay Jones wrote:
> On 30/10/13 17:14, Torsten Bögershausen wrote:
>> On 2013-10-30 18.01, Vicent Martí wrote:
>>> On Wed, Oct 30, 2013 at 5:51 PM, Torsten Bögershausen <tboegi@web.de> wrote:
>>>> There is a name clash under cygwin 1.7 (1.5 is OK)
>>>> The following "first aid hot fix" works for me:
>>>> /Torsten
>>>
>>> If Cygwin declares its own bswap_64, wouldn't it be better to use it
>>> instead of overwriting it with our own?
>> Yes,
>> this will be part of a longer patch.
>> I found that some systems have something like this:
>>
>> #define htobe64(x) bswap_64(x)
>> And bswap_64 is a function, so we can not detect it by "asking"
>> #ifdef bswap_64
>> ..
>> #endif
>>
>>
>> But we can use
>> #ifdef htobe64
>> ...
>> #endif
>> and this will be part of a bigger patch.
>>
>> And, in general, we should avoid to introduce functions which may have a
>> name clash.
>> Using the git_ prefix for function names is a good practice.
>> So in order to unbrake the compilation error under cygwin 17,
>> the "hotfix" can be used.
> 
> heh, my patch (given below) took a different approach, but ....
> 
> ATB,
> Ramsay Jones
> 
> -- >8 --
> Subject: [PATCH] compat/bswap.h: Fix redefinition of bswap_64 error on cygwin
> MIME-Version: 1.0
> Content-Type: text/plain; charset=UTF-8
> Content-Transfer-Encoding: 8bit
> 
> Since commit 452e0f20 ("compat: add endianness helpers", 24-10-2013)
> the cygwin build has failed like so:
> 
>     GIT_VERSION = 1.8.4.1.804.g1f3748b
>         * new build flags
>         CC credential-store.o
>     In file included from git-compat-util.h:305:0,
>                      from cache.h:4,
>                      from credential-store.c:1:
>     compat/bswap.h:67:24: error: redefinition of 'bswap_64'
>     In file included from /usr/include/endian.h:32:0,
>                      from /usr/include/cygwin/types.h:21,
>                      from /usr/include/sys/types.h:473,
>                      from /usr/include/sys/unistd.h:9,
>                      from /usr/include/unistd.h:4,
>                      from git-compat-util.h:98,
>                      from cache.h:4,
>                      from credential-store.c:1:
>     /usr/include/byteswap.h:31:1: note: previous definition of \
> 	‘bswap_64’ was here
>     Makefile:1985: recipe for target 'credential-store.o' failed
>     make: *** [credential-store.o] Error 1
> 
> Note that cygwin has a defintion of 'bswap_64' in the <byteswap.h>
> header file (which had already been included by git-compat-util.h).
> In order to suppress the error, ensure that the <byteswap.h> header
> is included, just like the __GNUC__/__GLIBC__ case, rather than
> attempting to define a fallback implementation.
> 
> Signed-off-by: Ramsay Jones <ramsay@ramsay1.demon.co.uk>
> ---
>  compat/bswap.h | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/compat/bswap.h b/compat/bswap.h
> index ea1a9ed..b864abd 100644
> --- a/compat/bswap.h
> +++ b/compat/bswap.h
> @@ -61,7 +61,7 @@ static inline uint32_t git_bswap32(uint32_t x)
>  # define ntohll(n) (n)
>  # define htonll(n) (n)
>  #elif __BYTE_ORDER == __LITTLE_ENDIAN
> -#	if defined(__GNUC__) && defined(__GLIBC__)
> +#	if defined(__GNUC__) && (defined(__GLIBC__) || defined(__CYGWIN__))
>  #		include <byteswap.h>
>  #	else /* GNUC & GLIBC */
>  static inline uint64_t bswap_64(uint64_t val)
> 

Nice, much better.

And here comes a patch for a big endian machine.
I tryied to copy-paste a patch in my mail program,
not sure if this applies.

-- >8 --
Subject: [PATCH] compat/bswap.h: htonll and ntohll for big endian

Since commit 452e0f20 ("compat: add endianness helpers", 24-10-2013)
the build on a Linux/ppc gave a warning like this:
    CC ewah/ewah_io.o
ewah/ewah_io.c: In function ‘ewah_serialize_to’:
ewah/ewah_io.c:81: warning: implicit declaration of function ‘htonll’
ewah/ewah_io.c: In function ‘ewah_read_mmap’:
ewah/ewah_io.c:132: warning: implicit declaration of function ‘ntohll’

Fix it by placing the #endif for "#ifdef bswap32" at the right place.

Signed-off-by: Torsten Bögershausen <tboegi@web.de>
---
 compat/bswap.h | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/compat/bswap.h b/compat/bswap.h
index ea1a9ed..b4ddab0
--- a/compat/bswap.h
+++ b/compat/bswap.h
@@ -46,6 +46,7 @@ static inline uint32_t git_bswap32(uint32_t x)
 #undef htonl
 #define ntohl(x) bswap32(x)
 #define htonl(x) bswap32(x)
+#endif
 
 #ifndef __BYTE_ORDER
 #      if defined(BYTE_ORDER) && defined(LITTLE_ENDIAN) && defined(BIG_ENDIAN)
@@ -82,4 +83,3 @@ static inline uint64_t bswap_64(uint64_t val)
 #      error "Can't define htonll or ntohll!"
 #endif
 
-#endif

Re: What's cooking in git.git (Oct 2013, #07; Mon, 28)

[Ramsay Jones]

On 30/10/13 20:30, Torsten Bögershausen wrote:
> On 2013-10-30 20.06, Ramsay Jones wrote:
>> On 30/10/13 17:14, Torsten Bögershausen wrote:
>>> On 2013-10-30 18.01, Vicent Martí wrote:
>>>> On Wed, Oct 30, 2013 at 5:51 PM, Torsten Bögershausen <tboegi@web.de> wrote:
>>>>> There is a name clash under cygwin 1.7 (1.5 is OK)
>>>>> The following "first aid hot fix" works for me:
>>>>> /Torsten
>>>>
>>>> If Cygwin declares its own bswap_64, wouldn't it be better to use it
>>>> instead of overwriting it with our own?
>>> Yes,
>>> this will be part of a longer patch.
>>> I found that some systems have something like this:
>>>
>>> #define htobe64(x) bswap_64(x)
>>> And bswap_64 is a function, so we can not detect it by "asking"
>>> #ifdef bswap_64
>>> ..
>>> #endif
>>>
>>>
>>> But we can use
>>> #ifdef htobe64
>>> ...
>>> #endif
>>> and this will be part of a bigger patch.
>>>
>>> And, in general, we should avoid to introduce functions which may have a
>>> name clash.
>>> Using the git_ prefix for function names is a good practice.
>>> So in order to unbrake the compilation error under cygwin 17,
>>> the "hotfix" can be used.
>>
>> heh, my patch (given below) took a different approach, but ....
>>
>> ATB,
>> Ramsay Jones
>>
>> -- >8 --
>> Subject: [PATCH] compat/bswap.h: Fix redefinition of bswap_64 error on cygwin
>> MIME-Version: 1.0
>> Content-Type: text/plain; charset=UTF-8
>> Content-Transfer-Encoding: 8bit
>>
>> Since commit 452e0f20 ("compat: add endianness helpers", 24-10-2013)
>> the cygwin build has failed like so:
>>
>>     GIT_VERSION = 1.8.4.1.804.g1f3748b
>>         * new build flags
>>         CC credential-store.o
>>     In file included from git-compat-util.h:305:0,
>>                      from cache.h:4,
>>                      from credential-store.c:1:
>>     compat/bswap.h:67:24: error: redefinition of 'bswap_64'
>>     In file included from /usr/include/endian.h:32:0,
>>                      from /usr/include/cygwin/types.h:21,
>>                      from /usr/include/sys/types.h:473,
>>                      from /usr/include/sys/unistd.h:9,
>>                      from /usr/include/unistd.h:4,
>>                      from git-compat-util.h:98,
>>                      from cache.h:4,
>>                      from credential-store.c:1:
>>     /usr/include/byteswap.h:31:1: note: previous definition of \
>> 	‘bswap_64’ was here
>>     Makefile:1985: recipe for target 'credential-store.o' failed
>>     make: *** [credential-store.o] Error 1
>>
>> Note that cygwin has a defintion of 'bswap_64' in the <byteswap.h>
>> header file (which had already been included by git-compat-util.h).
>> In order to suppress the error, ensure that the <byteswap.h> header
>> is included, just like the __GNUC__/__GLIBC__ case, rather than
>> attempting to define a fallback implementation.
>>
>> Signed-off-by: Ramsay Jones <ramsay@ramsay1.demon.co.uk>
>> ---
>>  compat/bswap.h | 2 +-
>>  1 file changed, 1 insertion(+), 1 deletion(-)
>>
>> diff --git a/compat/bswap.h b/compat/bswap.h
>> index ea1a9ed..b864abd 100644
>> --- a/compat/bswap.h
>> +++ b/compat/bswap.h
>> @@ -61,7 +61,7 @@ static inline uint32_t git_bswap32(uint32_t x)
>>  # define ntohll(n) (n)
>>  # define htonll(n) (n)
>>  #elif __BYTE_ORDER == __LITTLE_ENDIAN
>> -#	if defined(__GNUC__) && defined(__GLIBC__)
>> +#	if defined(__GNUC__) && (defined(__GLIBC__) || defined(__CYGWIN__))
>>  #		include <byteswap.h>
>>  #	else /* GNUC & GLIBC */
>>  static inline uint64_t bswap_64(uint64_t val)
>>
> 
> Nice, much better.
> 
> And here comes a patch for a big endian machine.
> I tryied to copy-paste a patch in my mail program,
> not sure if this applies.
> 
> -- >8 --
> Subject: [PATCH] compat/bswap.h: htonll and ntohll for big endian
> 
> Since commit 452e0f20 ("compat: add endianness helpers", 24-10-2013)
> the build on a Linux/ppc gave a warning like this:
>     CC ewah/ewah_io.o
> ewah/ewah_io.c: In function ‘ewah_serialize_to’:
> ewah/ewah_io.c:81: warning: implicit declaration of function ‘htonll’
> ewah/ewah_io.c: In function ‘ewah_read_mmap’:
> ewah/ewah_io.c:132: warning: implicit declaration of function ‘ntohll’
> 
> Fix it by placing the #endif for "#ifdef bswap32" at the right place.
> 
> Signed-off-by: Torsten Bögershausen <tboegi@web.de>
> ---
>  compat/bswap.h | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/compat/bswap.h b/compat/bswap.h
> index ea1a9ed..b4ddab0
> --- a/compat/bswap.h
> +++ b/compat/bswap.h
> @@ -46,6 +46,7 @@ static inline uint32_t git_bswap32(uint32_t x)
>  #undef htonl
>  #define ntohl(x) bswap32(x)
>  #define htonl(x) bswap32(x)
> +#endif
>  
>  #ifndef __BYTE_ORDER
>  #      if defined(BYTE_ORDER) && defined(LITTLE_ENDIAN) && defined(BIG_ENDIAN)
> @@ -82,4 +83,3 @@ static inline uint64_t bswap_64(uint64_t val)
>  #      error "Can't define htonll or ntohll!"
>  #endif
>  
> -#endif
> 
> .
> 

Yep, this was the first thing I did as well! ;-) (*late* last night)

I haven't had time today to look into fixing up the msvc build
(or a complete re-write), so I look forward to seeing your solution.
(do you have msvc available? - or do you want me to look at fixing
it? maybe in a day or two?)

ATB,
Ramsay Jones

Re: htonll, ntohll

[Torsten Bögershausen]

On 2013-10-30 22.07, Ramsay Jones wrote:
> On 30/10/13 20:30, Torsten Bögershausen wrote:
>> On 2013-10-30 20.06, Ramsay Jones wrote:
>>> On 30/10/13 17:14, Torsten Bögershausen wrote:
>>>> On 2013-10-30 18.01, Vicent Martí wrote:
>>>>> On Wed, Oct 30, 2013 at 5:51 PM, Torsten Bögershausen <tboegi@web.de> wrote:
>>>>>> There is a name clash under cygwin 1.7 (1.5 is OK)
>>>>>> The following "first aid hot fix" works for me:
>>>>>> /Torsten
>>>>>
>>>>> If Cygwin declares its own bswap_64, wouldn't it be better to use it
>>>>> instead of overwriting it with our own?
>>>> Yes,
>>>> this will be part of a longer patch.
>>>> I found that some systems have something like this:
>>>>
>>>> #define htobe64(x) bswap_64(x)
>>>> And bswap_64 is a function, so we can not detect it by "asking"
>>>> #ifdef bswap_64
>>>> ..
>>>> #endif
>>>>
>>>>
>>>> But we can use
>>>> #ifdef htobe64
>>>> ...
>>>> #endif
>>>> and this will be part of a bigger patch.
>>>>
>>>> And, in general, we should avoid to introduce functions which may have a
>>>> name clash.
>>>> Using the git_ prefix for function names is a good practice.
>>>> So in order to unbrake the compilation error under cygwin 17,
>>>> the "hotfix" can be used.
>>>
>>> heh, my patch (given below) took a different approach, but ....
>>>
>>> ATB,
>>> Ramsay Jones
>>>
>>> -- >8 --
>>> Subject: [PATCH] compat/bswap.h: Fix redefinition of bswap_64 error on cygwin
>>> MIME-Version: 1.0
>>> Content-Type: text/plain; charset=UTF-8
>>> Content-Transfer-Encoding: 8bit
>>>
>>> Since commit 452e0f20 ("compat: add endianness helpers", 24-10-2013)
>>> the cygwin build has failed like so:
>>>
>>>     GIT_VERSION = 1.8.4.1.804.g1f3748b
>>>         * new build flags
>>>         CC credential-store.o
>>>     In file included from git-compat-util.h:305:0,
>>>                      from cache.h:4,
>>>                      from credential-store.c:1:
>>>     compat/bswap.h:67:24: error: redefinition of 'bswap_64'
>>>     In file included from /usr/include/endian.h:32:0,
>>>                      from /usr/include/cygwin/types.h:21,
>>>                      from /usr/include/sys/types.h:473,
>>>                      from /usr/include/sys/unistd.h:9,
>>>                      from /usr/include/unistd.h:4,
>>>                      from git-compat-util.h:98,
>>>                      from cache.h:4,
>>>                      from credential-store.c:1:
>>>     /usr/include/byteswap.h:31:1: note: previous definition of \
>>> 	‘bswap_64’ was here
>>>     Makefile:1985: recipe for target 'credential-store.o' failed
>>>     make: *** [credential-store.o] Error 1
>>>
>>> Note that cygwin has a defintion of 'bswap_64' in the <byteswap.h>
>>> header file (which had already been included by git-compat-util.h).
>>> In order to suppress the error, ensure that the <byteswap.h> header
>>> is included, just like the __GNUC__/__GLIBC__ case, rather than
>>> attempting to define a fallback implementation.
>>>
>>> Signed-off-by: Ramsay Jones <ramsay@ramsay1.demon.co.uk>
>>> ---
>>>  compat/bswap.h | 2 +-
>>>  1 file changed, 1 insertion(+), 1 deletion(-)
>>>
>>> diff --git a/compat/bswap.h b/compat/bswap.h
>>> index ea1a9ed..b864abd 100644
>>> --- a/compat/bswap.h
>>> +++ b/compat/bswap.h
>>> @@ -61,7 +61,7 @@ static inline uint32_t git_bswap32(uint32_t x)
>>>  # define ntohll(n) (n)
>>>  # define htonll(n) (n)
>>>  #elif __BYTE_ORDER == __LITTLE_ENDIAN
>>> -#	if defined(__GNUC__) && defined(__GLIBC__)
>>> +#	if defined(__GNUC__) && (defined(__GLIBC__) || defined(__CYGWIN__))
>>>  #		include <byteswap.h>
>>>  #	else /* GNUC & GLIBC */
>>>  static inline uint64_t bswap_64(uint64_t val)
>>>
>>
>> Nice, much better.
>>
>> And here comes a patch for a big endian machine.
>> I tryied to copy-paste a patch in my mail program,
>> not sure if this applies.
>>
>> -- >8 --
>> Subject: [PATCH] compat/bswap.h: htonll and ntohll for big endian
>>
>> Since commit 452e0f20 ("compat: add endianness helpers", 24-10-2013)
>> the build on a Linux/ppc gave a warning like this:
>>     CC ewah/ewah_io.o
>> ewah/ewah_io.c: In function ‘ewah_serialize_to’:
>> ewah/ewah_io.c:81: warning: implicit declaration of function ‘htonll’
>> ewah/ewah_io.c: In function ‘ewah_read_mmap’:
>> ewah/ewah_io.c:132: warning: implicit declaration of function ‘ntohll’
>>
>> Fix it by placing the #endif for "#ifdef bswap32" at the right place.
>>
>> Signed-off-by: Torsten Bögershausen <tboegi@web.de>
>> ---
>>  compat/bswap.h | 2 +-
>>  1 file changed, 1 insertion(+), 1 deletion(-)
>>
>> diff --git a/compat/bswap.h b/compat/bswap.h
>> index ea1a9ed..b4ddab0
>> --- a/compat/bswap.h
>> +++ b/compat/bswap.h
>> @@ -46,6 +46,7 @@ static inline uint32_t git_bswap32(uint32_t x)
>>  #undef htonl
>>  #define ntohl(x) bswap32(x)
>>  #define htonl(x) bswap32(x)
>> +#endif
>>  
>>  #ifndef __BYTE_ORDER
>>  #      if defined(BYTE_ORDER) && defined(LITTLE_ENDIAN) && defined(BIG_ENDIAN)
>> @@ -82,4 +83,3 @@ static inline uint64_t bswap_64(uint64_t val)
>>  #      error "Can't define htonll or ntohll!"
>>  #endif
>>  
>> -#endif
>>
>> .
>>
> 
> Yep, this was the first thing I did as well! ;-) (*late* last night)
> 
> I haven't had time today to look into fixing up the msvc build
> (or a complete re-write), so I look forward to seeing your solution.
> (do you have msvc available? - or do you want me to look at fixing
> it? maybe in a day or two?)
> 
> ATB,
> Ramsay Jones
Ramsay,
I don't have msvc, so feel free to go ahead, as much as you can.

I'll send a patch for the test code I have made, and put bswap.h on hold for a week
(to be able to continue with t5601/connect.c)

/Torsten

Re: htonll, ntohll

[Ramsay Jones]

On 31/10/13 13:24, Torsten Bögershausen wrote:
> On 2013-10-30 22.07, Ramsay Jones wrote:
[ ... ]
>> Yep, this was the first thing I did as well! ;-) (*late* last night)
>>
>> I haven't had time today to look into fixing up the msvc build
>> (or a complete re-write), so I look forward to seeing your solution.
>> (do you have msvc available? - or do you want me to look at fixing
>> it? maybe in a day or two?)
>>
> Ramsay,
> I don't have msvc, so feel free to go ahead, as much as you can.
> 
> I'll send a patch for the test code I have made, and put bswap.h on hold for a week
> (to be able to continue with t5601/connect.c)

Unfortunately, I haven't had much time to look into this.

I do have a patch (given below) that works on Linux, cygwin,
MinGW and msvc. However, the msvc build is still broken (as a
result of _other_ commits in this 'jk/pack-bitmap' branch; as
well as the use of a VLA in another commit).

So, I still have work to do! :(

Anyway, I thought I would send what I have, so you can take a look.
Note, that I don't have an big-endian machine to test this on, so
YMMV. Indeed, the *only* testing I have done is to run the test added
by this branch (t5310-pack-bitmaps.sh), which works on Linux, cygwin
and MinGW.

[Note: I have never particularly liked htons, htonl et.al., so adding
these htonll/ntohll functions doesn't thrill me! :-D For example see
this post[1], which echo's my sentiments exactly.]

HTH

ATB,
Ramsay Jones

[1] http://commandcenter.blogspot.co.uk/2012/04/byte-order-fallacy.html

-- >8 --
Subject: [PATCH] compat/bswap.h: Fix build on cygwin, MinGW and msvc

---
 compat/bswap.h | 97 ++++++++++++++++++++++++++++++++++++++++------------------
 1 file changed, 68 insertions(+), 29 deletions(-)

diff --git a/compat/bswap.h b/compat/bswap.h
index ea1a9ed..c18a78e 100644
--- a/compat/bswap.h
+++ b/compat/bswap.h
@@ -17,7 +17,20 @@ static inline uint32_t default_swab32(uint32_t val)
 		((val & 0x000000ff) << 24));
 }
 
+static inline uint64_t default_bswap64(uint64_t val)
+{
+	return (((val & (uint64_t)0x00000000000000ffULL) << 56) |
+		((val & (uint64_t)0x000000000000ff00ULL) << 40) |
+		((val & (uint64_t)0x0000000000ff0000ULL) << 24) |
+		((val & (uint64_t)0x00000000ff000000ULL) <<  8) |
+		((val & (uint64_t)0x000000ff00000000ULL) >>  8) |
+		((val & (uint64_t)0x0000ff0000000000ULL) >> 24) |
+		((val & (uint64_t)0x00ff000000000000ULL) >> 40) |
+		((val & (uint64_t)0xff00000000000000ULL) >> 56));
+}
+
 #undef bswap32
+#undef bswap64
 
 #if defined(__GNUC__) && (defined(__i386__) || defined(__x86_64__))
 
@@ -32,54 +45,80 @@ static inline uint32_t git_bswap32(uint32_t x)
 	return result;
 }
 
+#define bswap64 git_bswap64
+#if defined(__x86_64__)
+static inline uint64_t git_bswap64(uint64_t x)
+{
+	uint64_t result;
+	if (__builtin_constant_p(x))
+		result = default_bswap64(x);
+	else
+		__asm__("bswap %q0" : "=r" (result) : "0" (x));
+	return result;
+}
+#else
+static inline uint64_t git_bswap64(uint64_t x)
+{
+	union { uint64_t i64; uint32_t i32[2]; } tmp, result;
+	if (__builtin_constant_p(x))
+		result.i64 = default_bswap64(x);
+	else {
+		tmp.i64 = x;
+		result.i32[0] = git_bswap32(tmp.i32[1]);
+		result.i32[1] = git_bswap32(tmp.i32[0]);
+	}
+	return result.i64;
+}
+#endif
+
 #elif defined(_MSC_VER) && (defined(_M_IX86) || defined(_M_X64))
 
 #include <stdlib.h>
 
 #define bswap32(x) _byteswap_ulong(x)
+#define bswap64(x) _byteswap_uint64(x)
 
 #endif
 
-#ifdef bswap32
+#if defined(bswap32)
 
 #undef ntohl
 #undef htonl
 #define ntohl(x) bswap32(x)
 #define htonl(x) bswap32(x)
 
-#ifndef __BYTE_ORDER
-#	if defined(BYTE_ORDER) && defined(LITTLE_ENDIAN) && defined(BIG_ENDIAN)
-#		define __BYTE_ORDER BYTE_ORDER
-#		define __LITTLE_ENDIAN LITTLE_ENDIAN
-#		define __BIG_ENDIAN BIG_ENDIAN
-#	else
-#		error "Cannot determine endianness"
-#	endif
+#endif
+
+#if defined(bswap64)
+
+#undef ntohll
+#undef htonll
+#define ntohll(x) bswap64(x)
+#define htonll(x) bswap64(x)
+
+#else
+
+#undef ntohll
+#undef htonll
+
+#if !defined(__BYTE_ORDER)
+# if defined(BYTE_ORDER) && defined(LITTLE_ENDIAN) && defined(BIG_ENDIAN)
+#  define __BYTE_ORDER BYTE_ORDER
+#  define __LITTLE_ENDIAN LITTLE_ENDIAN
+#  define __BIG_ENDIAN BIG_ENDIAN
+# endif
+#endif
+
+#if !defined(__BYTE_ORDER)
+# error "Cannot determine endianness"
 #endif
 
 #if __BYTE_ORDER == __BIG_ENDIAN
 # define ntohll(n) (n)
 # define htonll(n) (n)
-#elif __BYTE_ORDER == __LITTLE_ENDIAN
-#	if defined(__GNUC__) && defined(__GLIBC__)
-#		include <byteswap.h>
-#	else /* GNUC & GLIBC */
-static inline uint64_t bswap_64(uint64_t val)
-{
-	return ((val & (uint64_t)0x00000000000000ffULL) << 56)
-		| ((val & (uint64_t)0x000000000000ff00ULL) << 40)
-		| ((val & (uint64_t)0x0000000000ff0000ULL) << 24)
-		| ((val & (uint64_t)0x00000000ff000000ULL) <<  8)
-		| ((val & (uint64_t)0x000000ff00000000ULL) >>  8)
-		| ((val & (uint64_t)0x0000ff0000000000ULL) >> 24)
-		| ((val & (uint64_t)0x00ff000000000000ULL) >> 40)
-		| ((val & (uint64_t)0xff00000000000000ULL) >> 56);
-}
-#	endif /* GNUC & GLIBC */
-#	define ntohll(n) bswap_64(n)
-#	define htonll(n) bswap_64(n)
-#else /* __BYTE_ORDER */
-#	error "Can't define htonll or ntohll!"
+#else
+# define ntohll(n) default_bswap64(n)
+# define htonll(n) default_bswap64(n)
 #endif
 
 #endif
-- 
1.8.4

Re: htonll, ntohll

[Torsten Bögershausen]

On 2013-11-05 01.00, Ramsay Jones wrote:
> On 31/10/13 13:24, Torsten Bögershausen wrote:
>> On 2013-10-30 22.07, Ramsay Jones wrote:
> [ ... ]
>>> Yep, this was the first thing I did as well! ;-) (*late* last night)
>>>
>>> I haven't had time today to look into fixing up the msvc build
>>> (or a complete re-write), so I look forward to seeing your solution.
>>> (do you have msvc available? - or do you want me to look at fixing
>>> it? maybe in a day or two?)
>>>
>> Ramsay,
>> I don't have msvc, so feel free to go ahead, as much as you can.
>>
>> I'll send a patch for the test code I have made, and put bswap.h on hold for a week
>> (to be able to continue with t5601/connect.c)
> 
> Unfortunately, I haven't had much time to look into this.
> 
> I do have a patch (given below) that works on Linux, cygwin,
> MinGW and msvc. However, the msvc build is still broken (as a
> result of _other_ commits in this 'jk/pack-bitmap' branch; as
> well as the use of a VLA in another commit).
> 
> So, I still have work to do! :(
> 
> Anyway, I thought I would send what I have, so you can take a look.
> Note, that I don't have an big-endian machine to test this on, so
> YMMV. Indeed, the *only* testing I have done is to run the test added
> by this branch (t5310-pack-bitmaps.sh), which works on Linux, cygwin
> and MinGW.
> 
> [Note: I have never particularly liked htons, htonl et.al., so adding
> these htonll/ntohll functions doesn't thrill me! :-D For example see
> this post[1], which echo's my sentiments exactly.]
> 
> HTH
> 
> ATB,
> Ramsay Jones
> 
> [1] http://commandcenter.blogspot.co.uk/2012/04/byte-order-fallacy.html
> 
> -- >8 --
> Subject: [PATCH] compat/bswap.h: Fix build on cygwin, MinGW and msvc
> 
> ---
>  compat/bswap.h | 97 ++++++++++++++++++++++++++++++++++++++++------------------
>  1 file changed, 68 insertions(+), 29 deletions(-)
> 
> diff --git a/compat/bswap.h b/compat/bswap.h
> index ea1a9ed..c18a78e 100644
> --- a/compat/bswap.h
> +++ b/compat/bswap.h
> @@ -17,7 +17,20 @@ static inline uint32_t default_swab32(uint32_t val)
>  		((val & 0x000000ff) << 24));
>  }
>  
> +static inline uint64_t default_bswap64(uint64_t val)
> +{
> +	return (((val & (uint64_t)0x00000000000000ffULL) << 56) |
> +		((val & (uint64_t)0x000000000000ff00ULL) << 40) |
> +		((val & (uint64_t)0x0000000000ff0000ULL) << 24) |
> +		((val & (uint64_t)0x00000000ff000000ULL) <<  8) |
> +		((val & (uint64_t)0x000000ff00000000ULL) >>  8) |
> +		((val & (uint64_t)0x0000ff0000000000ULL) >> 24) |
> +		((val & (uint64_t)0x00ff000000000000ULL) >> 40) |
> +		((val & (uint64_t)0xff00000000000000ULL) >> 56));
> +}
> +
>  #undef bswap32
> +#undef bswap64
>  
>  #if defined(__GNUC__) && (defined(__i386__) || defined(__x86_64__))
>  
> @@ -32,54 +45,80 @@ static inline uint32_t git_bswap32(uint32_t x)
>  	return result;
>  }
>  
> +#define bswap64 git_bswap64
> +#if defined(__x86_64__)
> +static inline uint64_t git_bswap64(uint64_t x)
> +{
> +	uint64_t result;
> +	if (__builtin_constant_p(x))
> +		result = default_bswap64(x);
> +	else
> +		__asm__("bswap %q0" : "=r" (result) : "0" (x));
> +	return result;
> +}
> +#else
> +static inline uint64_t git_bswap64(uint64_t x)
> +{
> +	union { uint64_t i64; uint32_t i32[2]; } tmp, result;
> +	if (__builtin_constant_p(x))
> +		result.i64 = default_bswap64(x);
> +	else {
> +		tmp.i64 = x;
> +		result.i32[0] = git_bswap32(tmp.i32[1]);
> +		result.i32[1] = git_bswap32(tmp.i32[0]);
> +	}
> +	return result.i64;
> +}
> +#endif
> +
>  #elif defined(_MSC_VER) && (defined(_M_IX86) || defined(_M_X64))
>  
>  #include <stdlib.h>
>  
>  #define bswap32(x) _byteswap_ulong(x)
> +#define bswap64(x) _byteswap_uint64(x)
>  
>  #endif
>  
> -#ifdef bswap32
> +#if defined(bswap32)
>  
>  #undef ntohl
>  #undef htonl
>  #define ntohl(x) bswap32(x)
>  #define htonl(x) bswap32(x)
>  
> -#ifndef __BYTE_ORDER
> -#	if defined(BYTE_ORDER) && defined(LITTLE_ENDIAN) && defined(BIG_ENDIAN)
> -#		define __BYTE_ORDER BYTE_ORDER
> -#		define __LITTLE_ENDIAN LITTLE_ENDIAN
> -#		define __BIG_ENDIAN BIG_ENDIAN
> -#	else
> -#		error "Cannot determine endianness"
> -#	endif
> +#endif
> +
> +#if defined(bswap64)
> +
> +#undef ntohll
> +#undef htonll
> +#define ntohll(x) bswap64(x)
> +#define htonll(x) bswap64(x)
> +
> +#else
> +
> +#undef ntohll
> +#undef htonll
> +
> +#if !defined(__BYTE_ORDER)
> +# if defined(BYTE_ORDER) && defined(LITTLE_ENDIAN) && defined(BIG_ENDIAN)
> +#  define __BYTE_ORDER BYTE_ORDER
> +#  define __LITTLE_ENDIAN LITTLE_ENDIAN
> +#  define __BIG_ENDIAN BIG_ENDIAN
> +# endif
> +#endif
> +
> +#if !defined(__BYTE_ORDER)
> +# error "Cannot determine endianness"
>  #endif
>  
>  #if __BYTE_ORDER == __BIG_ENDIAN
>  # define ntohll(n) (n)
>  # define htonll(n) (n)
> -#elif __BYTE_ORDER == __LITTLE_ENDIAN
> -#	if defined(__GNUC__) && defined(__GLIBC__)
> -#		include <byteswap.h>
> -#	else /* GNUC & GLIBC */
> -static inline uint64_t bswap_64(uint64_t val)
> -{
> -	return ((val & (uint64_t)0x00000000000000ffULL) << 56)
> -		| ((val & (uint64_t)0x000000000000ff00ULL) << 40)
> -		| ((val & (uint64_t)0x0000000000ff0000ULL) << 24)
> -		| ((val & (uint64_t)0x00000000ff000000ULL) <<  8)
> -		| ((val & (uint64_t)0x000000ff00000000ULL) >>  8)
> -		| ((val & (uint64_t)0x0000ff0000000000ULL) >> 24)
> -		| ((val & (uint64_t)0x00ff000000000000ULL) >> 40)
> -		| ((val & (uint64_t)0xff00000000000000ULL) >> 56);
> -}
> -#	endif /* GNUC & GLIBC */
> -#	define ntohll(n) bswap_64(n)
> -#	define htonll(n) bswap_64(n)
> -#else /* __BYTE_ORDER */
> -#	error "Can't define htonll or ntohll!"
> +#else
> +# define ntohll(n) default_bswap64(n)
> +# define htonll(n) default_bswap64(n)
>  #endif
>  
>  #endif
> 
I have had time to test it, works on Linux/PPC (big endian)
and Mac OS.

What do we think about going ahead with this patch?
/Torsten

Re: htonll, ntohll

[Jakub Narębski]

W dniu 2013-11-05 01:00, Ramsay Jones pisze:

> [Note: I have never particularly liked htons, htonl et.al., so adding
> these htonll/ntohll functions doesn't thrill me! :-D For example see
> this post[1], which echo's my sentiments exactly.]
>
> HTH
>
> ATB,
> Ramsay Jones
>
> [1] http://commandcenter.blogspot.co.uk/2012/04/byte-order-fallacy.html

Errr... htonl is about host to network order, and not about big- or
little-endianness of architecture.  The macros are good, its their
implementation that might fail [1].

-- 
Jakub Narębski

Re: htonll, ntohll

[Andreas Ericsson]

On 2013-11-05 01:00, Ramsay Jones wrote:
>
> [Note: I have never particularly liked htons, htonl et.al., so adding
> these htonll/ntohll functions doesn't thrill me! :-D For example see
> this post[1], which echo's my sentiments exactly.]
>

That post actually contradicts your statement, as it clearly states
that "someone at Adobe figured out about byte order and there would
have been no problems transferring files between (big-endian and
little-endian) machines... if the people at Adobe wrote proper code
to encode and decode their files".

htonl(), ntohl(), htons(), ntohs() are those "encode" and "decode"
functions. If you or the author of the post you linked think otherwise,
you're misinformed and need to learn what encoding and decoding means.

-- 
Andreas Ericsson                   andreas.ericsson@op5.se
OP5 AB                             www.op5.se
Tel: +46 8-230225                  Fax: +46 8-230231

Considering the successes of the wars on alcohol, poverty, drugs and
terror, I think we should give some serious thought to declaring war
on peace.