* Re: [PATCH] setup: recognize bare repositories with packed-refs
From: Junio C Hamano @ 2023-11-19 23:24 UTC (permalink / raw)
To: Glen Choo, Josh Steadmon; +Cc: git, Adam Majer
In-Reply-To: <20231117203253.21143-1-adamm@zombino.com>
Adam Majer <adamm@zombino.com> writes:
> In a garbage collected bare git repository, the refs/ subdirectory is
> empty. In use-cases when such a repository is directly added into
> another repository, it no longer is detected as valid.
Josh & Glen [*], isn't this a layout that we explicitly discourage and
eventually plan to forbid anyway?
*1* who worked on e35f202b (setup: trace bare repository setups, 2023-05-01)
^ permalink raw reply
* Re: [PATCH] Fix a typo in `each_file_in_pack_dir_fn()`'s declaration
From: Junio C Hamano @ 2023-11-19 23:15 UTC (permalink / raw)
To: Johannes Schindelin via GitGitGadget; +Cc: git, Johannes Schindelin
In-Reply-To: <pull.1614.git.1700226915859.gitgitgadget@gmail.com>
"Johannes Schindelin via GitGitGadget" <gitgitgadget@gmail.com>
writes:
> Subject: Re: [PATCH] Fix a typo in `each_file_in_pack_dir_fn()`'s declaration
Let's have "packfile.[ch]: " before the title to tell what area the
helper function is about.
> From: Johannes Schindelin <johannes.schindelin@gmx.de>
>
> One parameter is called `file_pach`. On the face of it, this looks as if
> it was supposed to talk about a `path` instead of a `pach`.
>
> However, looking at the way this callback is called, it gets fed the
> `d_name` from a directory entry, which provides just the file name, not
> the full path. Therefore, let's fix this by calling the parameter
> `file_name` instead.
> ...
> typedef void each_file_in_pack_dir_fn(const char *full_path, size_t full_path_len,
> - const char *file_pach, void *data);
> + const char *file_name, void *data);
Very good observation. We form a pathname to a file in a
"objects/pack/" subdirectory by concatenating this parameter after
the full_path/full_path_len parameter, which has the path to that
subdirectory, so "file_name" definitely is a much better name.
The "full_path" that does not say full path to what directory may
have room for improvement ("leading_path" or even "packdir"), but
that's OK.
Thanks for spotting.
> void for_each_file_in_pack_dir(const char *objdir,
> each_file_in_pack_dir_fn fn,
> void *data);
>
> base-commit: cfb8a6e9a93adbe81efca66e6110c9b4d2e57169
^ permalink raw reply
* Re: [PATCH] merge-file: add --diff-algorithm option
From: Antonin Delpeuch @ 2023-11-19 19:29 UTC (permalink / raw)
To: phillip.wood, git; +Cc: Elijah Newren
In-Reply-To: <de04aec0-a195-45da-8951-bb30f2a629a3@gmail.com>
Hi Phillip,
Thank you so much for taking the time to review this!
On 19/11/2023 17:43, Phillip Wood wrote:
> I cannot comment on this particular use but I think in general calling
> "git merge-file" from a custom merge driver is perfectly sensible.
> Have you tested your driver with this patch to see if you get better
> results with the histogram diff algorithm?
Yes, I can confirm that the results are better in my use case indeed.
> I can see there's an argument for changing the default algorithm of
> "git merge-file" to match what "ort" uses. I know Elijah found the
> histogram algorithm gave better results in his testing when he was
> developing "ort". While it would be a breaking change if on the
> average the new default gives better conflicts it might be worth it.
> This patch would mean that someone wanting to use the "myers"
> algorithm could still do so.
Agreed. I would be happy to submit a follow-up patch to change the
default. Or would you prefer to have it in the same patch (as a separate
commit)? I was worried this would make my patch less likely to get merged.
> It would be nice to see some tests for this patch, ideally using a
> test case that gives different conflicts for "myers" and "histogram".
> We could add the other options later if there is a demand.
Will do.
> Perhaps we could list the available algorithms here so the user does
> not have to go searching for them in another man page.
This part is copied from "Documentation/merge-strategies.txt", which
redirects to the manual for git-diff in the same way. I assume it was
done so that whenever a new diff algorithm is introduced, it only needs
documenting in one place. But I agree it is definitely more
user-friendly to list the algorithms directly. Should I change the
documentation of merge strategies in the same way?
Best wishes,
Antonin
^ permalink raw reply
* Re: [PATCH] merge-file: add --diff-algorithm option
From: Phillip Wood @ 2023-11-19 16:43 UTC (permalink / raw)
To: Antonin Delpeuch, git; +Cc: Elijah Newren
In-Reply-To: <653b08fd-2df3-4a7a-8082-fdb809e87784@delpeuch.eu>
Hi Antonin
On 17/11/2023 21:42, Antonin Delpeuch wrote:
> Hi all,
>
> Here a few more thoughts about this patch, to explain what brought me to
> needing that. If this need is misguided, perhaps you could redirect me
> to a better solution.
>
> I am writing a custom merge driver for Java files. This merge driver
> internally calls git-merge-file and then solves the merge conflicts
> which only consist of import statements (there might be cases where it
> gets it wrong, but I can then use other tools to cleanup those import
> statements). When testing this, I noticed that the merge driver
> performed more poorly on other sorts of conflicts, compared to the
> standard "ort" merge strategy. This is because "ort" uses the
> "histogram" diff algorithm, which gives better results than the "myers"
> diff algorithm that merge-file uses.
I cannot comment on this particular use but I think in general calling
"git merge-file" from a custom merge driver is perfectly sensible. Have
you tested your driver with this patch to see if you get better results
with the histogram diff algorithm?
> Intuitively, if "histogram" is the default diff algorithm used by "git
> merge", then it would also make sense to have the same default for "git
> merge-file", but I assume that changing this default could be considered
> a bad breaking change. So I thought that making this diff algorithm
> configurable would be an acceptable move, hence my patch.
I can see there's an argument for changing the default algorithm of "git
merge-file" to match what "ort" uses. I know Elijah found the histogram
algorithm gave better results in his testing when he was developing
"ort". While it would be a breaking change if on the average the new
default gives better conflicts it might be worth it. This patch would
mean that someone wanting to use the "myers" algorithm could still do so.
> Of course, the diffing could be configured in other ways, for instance
> with its handling of whitespace or EOL (similarly to what the "git-diff"
> command offers). I think those options would definitely be worth
> exposing in merge-file as well. If you think this makes sense, then I
> would be happy to work on a new version of this patch which would
> attempt to include all the relevant options. I could also try to add the
> corresponding tests.
It would be nice to see some tests for this patch, ideally using a test
case that gives different conflicts for "myers" and "histogram". We
could add the other options later if there is a demand.
Best Wishes
Phillip
> But perhaps my need is misguided? Could it be that I should not be
> writing a custom merge driver, but instead use another extension point
> to only process the conflicting hunks after execution of the existing
> merge driver? I couldn't find such an extension point, but it can well
> be that I missed it.
>
> Thank you,
>
> Antonin
>
>
^ permalink raw reply
* Re: [PATCH] merge-file: add --diff-algorithm option
From: Phillip Wood @ 2023-11-19 16:42 UTC (permalink / raw)
To: Antonin Delpeuch via GitGitGadget, git; +Cc: Antonin Delpeuch
In-Reply-To: <pull.1606.git.git.1699480494355.gitgitgadget@gmail.com>
Hi Antonin
On 08/11/2023 21:54, Antonin Delpeuch via GitGitGadget wrote:
> From: Antonin Delpeuch <antonin@delpeuch.eu>
>
> This makes it possible to use other diff algorithms than the 'myers'
> default algorithm, when using the 'git merge-file' command.
I think being able to select the diff algorithm is reasonable. I might
be nice to mention the use of "git merge-file" in custom merge drivers
as a motivation in the commit message.
> Signed-off-by: Antonin Delpeuch <antonin@delpeuch.eu>
> ---
> merge-file: add --diff-algorithm option
>
> Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-git-1606%2Fwetneb%2Fmerge_file_configurable_diff_algorithm-v1
> Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-git-1606/wetneb/merge_file_configurable_diff_algorithm-v1
> Pull-Request: https://github.com/git/git/pull/1606
>
> Documentation/git-merge-file.txt | 5 +++++
> builtin/merge-file.c | 28 ++++++++++++++++++++++++++++
> 2 files changed, 33 insertions(+)
>
> diff --git a/Documentation/git-merge-file.txt b/Documentation/git-merge-file.txt
> index 6a081eacb72..917535217c1 100644
> --- a/Documentation/git-merge-file.txt
> +++ b/Documentation/git-merge-file.txt
> @@ -92,6 +92,11 @@ object store and the object ID of its blob is written to standard output.
> Instead of leaving conflicts in the file, resolve conflicts
> favouring our (or their or both) side of the lines.
>
> +--diff-algorithm <algorithm>::
> + Use a different diff algorithm while merging, which can help
> + avoid mismerges that occur due to unimportant matching lines
> + (such as braces from distinct functions). See also
> + linkgit:git-diff[1] `--diff-algorithm`.
Perhaps we could list the available algorithms here so the user does not
have to go searching for them in another man page.
> EXAMPLES
> --------
> diff --git a/builtin/merge-file.c b/builtin/merge-file.c
> index 832c93d8d54..1f987334a31 100644
> --- a/builtin/merge-file.c
> +++ b/builtin/merge-file.c
> @@ -1,5 +1,6 @@
> #include "builtin.h"
> #include "abspath.h"
> +#include "diff.h"
> #include "hex.h"
> #include "object-name.h"
> #include "object-store.h"
> @@ -28,6 +29,30 @@ static int label_cb(const struct option *opt, const char *arg, int unset)
> return 0;
> }
>
> +static int set_diff_algorithm(xpparam_t *xpp,
> + const char *alg)
> +{
> + long diff_algorithm = parse_algorithm_value(alg);
> + if (diff_algorithm < 0)
> + return -1;
> + xpp->flags = (xpp->flags & ~XDF_DIFF_ALGORITHM_MASK) | diff_algorithm;
> + return 0;
> +}
> +
> +static int diff_algorithm_cb(const struct option *opt,
> + const char *arg, int unset)
> +{
> + xpparam_t *xpp = opt->value;
> +
> + BUG_ON_OPT_NEG(unset);
> +
> + if (set_diff_algorithm(xpp, arg))
> + return error(_("option diff-algorithm accepts \"myers\", "
> + "\"minimal\", \"patience\" and \"histogram\""));
> +
> + return 0;
> +}
> +
> int cmd_merge_file(int argc, const char **argv, const char *prefix)
> {
> const char *names[3] = { 0 };
> @@ -48,6 +73,9 @@ int cmd_merge_file(int argc, const char **argv, const char *prefix)
> XDL_MERGE_FAVOR_THEIRS),
> OPT_SET_INT(0, "union", &xmp.favor, N_("for conflicts, use a union version"),
> XDL_MERGE_FAVOR_UNION),
> + OPT_CALLBACK_F(0, "diff-algorithm", &xmp.xpp, N_("<algorithm>"),
> + N_("choose a diff algorithm"),
> + PARSE_OPT_NONEG, diff_algorithm_cb),
> OPT_INTEGER(0, "marker-size", &xmp.marker_size,
> N_("for conflicts, use this marker size")),
> OPT__QUIET(&quiet, N_("do not warn about conflicts")),
This patch looks sensible to me, it would be nice to have some tests though.
Best Wishes
Phillip
> base-commit: 98009afd24e2304bf923a64750340423473809ff
^ permalink raw reply
* Re: [PATCH v4] subtree: fix split processing with multiple subtrees present
From: Christian Couder @ 2023-11-18 11:28 UTC (permalink / raw)
To: Zach FettersMoore via GitGitGadget; +Cc: git, Zach FettersMoore
In-Reply-To: <pull.1587.v4.git.1698347871200.gitgitgadget@gmail.com>
On Thu, Oct 26, 2023 at 9:18 PM Zach FettersMoore via GitGitGadget
<gitgitgadget@gmail.com> wrote:
>
> From: Zach FettersMoore <zach.fetters@apollographql.com>
>
> When there are multiple subtrees present in a repository and they are
> all using 'git subtree split', the 'split' command can take a
> significant (and constantly growing) amount of time to run even when
> using the '--rejoin' flag. This is due to the fact that when processing
> commits to determine the last known split to start from when looking
> for changes, if there has been a split/merge done from another subtree
> there will be 2 split commits, one mainline and one subtree, for the
> second subtree that are part of the processing. The non-mainline
> subtree split commit will cause the processing to always need to search
> the entire history of the given subtree as part of its processing even
> though those commits are totally irrelevant to the current subtree
> split being run.
Thanks for your continued work on this!
I am not familiar with git subtree so I might miss obvious things. On
the other hand, my comments might help increase a bit the number of
people who could review this patch.
> In the diagram below, 'M' represents the mainline repo branch, 'A'
> represents one subtree, and 'B' represents another. M3 and B1 represent
> a split commit for subtree B that was created from commit M4. M2 and A1
> represent a split commit made from subtree A that was also created
> based on changes back to and including M4. M1 represents new changes to
> the repo, in this scenario if you try to run a 'git subtree split
> --rejoin' for subtree B, commits M1, M2, and A1, will be included in
> the processing of changes for the new split commit since the last
> split/rejoin for subtree B was at M3. The issue is that by having A1
> included in this processing the command ends up needing to processing
> every commit down tree A even though none of that is needed or relevant
> to the current command and result.
>
> M1
> | \ \
> M2 | |
> | A1 |
> M3 | |
> | | B1
> M4 | |
About the above, Junio already commented the following:
-> The above paragraph explains which different things you drew in the
-> diagram are representing, but it is not clear how they relate to
-> each other. Do they for example depict parent-child commit
-> relationship? What are the wide gaps between these three tracks and
-> what are the short angled lines leaning to the left near the tip?
-> Is the time/topology flowing from bottom to top?
and it doesn't look like you have addressed that comment.
When you say "M3 and B1 represent a split commit for subtree B that
was created from commit M4." I am not sure what it means exactly.
Could you give example commands that could have created the M3 and B1
commits?
> So this commit makes a change to the processing of commits for the split
> command in order to ignore non-mainline commits from other subtrees such
> as A1 in the diagram by adding a new function
> 'should_ignore_subtree_commit' which is called during
> 'process_split_commit'. This allows the split/rejoin processing to still
> function as expected but removes all of the unnecessary processing that
> takes place currently which greatly inflates the processing time.
Could you tell a bit more what kind of processing time reduction is or
would be possible on what kind of repo? Have you benchmark-ed or just
timed this somehow on one of your repos or better on an open source
repo (so that we could reproduce if we wanted)?
> Added a test to validate that the proposed fix
> solves the issue.
>
> The test accomplishes this by checking the output
> of the split command to ensure the output from
> the progress of 'process_split_commit' function
> that represents the 'extracount' of commits
> processed does not increment.
Does not increment compared to what?
> This was tested against the original functionality
> to show the test failed, and then with this fix
> to show the test passes.
>
> This illustrated that when using multiple subtrees,
> A and B, when doing a split on subtree B, the
> processing does not traverse the entire history
> of subtree A which is unnecessary and would cause
> the 'extracount' of processed commits to climb
> based on the number of commits in the history of
> subtree A.
Does this mean that the test checks that the extracount is the same
when subtree A exists as when it doesn't exist?
[...]
> contrib/subtree/git-subtree.sh | 29 ++++++++++++++++++++-
> contrib/subtree/t/t7900-subtree.sh | 42 ++++++++++++++++++++++++++++++
> 2 files changed, 70 insertions(+), 1 deletion(-)
>
> diff --git a/contrib/subtree/git-subtree.sh b/contrib/subtree/git-subtree.sh
> index e0c5d3b0de6..e69991a9d80 100755
> --- a/contrib/subtree/git-subtree.sh
> +++ b/contrib/subtree/git-subtree.sh
> @@ -778,6 +778,20 @@ ensure_valid_ref_format () {
> die "fatal: '$1' does not look like a ref"
> }
>
> +# Usage: check if a commit from another subtree should be
> +# ignored from processing for splits
> +should_ignore_subtree_split_commit () {
Maybe adding:
assert test $# = 1
local rev="$1"
here, and using $rev instead of $1 in this function could make things
a bit clearer and similar to what is done in other functions.
> + if test -n "$(git log -1 --grep="git-subtree-dir:" $1)"
> + then
> + if test -z "$(git log -1 --grep="git-subtree-mainline:" $1)" &&
> + test -z "$(git log -1 --grep="git-subtree-dir: $arg_prefix$" $1)"
> + then
> + return 0
> + fi
> + fi
> + return 1
> +}
The above doesn't seem to be properly indented. We use tabs not spaces.
> # Usage: process_split_commit REV PARENTS
> process_split_commit () {
> assert test $# = 2
> @@ -963,7 +977,20 @@ cmd_split () {
> eval "$grl" |
> while read rev parents
> do
> - process_split_commit "$rev" "$parents"
> + if should_ignore_subtree_split_commit "$rev"
> + then
> + continue
> + fi
> + parsedParents=''
It seems to me that we name variables "parsed_parents" (or sometimes
"parsedparents") rather than "parsedParents".
> + for parent in $parents
> + do
> + should_ignore_subtree_split_commit "$parent"
> + if test $? -eq 1
I think the 2 lines above could be replaced by:
+ if ! should_ignore_subtree_split_commit "$parent"
> + then
> + parsedParents+="$parent "
It doesn't seem to me that we use "+=" much in our shell scripts.
https://www.shellcheck.net/ emits the following:
(warning): In POSIX sh, += is undefined.
so I guess we don't use it because it's not available in some usual shells.
(I haven't checked the script with https://www.shellcheck.net/ before
and after your patch, but it might help avoid bash-isms and such
issues.)
> + fi
> + done
> + process_split_commit "$rev" "$parsedParents"
> done || exit $?
It looks like we use "exit $?" a lot in git-subtree.sh while we use
just "exit" most often elsewhere. Not sure why.
> latest_new=$(cache_get latest_new) || exit $?
> diff --git a/contrib/subtree/t/t7900-subtree.sh b/contrib/subtree/t/t7900-subtree.sh
> index 49a21dd7c9c..87d59afd761 100755
> --- a/contrib/subtree/t/t7900-subtree.sh
> +++ b/contrib/subtree/t/t7900-subtree.sh
> @@ -385,6 +385,48 @@ test_expect_success 'split sub dir/ with --rejoin' '
> )
> '
>
> +test_expect_success 'split with multiple subtrees' '
> + subtree_test_create_repo "$test_count" &&
> + subtree_test_create_repo "$test_count/subA" &&
> + subtree_test_create_repo "$test_count/subB" &&
> + test_create_commit "$test_count" main1 &&
> + test_create_commit "$test_count/subA" subA1 &&
> + test_create_commit "$test_count/subA" subA2 &&
> + test_create_commit "$test_count/subA" subA3 &&
> + test_create_commit "$test_count/subB" subB1 &&
> + (
> + cd "$test_count" &&
> + git fetch ./subA HEAD &&
> + git subtree add --prefix=subADir FETCH_HEAD
> + ) &&
> + (
> + cd "$test_count" &&
> + git fetch ./subB HEAD &&
> + git subtree add --prefix=subBDir FETCH_HEAD
> + ) &&
> + test_create_commit "$test_count" subADir/main-subA1 &&
> + test_create_commit "$test_count" subBDir/main-subB1 &&
> + (
> + cd "$test_count" &&
> + git subtree split --prefix=subADir --squash --rejoin -m "Sub A Split 1"
> + ) &&
Not sure why there are so many sub-shells used, and why the -C option
is not used instead to tell Git to work in a subdirectory. I guess you
copied what most existing (old) tests in this test script do.
For example perhaps the 4 line above could be replaced by just:
+ git -C "$test_count" subtree split --prefix=subADir
--squash --rejoin -m "Sub A Split 1" &&
> + (
> + cd "$test_count" &&
> + git subtree split --prefix=subBDir --squash --rejoin -m "Sub B Split 1"
> + ) &&
> + test_create_commit "$test_count" subADir/main-subA2 &&
> + test_create_commit "$test_count" subBDir/main-subB2 &&
> + (
> + cd "$test_count" &&
> + git subtree split --prefix=subADir --squash --rejoin -m "Sub A Split 2"
> + ) &&
> + (
> + cd "$test_count" &&
> + test "$(git subtree split --prefix=subBDir --squash --rejoin \
> + -d -m "Sub B Split 1" 2>&1 | grep -w "\[1\]")" = ""
> + )
> +'
It's not clear to me what the test is doing. Maybe you could split it
into 2 tests. Perhaps one setting up a repo with multiple subtrees and
one checking that a new split ignores other subtree split commits.
Perhaps adding a few comments would help too.
Best,
Christian.
^ permalink raw reply
* Re: Migration of git-scm.com to a static web site: ready for review/testing
From: Johannes Schindelin @ 2023-11-18 9:46 UTC (permalink / raw)
To: Johannes Sixt; +Cc: git, Matt Burke, Victoria Dye, Matthias Aßhauer
In-Reply-To: <39f863b9-fc81-43ca-93f5-89e341e8615d@kdbg.org>
Hi Hannes,
Yes, keeping existing deep links working is very much a goal of this work.
Thank you,
Johannes
-------- Original Message --------
From: Johannes Sixt <j6t@kdbg.org>
Sent: November 18, 2023 10:41:11 AM GMT+01:00
To: Johannes Schindelin <Johannes.Schindelin@gmx.de>
Cc: git@vger.kernel.org, Matt Burke <spraints@gmail.com>, Victoria Dye <vdye@github.com>, "Matthias Aßhauer" <mha1993@live.de>
Subject: Re: Migration of git-scm.com to a static web site: ready for review/testing
Am 17.11.23 um 14:25 schrieb Johannes Schindelin:
> Hi,
>
> the idea of migrating https://git-scm.com/ from a Rails app to a static
> site has been discussed several times on this list in the past.
>
> Thanks to the heroic, multi-year efforts of Matt Burke, Victoria Dye and
> Matthias Aßhauer, there is now a Pull Request, ready for review:
> https://github.com/git/git-scm.com/pull/1804
>
> This Pull Request is not for the faint of heart, mainly because of the
> sheer amount of generated pages that are committed to the repository (such
> as the book, the manual pages, etc, a design decision necessary to run
> this as a static website).
>
> These pages are generated by GitHub workflows that are intended to run on
> a schedule, and the scripts that generate them are part of the Pull
> Request. For that reason, I do not consider it necessary to review those
> generated pages, those reviews have been done in the upstream sources from
> which the pages were generated.
>
> At this point, the patches are fairly robust and I am mainly hoping for
> help with verifying that the static site works as intended, that existing
> links will continue to work with the new site (essentially, find obscure
> references to the existing website, then insert `git.github.io/` in the
> URL and verify that it works as intended).
>
> To that end, I deployed this branch to GitHub Pages so that anyone
> interested (hopefully many!) can have a look at
> https://git.github.io/git-scm.com/ and compare to the existing
> https://git-scm.com/.
When a transition to static pages happens, an important aspect is that
external links that point into git-scm.com must not be invalidated.
There are many such links in Stackoverflow answers, for example.
I checked one link:
https://git-scm.com/docs/git-rebase#Documentation/git-rebase.txt--r
https://git.github.io/git-scm.com/docs/git-rebase#Documentation/git-rebase.txt--r
and it is looking very good. Thank you! I assume that keeping links
working is not just a happy accident, but part of the plan.
-- Hannes
^ permalink raw reply
* Re: Migration of git-scm.com to a static web site: ready for review/testing
From: Johannes Sixt @ 2023-11-18 9:41 UTC (permalink / raw)
To: Johannes Schindelin; +Cc: git, Matt Burke, Victoria Dye, Matthias Aßhauer
In-Reply-To: <6f7d20b4-a725-0ef9-f6d3-ff2810da9e7a@gmx.de>
Am 17.11.23 um 14:25 schrieb Johannes Schindelin:
> Hi,
>
> the idea of migrating https://git-scm.com/ from a Rails app to a static
> site has been discussed several times on this list in the past.
>
> Thanks to the heroic, multi-year efforts of Matt Burke, Victoria Dye and
> Matthias Aßhauer, there is now a Pull Request, ready for review:
> https://github.com/git/git-scm.com/pull/1804
>
> This Pull Request is not for the faint of heart, mainly because of the
> sheer amount of generated pages that are committed to the repository (such
> as the book, the manual pages, etc, a design decision necessary to run
> this as a static website).
>
> These pages are generated by GitHub workflows that are intended to run on
> a schedule, and the scripts that generate them are part of the Pull
> Request. For that reason, I do not consider it necessary to review those
> generated pages, those reviews have been done in the upstream sources from
> which the pages were generated.
>
> At this point, the patches are fairly robust and I am mainly hoping for
> help with verifying that the static site works as intended, that existing
> links will continue to work with the new site (essentially, find obscure
> references to the existing website, then insert `git.github.io/` in the
> URL and verify that it works as intended).
>
> To that end, I deployed this branch to GitHub Pages so that anyone
> interested (hopefully many!) can have a look at
> https://git.github.io/git-scm.com/ and compare to the existing
> https://git-scm.com/.
When a transition to static pages happens, an important aspect is that
external links that point into git-scm.com must not be invalidated.
There are many such links in Stackoverflow answers, for example.
I checked one link:
https://git-scm.com/docs/git-rebase#Documentation/git-rebase.txt--r
https://git.github.io/git-scm.com/docs/git-rebase#Documentation/git-rebase.txt--r
and it is looking very good. Thank you! I assume that keeping links
working is not just a happy accident, but part of the plan.
-- Hannes
^ permalink raw reply
* Re: Migration of git-scm.com to a static web site: ready for review/testing
From: Todd Zullinger @ 2023-11-18 2:57 UTC (permalink / raw)
To: Johannes Schindelin; +Cc: git, Matt Burke, Victoria Dye, Matthias Aßhauer
In-Reply-To: <4dd9b45d-b352-d8ba-3314-96ab48f7abf7@gmx.de>
Hi Johannes,
Johannes Schindelin wrote:
>> For checking links, a tool like linkcheker[1] is very handy.
>> This is run against the local docs in the Fedora package
>> builds to catch broken links.
>
> Hmm, `linkchecker` is really slow for me, even locally.
Yeah, it took an hour and a half to run for me, both on an
old laptop and a fast server with plenty of threads,
bandwidth, and memory.
Checking the git HTML documentation takes under 30 seconds,
which is largely the only place I've used it. It has been
very helpful in catching broken links in the docs during the
build and the time is short enough that I never minded.
> Granted, the added cross-references now increase the number of hyperlinks
> to check, but after I let the program run for a bit over an hour to look
> at https://git-scm.com/ (for comparison), it is now running on the local
> build (i.e. the `public/` folder generated by Hugo, not even an HTTP
> server) for over 45 minutes and still not done:
>
> -- snip --
> [...]
> 10 threads active, 112977 links queued, 206443 links in 100001 URLs checked, runtime 48 minutes, 46 seconds
> 10 threads active, 113455 links queued, 206689 links in 100001 URLs checked, runtime 48 minutes, 52 seconds
> 10 threads active, 113829 links queued, 206874 links in 100001 URLs checked, runtime 48 minutes, 57 seconds
> 10 threads active, 114230 links queued, 207136 links in 100001 URLs checked, runtime 49 minutes, 3 seconds
> 10 threads active, 114731 links queued, 207498 links in 100001 URLs checked, runtime 49 minutes, 9 seconds
> -- snap --
I would have thought that bumping the number of threads a
lot would really help, but I ran it on a dual Xeon system
with 40 threads and it took about the same time. Perhaps I
should have increased to double or more the system processor
count.
> Maybe something is going utterly wrong because the number
> of links seems to be dramatically larger than what the
> https://git-scm.com/ reported; Maybe linkchecker broke out
> of the `public/` directory and now indexes my entire
> harddrive ;-)
Heh, hopefully not. :)
I wondered if there were circular links that it was picking
up and not de-duplicating. I may try to run it with the
--verbose option which logs all checked URLs. Maybe that
will turn up something. It sure seems like there's a _lot_
of links here.
There is a --recursion-level option which might be helpful.
The --ignore-url and/or --no-follow-url may also be useful.
Though even if it's (very) slow, it might be worth running
to flush out some initial issues before making the site
live. Letting it run in the background for a few hours is
probably less effort than fielding a number of big reports
about broken URL here and there. :)
Of course, it would be even better if it were fast enough to
run as part of the site build process to catch broken links
before each deployment, but that would need to be measured
in some relatively small number of seconds instead of the
hours it seems to take now. :/
--
Todd
^ permalink raw reply
* Re: Migration of git-scm.com to a static web site: ready for review/testing
From: Johannes Schindelin @ 2023-11-18 1:14 UTC (permalink / raw)
To: Todd Zullinger; +Cc: git, Matt Burke, Victoria Dye, Matthias Aßhauer
In-Reply-To: <ZVeUQEG5jIzKbvmT@pobox.com>
[-- Attachment #1: Type: text/plain, Size: 3330 bytes --]
Hi Todd,
On Fri, 17 Nov 2023, Todd Zullinger wrote:
> Johannes Schindelin wrote:
> > At this point, the patches are fairly robust and I am mainly hoping for
> > help with verifying that the static site works as intended, that existing
> > links will continue to work with the new site (essentially, find obscure
> > references to the existing website, then insert `git.github.io/` in the
> > URL and verify that it works as intended).
> >
> > To that end, I deployed this branch to GitHub Pages so that anyone
> > interested (hopefully many!) can have a look at
> > https://git.github.io/git-scm.com/ and compare to the existing
> > https://git-scm.com/.
>
> This is nice. Thanks to all for working on it!
😊
> For checking links, a tool like linkcheker[1] is very handy.
> This is run against the local docs in the Fedora package
> builds to catch broken links.
Hmm, `linkchecker` is really slow for me, even locally.
> I ran it against the test site and it turned up _a lot_ of
> broken links. [...]
>
> URL `ch00/ch10-git-internals'
> Name `Git Internals'
> Parent URL https://git.github.io/git-scm.com/book/tr/v2/Ek-b%C3%B6l%C3%BCm-C:-Git-Commands-Plumbing-Commands/, line 106, col 1318
> Real URL https://git.github.io/git-scm.com/book/tr/v2/Ek-b%C3%B6l%C3%BCm-C:-Git-Commands-Plumbing-Commands/ch00/ch10-git-internals
> Check time 3.303 seconds
> Size 1KB
> Result Error: 404 Not Found
Good catch. I totally forgot to take care of the cross-references!
This is now fixed, as of
https://github.com/dscho/git-scm.com/commit/e599a57b2fadf8cb01e57af23fcb929b32e94bcb
I kicked off the GitHub workflow to re-generate the books, and the updated
GitHub Pages look fine (see e.g. the parent URL mentioned above and follow
the "Pull Request Refs" link).
> Running it against a local directory of the content would be
> a lot faster, if that's an option. It's also worth bumping
> the default number of threads from 10 to increase the speed
> a bit.
>
> [1] https://linkchecker.github.io/linkchecker/
Unfortunately it is actually quite slow.
Granted, the added cross-references now increase the number of hyperlinks
to check, but after I let the program run for a bit over an hour to look
at https://git-scm.com/ (for comparison), it is now running on the local
build (i.e. the `public/` folder generated by Hugo, not even an HTTP
server) for over 45 minutes and still not done:
-- snip --
[...]
10 threads active, 112977 links queued, 206443 links in 100001 URLs checked, runtime 48 minutes, 46 seconds
10 threads active, 113455 links queued, 206689 links in 100001 URLs checked, runtime 48 minutes, 52 seconds
10 threads active, 113829 links queued, 206874 links in 100001 URLs checked, runtime 48 minutes, 57 seconds
10 threads active, 114230 links queued, 207136 links in 100001 URLs checked, runtime 49 minutes, 3 seconds
10 threads active, 114731 links queued, 207498 links in 100001 URLs checked, runtime 49 minutes, 9 seconds
-- snap --
Maybe something is going utterly wrong because the number of links seems
to be dramatically larger than what the https://git-scm.com/ reported;
Maybe linkchecker broke out of the `public/` directory and now indexes my
entire harddrive ;-)
Ciao,
Johannes
^ permalink raw reply
* Re: [PATCH] merge-file: add --diff-algorithm option
From: Antonin Delpeuch @ 2023-11-17 21:42 UTC (permalink / raw)
To: git
In-Reply-To: <pull.1606.git.git.1699480494355.gitgitgadget@gmail.com>
Hi all,
Here a few more thoughts about this patch, to explain what brought me to
needing that. If this need is misguided, perhaps you could redirect me
to a better solution.
I am writing a custom merge driver for Java files. This merge driver
internally calls git-merge-file and then solves the merge conflicts
which only consist of import statements (there might be cases where it
gets it wrong, but I can then use other tools to cleanup those import
statements). When testing this, I noticed that the merge driver
performed more poorly on other sorts of conflicts, compared to the
standard "ort" merge strategy. This is because "ort" uses the
"histogram" diff algorithm, which gives better results than the "myers"
diff algorithm that merge-file uses.
Intuitively, if "histogram" is the default diff algorithm used by "git
merge", then it would also make sense to have the same default for "git
merge-file", but I assume that changing this default could be considered
a bad breaking change. So I thought that making this diff algorithm
configurable would be an acceptable move, hence my patch.
Of course, the diffing could be configured in other ways, for instance
with its handling of whitespace or EOL (similarly to what the "git-diff"
command offers). I think those options would definitely be worth
exposing in merge-file as well. If you think this makes sense, then I
would be happy to work on a new version of this patch which would
attempt to include all the relevant options. I could also try to add the
corresponding tests.
But perhaps my need is misguided? Could it be that I should not be
writing a custom merge driver, but instead use another extension point
to only process the conflicting hunks after execution of the existing
merge driver? I couldn't find such an extension point, but it can well
be that I missed it.
Thank you,
Antonin
^ permalink raw reply
* Re: [PATCH] setup: recognize bare repositories with packed-refs
From: Adam Majer @ 2023-11-17 20:44 UTC (permalink / raw)
To: git
In-Reply-To: <20231117203253.21143-1-adamm@zombino.com>
On 2023-11-17 21:32, Adam Majer wrote:
> +test_expect_success 'remove empty refs/ subdirectory' '
> + git -C bare.git cat-file -e 60dd8ad7d8ed49005c18b7ce9f5b7a45c28df814 &&
> + test_dir_is_empty bare.git/refs/heads &&
> + test_dir_is_empty bare.git/refs/tags &&
> + rm -r bare.git/refs &&
> + GIT_DIR="$PWD"/bare.git git -C bare.git cat-file -e 60dd8ad7d8ed49005c18b7ce9f5b7a45c28df814
In the test, I've first tried to use GIT_CEILING_DIRECTORIES="$PWD" here
instead, but git went up into the parent directory anyway. I'm not sure
if this is a bug, or feature, or my misunderstanding.
- Adam
^ permalink raw reply
* [PATCH] setup: recognize bare repositories with packed-refs
From: Adam Majer @ 2023-11-17 20:25 UTC (permalink / raw)
To: git; +Cc: Adam Majer
When a garbage collected bare git repository is added to another git
repository, the refs/ subdirectory is empty. In case-cases when such a
repository is added into another repository directly, it no longer is
detected as a valid. Git doesn't preserve empty paths so refs/
subdirectory is not present in parent git. Simply creating an empty
refs/ subdirectory fixes this problem.
Looking more carefully, there are two backends to handle various refs in
git -- the files backend that uses refs/ subdirectory and the
packed-refs backend that uses packed-refs file. If references are not
found in refs/ subdirectory (or directory doesn't exist), the
packed-refs directory will be consulted. Garbage collected repository
will have all its references in packed-refs file.
To allow the use-case when packed-refs is the only source of refs and
refs/ subdirectory is simply not present, augment 'is_git_directory()'
setup function to look for packed-refs file as an alternative to refs/
subdirectory.
Signed-off-by: Adam Majer <adamm@zombino.com>
---
setup.c | 10 +++++++---
t/t6500-gc.sh | 8 ++++++++
2 files changed, 15 insertions(+), 3 deletions(-)
diff --git a/setup.c b/setup.c
index fc592dc6dd..2a6dda6ae9 100644
--- a/setup.c
+++ b/setup.c
@@ -348,7 +348,7 @@ int get_common_dir_noenv(struct strbuf *sb, const char *gitdir)
*
* - either an objects/ directory _or_ the proper
* GIT_OBJECT_DIRECTORY environment variable
- * - a refs/ directory
+ * - a refs/ directory or packed-refs file
* - either a HEAD symlink or a HEAD file that is formatted as
* a proper "ref:", or a regular file HEAD that has a properly
* formatted sha1 object name.
@@ -384,8 +384,12 @@ int is_git_directory(const char *suspect)
strbuf_setlen(&path, len);
strbuf_addstr(&path, "/refs");
- if (access(path.buf, X_OK))
- goto done;
+ if (access(path.buf, X_OK)) {
+ strbuf_setlen(&path, len);
+ strbuf_addstr(&path, "/packed-refs");
+ if (access(path.buf, R_OK))
+ goto done;
+ }
ret = 1;
done:
diff --git a/t/t6500-gc.sh b/t/t6500-gc.sh
index 18fe1c25e6..e81eb7d2ab 100755
--- a/t/t6500-gc.sh
+++ b/t/t6500-gc.sh
@@ -214,6 +214,14 @@ test_expect_success 'gc.repackFilter launches repack with a filter' '
grep -E "^trace: (built-in|exec|run_command): git repack .* --filter=blob:none ?.*" trace.out
'
+test_expect_success 'remove empty refs/ subdirectory' '
+ git -C bare.git cat-file -e 60dd8ad7d8ed49005c18b7ce9f5b7a45c28df814 &&
+ test_dir_is_empty bare.git/refs/heads &&
+ test_dir_is_empty bare.git/refs/tags &&
+ rm -r bare.git/refs &&
+ GIT_DIR="$PWD"/bare.git git -C bare.git cat-file -e 60dd8ad7d8ed49005c18b7ce9f5b7a45c28df814
+'
+
test_expect_success 'gc.repackFilterTo store filtered out objects' '
test_when_finished "rm -rf bare.git filtered.git" &&
--
2.42.0
^ permalink raw reply related
* [PATCH] setup: recognize bare repositories with packed-refs
From: Adam Majer @ 2023-11-17 20:32 UTC (permalink / raw)
To: git; +Cc: Adam Majer
In-Reply-To: <20231117202513.20604-1-adamm@zombino.com>
In a garbage collected bare git repository, the refs/ subdirectory is
empty. In use-cases when such a repository is directly added into
another repository, it no longer is detected as valid. Git doesn't
preserve empty paths so refs/ subdirectory is not present. Simply
creating an empty refs/ subdirectory fixes this problem.
Looking more carefully, there are two backends to handle various refs in
git -- the files backend that uses refs/ subdirectory and the
packed-refs backend that uses packed-refs file. If references are not
found in refs/ subdirectory (or directory doesn't exist), the
packed-refs directory will be consulted. Garbage collected repository
will have all its references in packed-refs file.
To allow the use-case when packed-refs is the only source of refs and
refs/ subdirectory is simply not present, augment 'is_git_directory()'
setup function to look for packed-refs file as an alternative to refs/
subdirectory.
Signed-off-by: Adam Majer <adamm@zombino.com>
---
setup.c | 10 +++++++---
t/t6500-gc.sh | 8 ++++++++
2 files changed, 15 insertions(+), 3 deletions(-)
diff --git a/setup.c b/setup.c
index fc592dc6dd..2a6dda6ae9 100644
--- a/setup.c
+++ b/setup.c
@@ -348,7 +348,7 @@ int get_common_dir_noenv(struct strbuf *sb, const char *gitdir)
*
* - either an objects/ directory _or_ the proper
* GIT_OBJECT_DIRECTORY environment variable
- * - a refs/ directory
+ * - a refs/ directory or packed-refs file
* - either a HEAD symlink or a HEAD file that is formatted as
* a proper "ref:", or a regular file HEAD that has a properly
* formatted sha1 object name.
@@ -384,8 +384,12 @@ int is_git_directory(const char *suspect)
strbuf_setlen(&path, len);
strbuf_addstr(&path, "/refs");
- if (access(path.buf, X_OK))
- goto done;
+ if (access(path.buf, X_OK)) {
+ strbuf_setlen(&path, len);
+ strbuf_addstr(&path, "/packed-refs");
+ if (access(path.buf, R_OK))
+ goto done;
+ }
ret = 1;
done:
diff --git a/t/t6500-gc.sh b/t/t6500-gc.sh
index 18fe1c25e6..e81eb7d2ab 100755
--- a/t/t6500-gc.sh
+++ b/t/t6500-gc.sh
@@ -214,6 +214,14 @@ test_expect_success 'gc.repackFilter launches repack with a filter' '
grep -E "^trace: (built-in|exec|run_command): git repack .* --filter=blob:none ?.*" trace.out
'
+test_expect_success 'remove empty refs/ subdirectory' '
+ git -C bare.git cat-file -e 60dd8ad7d8ed49005c18b7ce9f5b7a45c28df814 &&
+ test_dir_is_empty bare.git/refs/heads &&
+ test_dir_is_empty bare.git/refs/tags &&
+ rm -r bare.git/refs &&
+ GIT_DIR="$PWD"/bare.git git -C bare.git cat-file -e 60dd8ad7d8ed49005c18b7ce9f5b7a45c28df814
+'
+
test_expect_success 'gc.repackFilterTo store filtered out objects' '
test_when_finished "rm -rf bare.git filtered.git" &&
--
2.42.0
^ permalink raw reply related
* [PATCH v4] fuzz: add new oss-fuzz fuzzer for date.c / date.h
From: Arthur Chan via GitGitGadget @ 2023-11-17 17:47 UTC (permalink / raw)
To: git; +Cc: Jeff King, Arthur Chan, Arthur Chan
In-Reply-To: <pull.1612.v3.git.1699959186146.gitgitgadget@gmail.com>
From: Arthur Chan <arthur.chan@adalogics.com>
Signed-off-by: Arthur Chan <arthur.chan@adalogics.com>
---
fuzz: add new oss-fuzz fuzzer for date.c / date.h
This patch is aimed to add a new oss-fuzz fuzzer to the oss-fuzz
directory for fuzzing date.c / date.h in the base directory.
The .gitignore of the oss-fuzz directory and the Makefile have been
modified to accommodate the new fuzzer fuzz-date.c.
Fixed the objects order in .gitignore and Makefiles and fixed some of
the logic and formatting for the fuzz-date.c fuzzer in v2.
Fixed the creation and memory allocation of the fuzzing str in v3. Also
fixed the tz type and sign-extended the data before passing to the tz
variable.
Fixed the tz variable allocations and some of the bytes used for fuzzing
variables in v4.
Comment: Yes, indeed. It is quite annoying to have that twice. Yes, the
tz should be considered as attacker controllable and thus negative
values should be considered. But it is tricky to fuzz it because the
date.c::gm_time_t() will call die() if the value is invalid and that
exit the fuzzer directly. OSS-Fuzz may consider it as an issue (or bug)
because the fuzzer exit "unexpectedly". I agree that if we consider the
tz as "attacker controllable, we should include negative values, but
since it will cause the fuzzer exit, I am not sure if it is the right
approach from the fuzzing perspective. Also, it is something that date.c
already take care of with the conditional checking, thus it may also be
worth to do some checking and exclude some invalid values before calling
date.c::show_date() but this may result in copying some conditional
checking code from date.c.
Additional comment for v4: Thanks for the suggestion. Yes, that maybe
the easier approach. Since the new logic is only using 2 bytes for the
int16_t tz, thus the local and dmtype variable could use separate bytes
to increase "randomness".
Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-1612%2Farthurscchan%2Fnew-fuzzer-date-v4
Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-1612/arthurscchan/new-fuzzer-date-v4
Pull-Request: https://github.com/gitgitgadget/git/pull/1612
Range-diff vs v3:
1: 046bca32889 ! 1: 33a72d4c197 fuzz: add new oss-fuzz fuzzer for date.c / date.h
@@ oss-fuzz/fuzz-date.c (new)
+{
+ int local;
+ int num;
-+ int tz;
+ char *str;
-+ int8_t *tmp_data;
++ int16_t tz;
+ timestamp_t ts;
+ enum date_mode_type dmtype;
+ struct date_mode *dm;
+
+ if (size <= 4)
+ /*
-+ * we use the first byte to fuzz dmtype and local,
-+ * then the next three bytes to fuzz tz offset,
-+ * and the remainder (at least one byte) is fed
-+ * as end-user input to approxidate_careful().
++ * we use the first byte to fuzz dmtype and the
++ * second byte to fuzz local, then the next two
++ * bytes to fuzz tz offset. The remainder
++ * (at least one byte) is fed as input to
++ * approxidate_careful().
+ */
+ return 0;
+
-+ local = !!(*data & 0x10);
-+ num = *data % DATE_UNIX;
++ local = !!(*data++ & 0x10);
++ num = *data++ % DATE_UNIX;
+ if (num >= DATE_STRFTIME)
+ num++;
+ dmtype = (enum date_mode_type)num;
-+ data++;
-+ size--;
++ size -= 2;
+
-+ tmp_data = (int8_t*)data;
-+ tz = *tmp_data++;
-+ tz = (tz << 8) | *tmp_data++;
-+ tz = (tz << 8) | *tmp_data++;
-+ data += 3;
-+ size -= 3;
++ tz = *data++;
++ tz = (tz << 8) | *data++;
++ size -= 2;
+
+ str = xmemdupz(data, size);
+
@@ oss-fuzz/fuzz-date.c (new)
+
+ dm = date_mode_from_type(dmtype);
+ dm->local = local;
-+ show_date(ts, tz, dm);
++ show_date(ts, (int)tz, dm);
+
+ date_mode_release(dm);
+
Makefile | 1 +
oss-fuzz/.gitignore | 1 +
oss-fuzz/fuzz-date.c | 49 ++++++++++++++++++++++++++++++++++++++++++++
3 files changed, 51 insertions(+)
create mode 100644 oss-fuzz/fuzz-date.c
diff --git a/Makefile b/Makefile
index 03adcb5a480..4b875ef6ce1 100644
--- a/Makefile
+++ b/Makefile
@@ -750,6 +750,7 @@ SCRIPTS = $(SCRIPT_SH_GEN) \
ETAGS_TARGET = TAGS
FUZZ_OBJS += oss-fuzz/fuzz-commit-graph.o
+FUZZ_OBJS += oss-fuzz/fuzz-date.o
FUZZ_OBJS += oss-fuzz/fuzz-pack-headers.o
FUZZ_OBJS += oss-fuzz/fuzz-pack-idx.o
.PHONY: fuzz-objs
diff --git a/oss-fuzz/.gitignore b/oss-fuzz/.gitignore
index 9acb74412ef..5b954088254 100644
--- a/oss-fuzz/.gitignore
+++ b/oss-fuzz/.gitignore
@@ -1,3 +1,4 @@
fuzz-commit-graph
+fuzz-date
fuzz-pack-headers
fuzz-pack-idx
diff --git a/oss-fuzz/fuzz-date.c b/oss-fuzz/fuzz-date.c
new file mode 100644
index 00000000000..036378b946f
--- /dev/null
+++ b/oss-fuzz/fuzz-date.c
@@ -0,0 +1,49 @@
+#include "git-compat-util.h"
+#include "date.h"
+
+int LLVMFuzzerTestOneInput(const uint8_t *data, size_t size);
+
+int LLVMFuzzerTestOneInput(const uint8_t *data, size_t size)
+{
+ int local;
+ int num;
+ char *str;
+ int16_t tz;
+ timestamp_t ts;
+ enum date_mode_type dmtype;
+ struct date_mode *dm;
+
+ if (size <= 4)
+ /*
+ * we use the first byte to fuzz dmtype and the
+ * second byte to fuzz local, then the next two
+ * bytes to fuzz tz offset. The remainder
+ * (at least one byte) is fed as input to
+ * approxidate_careful().
+ */
+ return 0;
+
+ local = !!(*data++ & 0x10);
+ num = *data++ % DATE_UNIX;
+ if (num >= DATE_STRFTIME)
+ num++;
+ dmtype = (enum date_mode_type)num;
+ size -= 2;
+
+ tz = *data++;
+ tz = (tz << 8) | *data++;
+ size -= 2;
+
+ str = xmemdupz(data, size);
+
+ ts = approxidate_careful(str, &num);
+ free(str);
+
+ dm = date_mode_from_type(dmtype);
+ dm->local = local;
+ show_date(ts, (int)tz, dm);
+
+ date_mode_release(dm);
+
+ return 0;
+}
base-commit: dadef801b365989099a9929e995589e455c51fed
--
gitgitgadget
^ permalink raw reply related
* Re: Migration of git-scm.com to a static web site: ready for review/testing
From: Todd Zullinger @ 2023-11-17 16:26 UTC (permalink / raw)
To: Johannes Schindelin; +Cc: git, Matt Burke, Victoria Dye, Matthias Aßhauer
In-Reply-To: <6f7d20b4-a725-0ef9-f6d3-ff2810da9e7a@gmx.de>
Hello,
Johannes Schindelin wrote:
> At this point, the patches are fairly robust and I am mainly hoping for
> help with verifying that the static site works as intended, that existing
> links will continue to work with the new site (essentially, find obscure
> references to the existing website, then insert `git.github.io/` in the
> URL and verify that it works as intended).
>
> To that end, I deployed this branch to GitHub Pages so that anyone
> interested (hopefully many!) can have a look at
> https://git.github.io/git-scm.com/ and compare to the existing
> https://git-scm.com/.
This is nice. Thanks to all for working on it!
For checking links, a tool like linkcheker[1] is very handy.
This is run against the local docs in the Fedora package
builds to catch broken links.
I ran it against the test site and it turned up _a lot_ of
broken links. It's enough that saving and sharing the
output is probably more work than having someone familiar
with the migration give it a run directly.
I ran `linkchecker https://git.github.io/git-scm.com/` and
the eventual result was:
That's it. 13459 links in 14126 URLs checked. 0 warnings found. 6763 errors found.
Stopped checking at 2023-11-17 11:11:17-004 (1 hour, 19 minutes)
The default output reports failures in a format like this:
URL `ch00/ch10-git-internals'
Name `Git Internals'
Parent URL https://git.github.io/git-scm.com/book/tr/v2/Ek-b%C3%B6l%C3%BCm-C:-Git-Commands-Plumbing-Commands/, line 106, col 1318
Real URL https://git.github.io/git-scm.com/book/tr/v2/Ek-b%C3%B6l%C3%BCm-C:-Git-Commands-Plumbing-Commands/ch00/ch10-git-internals
Check time 3.303 seconds
Size 1KB
Result Error: 404 Not Found
LinkChecker can be run in a mode which directs the failures
to a file. That would be more like:
linkchecker -F text/utf_8//tmp/git-scm-check.txt https://git.github.io/git-scm.com/
The format of the -F option is TYPE[/ENCODING][/FILENAME]
where TYPE can be text, html, sql, csv, gml, dot, xml,
sitemap, none or failures. The failures type is much more
terse:
1 "('https://git.github.io/git-scm.com/book/en/v2/Appendix-C:-Git-Commands-Plumbing-Commands/', 'https://git.github.io/git-scm.com/book/en/v2/Appendix-C:-Git-Commands-Plumbing-Commands/ch00/ch10-git-internals')"
I found the text type much more helpful in quickly spot
checking some of the failures since it includes the text
string used for the link.
Running it against a local directory of the content would be
a lot faster, if that's an option. It's also worth bumping
the default number of threads from 10 to increase the speed
a bit.
[1] https://linkchecker.github.io/linkchecker/
--
Todd
^ permalink raw reply
* Migration of git-scm.com to a static web site: ready for review/testing
From: Johannes Schindelin @ 2023-11-17 13:25 UTC (permalink / raw)
To: git; +Cc: Matt Burke, Victoria Dye, Matthias Aßhauer
[-- Attachment #1: Type: text/plain, Size: 1484 bytes --]
Hi,
the idea of migrating https://git-scm.com/ from a Rails app to a static
site has been discussed several times on this list in the past.
Thanks to the heroic, multi-year efforts of Matt Burke, Victoria Dye and
Matthias Aßhauer, there is now a Pull Request, ready for review:
https://github.com/git/git-scm.com/pull/1804
This Pull Request is not for the faint of heart, mainly because of the
sheer amount of generated pages that are committed to the repository (such
as the book, the manual pages, etc, a design decision necessary to run
this as a static website).
These pages are generated by GitHub workflows that are intended to run on
a schedule, and the scripts that generate them are part of the Pull
Request. For that reason, I do not consider it necessary to review those
generated pages, those reviews have been done in the upstream sources from
which the pages were generated.
At this point, the patches are fairly robust and I am mainly hoping for
help with verifying that the static site works as intended, that existing
links will continue to work with the new site (essentially, find obscure
references to the existing website, then insert `git.github.io/` in the
URL and verify that it works as intended).
To that end, I deployed this branch to GitHub Pages so that anyone
interested (hopefully many!) can have a look at
https://git.github.io/git-scm.com/ and compare to the existing
https://git-scm.com/.
Thank you,
Johannes
^ permalink raw reply
* [PATCH] Fix a typo in `each_file_in_pack_dir_fn()`'s declaration
From: Johannes Schindelin via GitGitGadget @ 2023-11-17 13:15 UTC (permalink / raw)
To: git; +Cc: Johannes Schindelin, Johannes Schindelin
From: Johannes Schindelin <johannes.schindelin@gmx.de>
One parameter is called `file_pach`. On the face of it, this looks as if
it was supposed to talk about a `path` instead of a `pach`.
However, looking at the way this callback is called, it gets fed the
`d_name` from a directory entry, which provides just the file name, not
the full path. Therefore, let's fix this by calling the parameter
`file_name` instead.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
packfile.h: fix a typo
I stumbled over this typo yesterday. Nothing about this patch is urgent,
of course, it can easily wait until v2.43.0 is released.
Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-1614%2Fdscho%2Fpackfile.h-typo-fix-v1
Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-1614/dscho/packfile.h-typo-fix-v1
Pull-Request: https://github.com/gitgitgadget/git/pull/1614
packfile.h | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/packfile.h b/packfile.h
index c3692308b8d..28c8fd3e39a 100644
--- a/packfile.h
+++ b/packfile.h
@@ -54,7 +54,7 @@ const char *pack_basename(struct packed_git *p);
struct packed_git *parse_pack_index(unsigned char *sha1, const char *idx_path);
typedef void each_file_in_pack_dir_fn(const char *full_path, size_t full_path_len,
- const char *file_pach, void *data);
+ const char *file_name, void *data);
void for_each_file_in_pack_dir(const char *objdir,
each_file_in_pack_dir_fn fn,
void *data);
base-commit: cfb8a6e9a93adbe81efca66e6110c9b4d2e57169
--
gitgitgadget
^ permalink raw reply related
* Re: What's cooking in git.git (Nov 2023, #07; Fri, 17)
From: Kristoffer Haugsbakk @ 2023-11-17 10:36 UTC (permalink / raw)
To: Junio C Hamano; +Cc: git
In-Reply-To: <xmqqpm08pyrh.fsf@gitster.g>
On Fri, Nov 17, 2023, at 09:30, Junio C Hamano wrote:
> * kh/t7900-cleanup (2023-10-17) 9 commits
> - t7900: fix register dependency
> - t7900: factor out packfile dependency
> - t7900: fix `print-args` dependency
> - t7900: fix `pfx` dependency
> - t7900: factor out common schedule setup
> - t7900: factor out inheritance test dependency
> - t7900: create commit so that branch is born
> - t7900: setup and tear down clones
> - t7900: remove register dependency
>
> Test clean-up.
>
> Perhaps discard?
> cf. <655ca147-c214-41be-919d-023c1b27b311@app.fastmail.com>
> source: <cover.1697319294.git.code@khaugsbakk.name>
There has been no interest in it so I say yes to discarding.
^ permalink raw reply
* What's cooking in git.git (Nov 2023, #07; Fri, 17)
From: Junio C Hamano @ 2023-11-17 8:30 UTC (permalink / raw)
To: git
Here are the topics that have been cooking in my tree. Commits
prefixed with '+' are in 'next' (being in 'next' is a sign that a
topic is stable enough to be used and are candidate to be in a
future release). Commits prefixed with '-' are only in 'seen', and
aren't considered "accepted" at all and may be annotated with an URL
to a message that raises issues but they are no means exhaustive. A
topic without enough support may be discarded after a long period of
no activity (of course they can be resubmit when new interests
arise).
Git 2.43-rc2 has been tagged.
Copies of the source code to Git live in many repositories, and the
following is a list of the ones I push into or their mirrors. Some
repositories have only a subset of branches.
With maint, master, next, seen, todo:
git://git.kernel.org/pub/scm/git/git.git/
git://repo.or.cz/alt-git.git/
https://kernel.googlesource.com/pub/scm/git/git/
https://github.com/git/git/
https://gitlab.com/git-vcs/git/
With all the integration branches and topics broken out:
https://github.com/gitster/git/
Even though the preformatted documentation in HTML and man format
are not sources, they are published in these repositories for
convenience (replace "htmldocs" with "manpages" for the manual
pages):
git://git.kernel.org/pub/scm/git/git-htmldocs.git/
https://github.com/gitster/git-htmldocs.git/
Release tarballs are available at:
https://www.kernel.org/pub/software/scm/git/
--------------------------------------------------
[New Topics]
* jw/builtin-objectmode-attr (2023-11-16) 2 commits
- SQUASH???
- attr: add builtin objectmode values support
The builtin_objectmode attribute is populated for each path
without adding anything in .gitattributes files, which would be
useful in magic pathspec, e.g., ":(attr:builtin_objectmode=100755)"
to limit to executables.
source: <20231116054437.2343549-1-jojwang@google.com>
* ps/ref-deletion-updates (2023-11-17) 4 commits
- refs: remove `delete_refs` callback from backends
- refs: deduplicate code to delete references
- refs/files: use transactions to delete references
- t5510: ensure that the packed-refs file needs locking
Simplify API implementation to delete references by eliminating
duplication.
Will merge to 'next'.
source: <cover.1699951815.git.ps@pks.im>
* tz/send-email-helpfix (2023-11-16) 1 commit
(merged to 'next' on 2023-11-17 at 8422271795)
+ send-email: remove stray characters from usage
Typoes in "git send-email -h" have been corrected.
Will merge to 'master'.
source: <20231115173952.339303-3-tmz@pobox.com>
* tz/send-email-negatable-options (2023-11-17) 2 commits
(merged to 'next' on 2023-11-17 at f09e533e43)
+ send-email: avoid duplicate specification warnings
+ perl: bump the required Perl version to 5.8.1 from 5.8.0
Newer versions of Getopt::Long started giving warnings against our
(ab)use of it in "git send-email". Bump the minimum version
requirement for Perl to 5.8.1 (from September 2002) to allow
simplifying our implementation.
Will cook in 'next'.
source: <20231116193014.470420-1-tmz@pobox.com>
--------------------------------------------------
[Stalled]
* pw/rebase-sigint (2023-09-07) 1 commit
- rebase -i: ignore signals when forking subprocesses
If the commit log editor or other external programs (spawned via
"exec" insn in the todo list) receive internactive signal during
"git rebase -i", it caused not just the spawned program but the
"Git" process that spawned them, which is often not what the end
user intended. "git" learned to ignore SIGINT and SIGQUIT while
waiting for these subprocesses.
Expecting a reroll.
cf. <12c956ea-330d-4441-937f-7885ab519e26@gmail.com>
source: <pull.1581.git.1694080982621.gitgitgadget@gmail.com>
* tk/cherry-pick-sequence-requires-clean-worktree (2023-06-01) 1 commit
- cherry-pick: refuse cherry-pick sequence if index is dirty
"git cherry-pick A" that replays a single commit stopped before
clobbering local modification, but "git cherry-pick A..B" did not,
which has been corrected.
Expecting a reroll.
cf. <999f12b2-38d6-f446-e763-4985116ad37d@gmail.com>
source: <pull.1535.v2.git.1685264889088.gitgitgadget@gmail.com>
* jc/diff-cached-fsmonitor-fix (2023-09-15) 3 commits
- diff-lib: fix check_removed() when fsmonitor is active
- Merge branch 'jc/fake-lstat' into jc/diff-cached-fsmonitor-fix
- Merge branch 'js/diff-cached-fsmonitor-fix' into jc/diff-cached-fsmonitor-fix
(this branch uses jc/fake-lstat.)
The optimization based on fsmonitor in the "diff --cached"
codepath is resurrected with the "fake-lstat" introduced earlier.
It is unknown if the optimization is worth resurrecting, but in case...
source: <xmqqr0n0h0tw.fsf@gitster.g>
--------------------------------------------------
[Cooking]
* js/ci-discard-prove-state (2023-11-14) 1 commit
(merged to 'next' on 2023-11-14 at fade3ba143)
+ ci: avoid running the test suite _twice_
(this branch uses ps/ci-gitlab.)
The way CI testing used "prove" could lead to running the test
suite twice needlessly, which has been corrected.
Will cook in 'next'.
source: <pull.1613.git.1699894837844.gitgitgadget@gmail.com>
* jk/chunk-bounds-more (2023-11-09) 9 commits
(merged to 'next' on 2023-11-13 at 3df4b18bea)
+ commit-graph: mark chunk error messages for translation
+ commit-graph: drop verify_commit_graph_lite()
+ commit-graph: check order while reading fanout chunk
+ commit-graph: use fanout value for graph size
+ commit-graph: abort as soon as we see a bogus chunk
+ commit-graph: clarify missing-chunk error messages
+ commit-graph: drop redundant call to "lite" verification
+ midx: check consistency of fanout table
+ commit-graph: handle overflow in chunk_size checks
(this branch is used by tb/pair-chunk-expect.)
Code clean-up for jk/chunk-bounds topic.
Will cook in 'next'.
source: <20231109070310.GA2697602@coredump.intra.peff.net>
* ps/httpd-tests-on-nixos (2023-11-11) 3 commits
(merged to 'next' on 2023-11-13 at 81bd6f5334)
+ t9164: fix inability to find basename(1) in Subversion hooks
+ t/lib-httpd: stop using legacy crypt(3) for authentication
+ t/lib-httpd: dynamically detect httpd and modules path
Portability tweak.
Will cook in 'next'.
source: <cover.1699596457.git.ps@pks.im>
* ss/format-patch-use-encode-headers-for-cover-letter (2023-11-10) 1 commit
(merged to 'next' on 2023-11-14 at 1a4bd59e15)
+ format-patch: fix ignored encode_email_headers for cover letter
"git format-patch --encode-email-headers" ignored the option when
preparing the cover letter, which has been corrected.
Will cook in 'next'.
source: <20231109111950.387219-1-contact@emersion.fr>
* ps/ban-a-or-o-operator-with-test (2023-11-11) 4 commits
(merged to 'next' on 2023-11-14 at d84471baab)
+ Makefile: stop using `test -o` when unlinking duplicate executables
+ contrib/subtree: convert subtree type check to use case statement
+ contrib/subtree: stop using `-o` to test for number of args
+ global: convert trivial usages of `test <expr> -a/-o <expr>`
Test and shell scripts clean-up.
Will cook in 'next'.
source: <cover.1699609940.git.ps@pks.im>
* vd/glossary-dereference-peel (2023-11-14) 1 commit
(merged to 'next' on 2023-11-17 at bac3ab0c0b)
+ glossary: add definitions for dereference & peel
"To dereference" and "to peel" were sometimes used in in-code
comments and documentation but without description in the glossary.
Will merge to 'master'.
source: <pull.1610.v2.git.1699917471769.gitgitgadget@gmail.com>
* ak/rebase-autosquash (2023-11-16) 3 commits
(merged to 'next' on 2023-11-17 at 3ed6e79445)
+ rebase: rewrite --(no-)autosquash documentation
+ rebase: support --autosquash without -i
+ rebase: fully ignore rebase.autoSquash without -i
"git rebase --autosquash" is now enabled for non-interactive rebase,
but it is still incompatible with the apply backend.
Will cook in 'next'.
source: <20231114214339.10925-1-andy.koppe@gmail.com>
* vd/for-each-ref-unsorted-optimization (2023-11-16) 10 commits
(merged to 'next' on 2023-11-17 at ff99420bf6)
+ t/perf: add perf tests for for-each-ref
+ ref-filter.c: use peeled tag for '*' format fields
+ for-each-ref: clean up documentation of --format
+ ref-filter.c: filter & format refs in the same callback
+ ref-filter.c: refactor to create common helper functions
+ ref-filter.c: rename 'ref_filter_handler()' to 'filter_one()'
+ ref-filter.h: add functions for filter/format & format-only
+ ref-filter.h: move contains caches into filter
+ ref-filter.h: add max_count and omit_empty to ref_format
+ ref-filter.c: really don't sort when using --no-sort
"git for-each-ref --no-sort" still sorted the refs alphabetically
which paid non-trivial cost. It has been redefined to show output
in an unspecified order, to allow certain optimizations to take
advantage of.
Will cook in 'next'.
source: <pull.1609.v2.git.1699991638.gitgitgadget@gmail.com>
* jw/git-add-attr-pathspec (2023-11-04) 1 commit
(merged to 'next' on 2023-11-13 at b61be94e4d)
+ attr: enable attr pathspec magic for git-add and git-stash
"git add" and "git stash" learned to support the ":(attr:...)"
magic pathspec.
Will cook in 'next'.
source: <20231103163449.1578841-1-jojwang@google.com>
* ps/ci-gitlab (2023-11-09) 8 commits
(merged to 'next' on 2023-11-10 at ea7ed67945)
+ ci: add support for GitLab CI
+ ci: install test dependencies for linux-musl
+ ci: squelch warnings when testing with unusable Git repo
+ ci: unify setup of some environment variables
+ ci: split out logic to set up failed test artifacts
+ ci: group installation of Docker dependencies
+ ci: make grouping setup more generic
+ ci: reorder definitions for grouping functions
(this branch is used by js/ci-discard-prove-state.)
Add support for GitLab CI.
Will cook in 'next'.
source: <cover.1699514143.git.ps@pks.im>
* ps/ref-tests-update (2023-11-03) 10 commits
(merged to 'next' on 2023-11-13 at dc26e55d6f)
+ t: mark several tests that assume the files backend with REFFILES
+ t7900: assert the absence of refs via git-for-each-ref(1)
+ t7300: assert exact states of repo
+ t4207: delete replace references via git-update-ref(1)
+ t1450: convert tests to remove worktrees via git-worktree(1)
+ t: convert tests to not access reflog via the filesystem
+ t: convert tests to not access symrefs via the filesystem
+ t: convert tests to not write references via the filesystem
+ t: allow skipping expected object ID in `ref-store update-ref`
+ Merge branch 'ps/show-ref' into ps/ref-tests-update
Update ref-related tests.
Will cook in 'next'.
source: <cover.1698914571.git.ps@pks.im>
* jx/fetch-atomic-error-message-fix (2023-10-19) 2 commits
- fetch: no redundant error message for atomic fetch
- t5574: test porcelain output of atomic fetch
"git fetch --atomic" issued an unnecessary empty error message,
which has been corrected.
Expecting an update.
cf. <ZTjQIrCgSANAT8wR@tanuki>
source: <ced46baeb1c18b416b4b4cc947f498bea2910b1b.1697725898.git.zhiyou.jx@alibaba-inc.com>
* js/bugreport-in-the-same-minute (2023-10-16) 1 commit
- bugreport: include +i in outfile suffix as needed
Instead of auto-generating a filename that is already in use for
output and fail the command, `git bugreport` learned to fuzz the
filename to avoid collisions with existing files.
Expecting a reroll.
cf. <ZTtZ5CbIGETy1ucV.jacob@initialcommit.io>
source: <20231016214045.146862-2-jacob@initialcommit.io>
* kh/t7900-cleanup (2023-10-17) 9 commits
- t7900: fix register dependency
- t7900: factor out packfile dependency
- t7900: fix `print-args` dependency
- t7900: fix `pfx` dependency
- t7900: factor out common schedule setup
- t7900: factor out inheritance test dependency
- t7900: create commit so that branch is born
- t7900: setup and tear down clones
- t7900: remove register dependency
Test clean-up.
Perhaps discard?
cf. <655ca147-c214-41be-919d-023c1b27b311@app.fastmail.com>
source: <cover.1697319294.git.code@khaugsbakk.name>
* tb/merge-tree-write-pack (2023-10-23) 5 commits
- builtin/merge-tree.c: implement support for `--write-pack`
- bulk-checkin: introduce `index_tree_bulk_checkin_incore()`
- bulk-checkin: introduce `index_blob_bulk_checkin_incore()`
- bulk-checkin: generify `stream_blob_to_pack()` for arbitrary types
- bulk-checkin: extract abstract `bulk_checkin_source`
"git merge-tree" learned "--write-pack" to record its result
without creating loose objects.
Broken when an object created during a merge is needed to continue merge
cf. <CABPp-BEfy9VOvimP9==ry_rZXu=metOQ8s=_-XiG_Pdx9c06Ww@mail.gmail.com>
source: <cover.1698101088.git.me@ttaylorr.com>
* tb/pair-chunk-expect (2023-11-10) 8 commits
- midx: read `OOFF` chunk with `pair_chunk_expect()`
- midx: read `OIDL` chunk with `pair_chunk_expect()`
- commit-graph: read `BIDX` chunk with `pair_chunk_expect()`
- commit-graph: read `GDAT` chunk with `pair_chunk_expect()`
- commit-graph: read `CDAT` chunk with `pair_chunk_expect()`
- commit-graph: read `OIDL` chunk with `pair_chunk_expect()`
- chunk-format: introduce `pair_chunk_expect()` helper
- Merge branch 'jk/chunk-bounds-more' into HEAD
(this branch uses jk/chunk-bounds-more.)
Further code clean-up.
Needs review.
source: <cover.1699569246.git.me@ttaylorr.com>
* tb/path-filter-fix (2023-10-18) 17 commits
- bloom: introduce `deinit_bloom_filters()`
- commit-graph: reuse existing Bloom filters where possible
- object.h: fix mis-aligned flag bits table
- commit-graph: drop unnecessary `graph_read_bloom_data_context`
- commit-graph.c: unconditionally load Bloom filters
- bloom: prepare to discard incompatible Bloom filters
- bloom: annotate filters with hash version
- commit-graph: new filter ver. that fixes murmur3
- repo-settings: introduce commitgraph.changedPathsVersion
- t4216: test changed path filters with high bit paths
- t/helper/test-read-graph: implement `bloom-filters` mode
- bloom.h: make `load_bloom_filter_from_graph()` public
- t/helper/test-read-graph.c: extract `dump_graph_info()`
- gitformat-commit-graph: describe version 2 of BDAT
- commit-graph: ensure Bloom filters are read with consistent settings
- revision.c: consult Bloom filters for root commits
- t/t4216-log-bloom.sh: harden `test_bloom_filters_not_used()`
The Bloom filter used for path limited history traversal was broken
on systems whose "char" is unsigned; update the implementation and
bump the format version to 2.
Needs (hopefully final and quick) review.
source: <cover.1697653929.git.me@ttaylorr.com>
* cc/git-replay (2023-11-16) 14 commits
- replay: stop assuming replayed branches do not diverge
- replay: add --contained to rebase contained branches
- replay: add --advance or 'cherry-pick' mode
- replay: use standard revision ranges
- replay: make it a minimal server side command
- replay: remove HEAD related sanity check
- replay: remove progress and info output
- replay: add an important FIXME comment about gpg signing
- replay: change rev walking options
- replay: introduce pick_regular_commit()
- replay: die() instead of failing assert()
- replay: start using parse_options API
- replay: introduce new builtin
- t6429: remove switching aspects of fast-rebase
Introduce "git replay", a tool meant on the server side without
working tree to recreate a history.
source: <20231115143327.2441397-1-christian.couder@gmail.com>
* ak/color-decorate-symbols (2023-10-23) 7 commits
- log: add color.decorate.pseudoref config variable
- refs: exempt pseudorefs from pattern prefixing
- refs: add pseudorefs array and iteration functions
- log: add color.decorate.ref config variable
- log: add color.decorate.symbol config variable
- log: use designated inits for decoration_colors
- config: restructure color.decorate documentation
A new config for coloring.
Needs review.
source: <20231023221143.72489-1-andy.koppe@gmail.com>
* js/update-urls-in-doc-and-comment (2023-09-26) 4 commits
- doc: refer to internet archive
- doc: update links for andre-simon.de
- doc: update links to current pages
- doc: switch links to https
Stale URLs have been updated to their current counterparts (or
archive.org) and HTTP links are replaced with working HTTPS links.
Needs review.
source: <pull.1589.v2.git.1695553041.gitgitgadget@gmail.com>
* la/trailer-cleanups (2023-10-20) 3 commits
- trailer: use offsets for trailer_start/trailer_end
- trailer: find the end of the log message
- commit: ignore_non_trailer computes number of bytes to ignore
Code clean-up.
Comments?
source: <pull.1563.v5.git.1697828495.gitgitgadget@gmail.com>
* eb/hash-transition (2023-10-02) 30 commits
- t1016-compatObjectFormat: add tests to verify the conversion between objects
- t1006: test oid compatibility with cat-file
- t1006: rename sha1 to oid
- test-lib: compute the compatibility hash so tests may use it
- builtin/ls-tree: let the oid determine the output algorithm
- object-file: handle compat objects in check_object_signature
- tree-walk: init_tree_desc take an oid to get the hash algorithm
- builtin/cat-file: let the oid determine the output algorithm
- rev-parse: add an --output-object-format parameter
- repository: implement extensions.compatObjectFormat
- object-file: update object_info_extended to reencode objects
- object-file-convert: convert commits that embed signed tags
- object-file-convert: convert commit objects when writing
- object-file-convert: don't leak when converting tag objects
- object-file-convert: convert tag objects when writing
- object-file-convert: add a function to convert trees between algorithms
- object: factor out parse_mode out of fast-import and tree-walk into in object.h
- cache: add a function to read an OID of a specific algorithm
- tag: sign both hashes
- commit: export add_header_signature to support handling signatures on tags
- commit: convert mergetag before computing the signature of a commit
- commit: write commits for both hashes
- object-file: add a compat_oid_in parameter to write_object_file_flags
- object-file: update the loose object map when writing loose objects
- loose: compatibilty short name support
- loose: add a mapping between SHA-1 and SHA-256 for loose objects
- repository: add a compatibility hash algorithm
- object-names: support input of oids in any supported hash
- oid-array: teach oid-array to handle multiple kinds of oids
- object-file-convert: stubs for converting from one object format to another
Teach a repository to work with both SHA-1 and SHA-256 hash algorithms.
Needs review.
source: <878r8l929e.fsf@gmail.froward.int.ebiederm.org>
* jx/remote-archive-over-smart-http (2023-10-04) 4 commits
- archive: support remote archive from stateless transport
- transport-helper: call do_take_over() in connect_helper
- transport-helper: call do_take_over() in process_connect
- transport-helper: no connection restriction in connect_helper
"git archive --remote=<remote>" learned to talk over the smart
http (aka stateless) transport.
Needs review.
source: <cover.1696432593.git.zhiyou.jx@alibaba-inc.com>
* jx/sideband-chomp-newline-fix (2023-10-04) 3 commits
- pkt-line: do not chomp newlines for sideband messages
- pkt-line: memorize sideband fragment in reader
- test-pkt-line: add option parser for unpack-sideband
Sideband demultiplexer fixes.
Needs review.
source: <cover.1696425168.git.zhiyou.jx@alibaba-inc.com>
* js/config-parse (2023-09-21) 5 commits
- config-parse: split library out of config.[c|h]
- config.c: accept config_parse_options in git_config_from_stdin
- config: report config parse errors using cb
- config: split do_event() into start and flush operations
- config: split out config_parse_options
The parsing routines for the configuration files have been split
into a separate file.
Needs review.
source: <cover.1695330852.git.steadmon@google.com>
* jc/fake-lstat (2023-09-15) 1 commit
- cache: add fake_lstat()
(this branch is used by jc/diff-cached-fsmonitor-fix.)
A new helper to let us pretend that we called lstat() when we know
our cache_entry is up-to-date via fsmonitor.
Needs review.
source: <xmqqcyykig1l.fsf@gitster.g>
* js/doc-unit-tests (2023-11-10) 3 commits
(merged to 'next' on 2023-11-10 at 7d00ffd06b)
+ ci: run unit tests in CI
+ unit tests: add TAP unit test framework
+ unit tests: add a project plan document
(this branch is used by js/doc-unit-tests-with-cmake.)
Process to add some form of low-level unit tests has started.
Will cook in 'next'.
source: <cover.1699555664.git.steadmon@google.com>
* js/doc-unit-tests-with-cmake (2023-11-10) 7 commits
(merged to 'next' on 2023-11-10 at b4503c9c8c)
+ cmake: handle also unit tests
+ cmake: use test names instead of full paths
+ cmake: fix typo in variable name
+ artifacts-tar: when including `.dll` files, don't forget the unit-tests
+ unit-tests: do show relative file paths
+ unit-tests: do not mistake `.pdb` files for being executable
+ cmake: also build unit tests
(this branch uses js/doc-unit-tests.)
Update the base topic to work with CMake builds.
Will cook in 'next'.
source: <pull.1579.v3.git.1695640836.gitgitgadget@gmail.com>
* jc/rerere-cleanup (2023-08-25) 4 commits
- rerere: modernize use of empty strbuf
- rerere: try_merge() should use LL_MERGE_ERROR when it means an error
- rerere: fix comment on handle_file() helper
- rerere: simplify check_one_conflict() helper function
Code clean-up.
Not ready to be reviewed yet.
source: <20230824205456.1231371-1-gitster@pobox.com>
* rj/status-bisect-while-rebase (2023-10-16) 1 commit
- status: fix branch shown when not only bisecting
"git status" is taught to show both the branch being bisected and
being rebased when both are in effect at the same time.
Needs review.
source: <2e24ca9b-9c5f-f4df-b9f8-6574a714dfb2@gmail.com>
--------------------------------------------------
[Discarded]
* jc/strbuf-comment-line-char (2023-11-01) 4 commits
. strbuf: move env-using functions to environment.c
. strbuf: make add_lines() public
. strbuf_add_commented_lines(): drop the comment_line_char parameter
. strbuf_commented_addf(): drop the comment_line_char parameter
Code simplification that goes directly against a past libification
topic. It is hard to judge because the "libification" is done
piecewise without seemingly clear design principle.
Will discard.
source: <cover.1698791220.git.jonathantanmy@google.com>
^ permalink raw reply
* Re: [PATCH v3 0/2] send-email: avoid duplicate specification warnings
From: Junio C Hamano @ 2023-11-16 22:32 UTC (permalink / raw)
To: Todd Zullinger
Cc: git, Jeff King, Ævar Arnfjörð Bjarmason,
Ondřej Pohořelský
In-Reply-To: <20231116193014.470420-1-tmz@pobox.com>
Todd Zullinger <tmz@pobox.com> writes:
>> This sounds like something that is worth describing in the log
>> message (and Release Notes).
>
> I think the new commit messages describe the changes better. I didn't
> include anything in RelNotes as I was presuming we'd leave this for
> 2.44 rather than risk causing any problems this late in the 2.43 cycle.
> If you think the risk is low and/or the benefit is high, I can add it to
> the 2.43.0 RelNotes.
Please don't worry about the release notes, which I'll do only when
the topic hits the 'master' branch. It was meant primarily as a
note to myself. And I agree that this would be material for 2.44
and later.
Both patches, plus the stray single quote fix patch, looked good.
Thanks.
^ permalink raw reply
* Re: gitmodulesSymlink .gitmodules is a symbolic link
From: Jeff King @ 2023-11-16 20:27 UTC (permalink / raw)
To: flobee.code; +Cc: git
In-Reply-To: <CAM30mqasEjEBbn1vSUwvKP6LLjAFSw3xoeLrB1zLWnH37Z2fUg@mail.gmail.com>
On Wed, Nov 15, 2023 at 05:01:10PM +0100, flobee.code wrote:
> But in general I think the exclusion of symlinks to git system files
> is a mistake. It is implemented too sweepingly in my eyes.
I agree that the fact that we reject these at such a low level makes it
hard to use the tooling to rewrite the history to fix it.
But because of the security implications of out-of-tree symlinks from
untrusted repositories, it's important to catch these consistently. I
see your use case is for in-tree links, but detecting that makes the
checks much more complex. It also has always been the case that symlinks
do not behave consistently, as Git does not follow them when reading
in-index versions of meta files.
> And `git` itself also aborts. So I can't solve the problems this way.
>
> git filter-branch --tree-filter 'rm -f .gitmodules' HEAD
> Rewrite [SomeHash] (3/185) (0 seconds passed, remaining 0 predicted) \
> error: Invalid path '.gitmodules' Could not initialize the index
I didn't test, but you could probably get by with using "git replace" to
first fix up the offending trees, and then run filter-branch (though
there may be a lot of such trees, so you'd probably to script that
step). I also suspect that filter-repo would handle this better:
https://github.com/newren/git-filter-repo
but didn't try it myself.
-Peff
^ permalink raw reply
* Re: tb/merge-tree-write-pack, was Re: What's cooking in git.git (Nov 2023, #04; Thu, 9)
From: Jeff King @ 2023-11-16 20:17 UTC (permalink / raw)
To: Johannes Schindelin; +Cc: Junio C Hamano, Taylor Blau, git
In-Reply-To: <14bdff31-5229-bbd5-3715-f77f52b832d5@gmx.de>
On Wed, Nov 15, 2023 at 01:57:00PM +0100, Johannes Schindelin wrote:
> - The scenario I want to address (and that I assumed the patch series
> under discussion tried to address, too) is a very specific, server-side
> scenario where many `merge-tree`/`replay` runs produce _many_ loose
> objects. Quite a fraction of those are produced by processes that run
> into a SIGTERM-enforced timeout, and the `tmp_objdir` approach would
> naturally help: unneeded loose objects would not even be added to the
> primary object database but be reaped with the temporary object
> databases.
Thanks, this paragraph helped me to understand why you are interested in
tmp_objdir in the first place (as opposed to just doing a gc-auto repack
after finishing the merge).
-Peff
^ permalink raw reply
* Re: [PATCH v3 1/2] perl: bump the required Perl version to 5.8.1 from 5.8.0
From: Jeff King @ 2023-11-16 20:16 UTC (permalink / raw)
To: Todd Zullinger
Cc: Junio C Hamano, git, Ævar Arnfjörð Bjarmason,
Ondřej Pohořelský
In-Reply-To: <20231116193014.470420-2-tmz@pobox.com>
On Thu, Nov 16, 2023 at 02:30:10PM -0500, Todd Zullinger wrote:
> The following commit will make use of a Getopt::Long feature which is
> only present in Perl >= 5.8.1. Document that as the minimum version we
> support.
>
> Many of our Perl scripts will continue to run with 5.8.0 but this change
> allows us to adjust them as needed without breaking any promises to our
> users.
>
> The Perl requirement was last changed in d48b284183 (perl: bump the
> required Perl version to 5.8 from 5.6.[21], 2010-09-24). At that time,
> 5.8.0 was 8 years old. It is now over 21 years old.
Thanks, IMHO this is long overdue. You mentioned 5.10 elsewhere in the
thread, and it came up recently in a discussion (it would allow the use
of "//" for defined-or). So we could perhaps go a bit farther. But I am
also fine with 5.8.1 for now, if that is all it takes for this fix (and
I expect the chance that it causes a problem for anybody to be close to
zero).
> Signed-off-by: Todd Zullinger <tmz@pobox.com>
> ---
> I debated changing all the 'use 5.008;' lines here, as most don't
> actually require a newer Perl, but the previous bump did the same.
>
> I can see the merit in either direction.
>
> Changing it allows future contributors to be confident in relying on
> 5.8.1 features.
>
> Not changing it allows anyone stuck on 5.8.0 to continue using the perl
> scripts which don't actually require 5.8.1.
Yeah, I can see both sides of the argument. I think I'd err on the side
of bumping (as you did here). That lets somebody who will be affected
know immediately, rather than only finding out when we randomly depend
on a feature later.
All of this discussion could likewise go in the commit message. :)
> Tangentially, the Perl docs for 'use' function recommend against the
> 5.008001 form[1]:
>
> Specifying VERSION as a numeric argument of the form 5.024001 should
> generally be avoided as older less readable syntax compared to
> v5.24.1. Before perl 5.8.0 released in 2002 the more verbose numeric
> form was the only supported syntax, which is why you might see it in
> older code.
>
> use v5.24.1; # compile time version check
> use 5.24.1; # ditto
> use 5.024_001; # ditto; older syntax compatible with perl 5.6
>
> I'm not enough of a Perl coder to have a strong preference or desire to
> push for such a change, but I thought it was worth mentioning in case
> others wonder why we're using the 5.008001 form.
I doubt it matters too much either way. I suspect at the time we moved
to v5.8 the nicer syntax was still pretty new (having only been
introduced by v5.6, which we were moving off of) and that older versions
of perl might not give as nice a message when they see it. But given
that 5.6 is now 23 years old, we can probably assume nobody will use it
(or at least they will be accustomed to whatever ugly message it
produces).
But IMHO that should be done as a separate patch anyway.
-Peff
^ permalink raw reply
* Re: [PATCH] ci: avoid running the test suite _twice_
From: Jeff King @ 2023-11-16 20:02 UTC (permalink / raw)
To: Josh Steadmon
Cc: Johannes Schindelin via GitGitGadget, Phillip Wood, git,
Johannes Schindelin
In-Reply-To: <ZVU4EVcj0MDrSNcG@google.com>
On Wed, Nov 15, 2023 at 01:28:49PM -0800, Josh Steadmon wrote:
> The first part is easy, but I don't see a good way to get both shell
> tests and unit tests executing under the same `prove` process. For shell
> tests, we pass `--exec '$(TEST_SHELL_PATH_SQ)'` to prove, meaning that
> we use the specified shell as an interpreter for the test files. That
> will not work for unit test executables.
Yes, it's unfortunate that you can't set the "exec" flag per-script
(especially because without --exec it will auto-detect the right thing,
but then of course it won't use TEST_SHELL_PATH). But we can intercept
and do it ourselves, like:
diff --git a/t/Makefile b/t/Makefile
index 225aaf78ed..0b7c028eea 100644
--- a/t/Makefile
+++ b/t/Makefile
@@ -61,7 +61,7 @@ failed:
test -z "$$failed" || $(MAKE) $$failed
prove: pre-clean check-chainlint $(TEST_LINT)
- @echo "*** prove ***"; $(CHAINLINTSUPPRESS) $(PROVE) --exec '$(TEST_SHELL_PATH_SQ)' $(GIT_PROVE_OPTS) $(T) :: $(GIT_TEST_OPTS)
+ @echo "*** prove ***"; TEST_SHELL_PATH='$(TEST_SHELL_PATH_SQ)' $(CHAINLINTSUPPRESS) $(PROVE) --exec ./run-test.sh $(GIT_PROVE_OPTS) $(T) $(UNIT_TESTS) :: $(GIT_TEST_OPTS)
$(MAKE) clean-except-prove-cache
$(T):
diff --git a/t/run-test.sh b/t/run-test.sh
new file mode 100755
index 0000000000..69944029c8
--- /dev/null
+++ b/t/run-test.sh
@@ -0,0 +1,10 @@
+#!/bin/sh
+
+case "$1" in
+*.sh)
+ exec ${TEST_SHELL_PATH:-/bin/sh} "$@"
+ ;;
+*)
+ exec "$@"
+ ;;
+esac
You can actually do this inside the prove script using their plugin
interface, but the necessary bits are somewhat arcane.
> We could bundle all the unit tests into a single shell script, but then
> we lose parallelization and add hoops to jump through to determine what
> breaks. Or we could autogenerate a corresponding shell script to run
> each individual unit test, but that seems gross. Of course, these are
> hypothetical concerns for now, since we only have a single unit test at
> the moment.
We can't just stick them all in a single script; there must be exactly
one "plan" line in the TAP output from a given source. I had imagined
just manually adding a thin wrapper for each ("t9970-unit-strbuf" or
something). But it would also be easy to autogenerate them while
compiling. (Although all of that is moot with the wrapper I showed
above).
> There's also the issue that the shell test arguments we pass on from
> prove would be shared with the unit tests. That's fine for now, as
> t-strbuf doesn't accept any runtime arguments, but it's possible that
> either the framework or individual unit tests might grow to need
> arguments, and it might not be convenient to stay compatible with the
> shell tests.
Sharing the options between the two seems like a benefit to me. I'd
think that "-v" and "-i" would be useful, at least. Options which don't
apply (e.g., "--root") could be quietly ignored.
-Peff
^ permalink raw reply related
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox