* [PATCH 0/2] worktree: copy-on-write creation and shared-branch worktrees
@ 2026-06-05 18:49 Jason Newton via GitGitGadget
2026-06-05 18:49 ` [PATCH 1/2] worktree: add --reflink for copy-on-write worktree creation Jason Newton via GitGitGadget
` (5 more replies)
0 siblings, 6 replies; 21+ messages in thread
From: Jason Newton via GitGitGadget @ 2026-06-05 18:49 UTC (permalink / raw)
To: git; +Cc: Jason Newton
When many worktrees share one repository -- e .g. a fleet of agents each
needing an isolated checkout -- "git worktree add" is costly at scale.
Objects are shared via the common dir, but the working tree is not: each add
rewrites every tracked file, so N worktrees cost N full checkouts of disk
and I/O. And a branch can only be checked out in one worktree.
Patch 1 adds "git worktree add --reflink": on a copy-on-write filesystem it
populates the new worktree by reflinking the current worktree's files and
index, then "git reset --hard" rewrites only the paths that differ from . A
reflink_file() helper in copy.c uses FICLONE (Linux) and clonefile()
(macOS); elsewhere (other filesystems, Windows) it is probed up front and
falls back to a normal checkout. Defaulting is via the worktree.reflink
config (true/false/auto); --no-reflink overrides.
Patch 2 lets a branch be checked out in several worktrees, for parallel work
on one checkout. A branch mid-rebase or mid-bisect elsewhere is still
refused.
Benchmark (Linux-kernel fork, 93k files, ~33 GB tree incl. build output,
btrfs): a normal add allocates ~0.9 GB of real disk per worktree (~5.3 GB
for four, linear); --reflink allocates ~0 at any count and also carries the
untracked build tree. ("Real disk" = btrfs exclusive bytes.)
worktree-reflink-bench
[https://github.com/user-attachments/assets/e3e721c8-2206-4b78-ad08-21677ef30753]
Note: patch 2 changes a default (same-branch checkout now allowed); two
t2400 assertions were updated accordingly.
Jason Newton (2):
worktree: add --reflink for copy-on-write worktree creation
worktree: allow sharing a checked-out branch across worktrees
Documentation/config/worktree.adoc | 10 ++
Documentation/git-worktree.adoc | 47 +++++-
builtin/worktree.c | 257 ++++++++++++++++++++++++++++-
copy.c | 65 ++++++++
copy.h | 13 ++
t/t2400-worktree-add.sh | 119 ++++++++++++-
6 files changed, 493 insertions(+), 18 deletions(-)
base-commit: c69baaf57ba26cf117c2b6793802877f19738b0d
Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-git-2317%2Fnevion%2Fworktree-reflink-cow-v1
Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-git-2317/nevion/worktree-reflink-cow-v1
Pull-Request: https://github.com/git/git/pull/2317
--
gitgitgadget
^ permalink raw reply [flat|nested] 21+ messages in thread* [PATCH 1/2] worktree: add --reflink for copy-on-write worktree creation 2026-06-05 18:49 [PATCH 0/2] worktree: copy-on-write creation and shared-branch worktrees Jason Newton via GitGitGadget @ 2026-06-05 18:49 ` Jason Newton via GitGitGadget 2026-06-05 18:49 ` [PATCH 2/2] worktree: allow sharing a checked-out branch across worktrees Jason Newton via GitGitGadget ` (4 subsequent siblings) 5 siblings, 0 replies; 21+ messages in thread From: Jason Newton via GitGitGadget @ 2026-06-05 18:49 UTC (permalink / raw) To: git; +Cc: Jason Newton, Jason Newton From: Jason Newton <nevion@gmail.com> Creating many worktrees from the same base -- for example to run a fleet of automated agents in parallel -- is expensive today: every "git worktree add" materializes the entire working tree by writing each tracked file out from the object store. The objects are shared via the common directory, but the working tree is not: N worktrees mean N full checkouts on disk and N times the file I/O. Add a "--reflink" option that, on copy-on-write filesystems, populates the new worktree by reflinking the current worktree's files and index instead. The subsequent "git reset --hard" then only rewrites the paths that actually differ between the current worktree and <commit-ish>; everything else (including untracked files such as build outputs) keeps sharing storage with the source until modified. Because the cloned index still carries the source files' stat data, it is refreshed against the reflinked files first so that reset recognizes the unchanged paths as up to date and leaves them sharing extents rather than rewriting them. The clones are made by a new reflink_file() helper in copy.c, which uses the FICLONE ioctl on Linux and clonefile() on macOS and reports an error otherwise so callers fall back to a normal copy. Support is probed up front; when unavailable -- including on filesystems without copy-on-write and on platforms such as Windows that lack a reflink primitive -- "--reflink" transparently falls back to an ordinary checkout, so the worst case is no slower than today rather than a byte-for-byte copy of the source tree. The directory walk skips the new worktree itself when it lives inside the source one, and preserves symlinks and modes. The behavior can be made the default with the worktree.reflink configuration ("true", "false" or "auto", the last suppressing the unsupported-filesystem warning), and turned off per-invocation with --no-reflink. A configured default degrades quietly in modes that cannot reflink (--orphan, --no-checkout) instead of erroring, so enabling it never breaks those commands. The checkout step continues to honor checkout.workers, so parallel checkout composes with --reflink for the paths that do need rewriting. Signed-off-by: Jason Newton <nevion@gmail.com> --- Documentation/config/worktree.adoc | 10 ++ Documentation/git-worktree.adoc | 30 +++- builtin/worktree.c | 227 ++++++++++++++++++++++++++++- copy.c | 65 +++++++++ copy.h | 13 ++ t/t2400-worktree-add.sh | 88 +++++++++++ 6 files changed, 431 insertions(+), 2 deletions(-) diff --git a/Documentation/config/worktree.adoc b/Documentation/config/worktree.adoc index a248076ea5..d3a03c86d4 100644 --- a/Documentation/config/worktree.adoc +++ b/Documentation/config/worktree.adoc @@ -17,3 +17,13 @@ Note that setting `worktree.useRelativePaths` to "`true`" implies enabling the `extensions.relativeWorktrees` config (see linkgit:git-config[1]), thus making it incompatible with older versions of Git. + +`worktree.reflink`:: + Controls whether `git worktree add` populates new worktrees with + copy-on-write (reflink) clones, as if `--reflink` had been given + (see linkgit:git-worktree[1]). May be set to "`true`", "`false`" + (the default), or "`auto`". With "`true`", a filesystem that does + not support reflinks produces a warning before falling back to an + ordinary checkout; with "`auto`", the fallback is silent. An + explicit `--reflink` or `--no-reflink` on the command line + overrides this setting. diff --git a/Documentation/git-worktree.adoc b/Documentation/git-worktree.adoc index fbf8426cd9..1ca81718b7 100644 --- a/Documentation/git-worktree.adoc +++ b/Documentation/git-worktree.adoc @@ -10,7 +10,7 @@ SYNOPSIS -------- [synopsis] git worktree add [-f] [--detach] [--checkout] [--lock [--reason <string>]] - [--orphan] [(-b | -B) <new-branch>] <path> [<commit-ish>] + [--orphan] [--reflink] [(-b | -B) <new-branch>] <path> [<commit-ish>] git worktree list [-v | --porcelain [-z]] git worktree lock [--reason <string>] <worktree> git worktree move <worktree> <new-path> @@ -213,6 +213,34 @@ To remove a locked worktree, specify `--force` twice. such as configuring sparse-checkout. See "Sparse checkout" in linkgit:git-read-tree[1]. +`--reflink`:: +`--no-reflink`:: + Populate the new worktree by creating copy-on-write (reflink) + clones of the current worktree's files and index instead of + writing every file out from the object store. The checkout that + follows then only has to rewrite the paths that actually differ + between the current worktree and _<commit-ish>_; everything else + (including untracked files such as build outputs) keeps sharing + storage with the source worktree until modified. This makes + `add` much faster and far cheaper on disk when creating many + worktrees from the same base. `--no-reflink` forces an ordinary + checkout, overriding the `worktree.reflink` configuration. ++ +Reflinks require a copy-on-write filesystem (for example btrfs, XFS, +bcachefs or ZFS on Linux, or APFS on macOS). On filesystems or platforms +that do not support reflinks, `--reflink` transparently falls back to an +ordinary checkout. Because the source worktree's untracked and ignored +files are cloned as well, only use `--reflink` when that is acceptable. ++ +This option cannot be combined with `--no-checkout` or `--orphan`. It can +be enabled by default with the `worktree.reflink` configuration; see +linkgit:git-config[1]. ++ +The checkout that populates a new worktree also honors the +`checkout.workers` configuration (see linkgit:git-config[1]), so setting it +parallelizes the file writes and can further speed up `add`, with or +without `--reflink`. + `--guess-remote`:: `--no-guess-remote`:: With `worktree add <path>`, without _<commit-ish>_, instead diff --git a/builtin/worktree.c b/builtin/worktree.c index d21c43fde3..973da33051 100644 --- a/builtin/worktree.c +++ b/builtin/worktree.c @@ -30,7 +30,7 @@ #define BUILTIN_WORKTREE_ADD_USAGE \ N_("git worktree add [-f] [--detach] [--checkout] [--lock [--reason <string>]]\n" \ - " [--orphan] [(-b | -B) <new-branch>] <path> [<commit-ish>]") + " [--orphan] [--reflink] [(-b | -B) <new-branch>] <path> [<commit-ish>]") #define BUILTIN_WORKTREE_LIST_USAGE \ N_("git worktree list [-v | --porcelain [-z]]") @@ -47,6 +47,11 @@ #define BUILTIN_WORKTREE_UNLOCK_USAGE \ N_("git worktree unlock <worktree>") +/* values for add_opts.reflink and the worktree.reflink config */ +#define REFLINK_OFF 0 +#define REFLINK_ON 1 /* warn and fall back if unsupported */ +#define REFLINK_AUTO 2 /* silently fall back if unsupported */ + #define WORKTREE_ADD_DWIM_ORPHAN_INFER_TEXT \ _("No possible source branch, inferring '--orphan'") @@ -123,6 +128,7 @@ struct add_opts { int checkout; int orphan; int relative_paths; + int reflink; const char *keep_locked; }; @@ -130,6 +136,7 @@ static int show_only; static int verbose; static int guess_remote; static int use_relative_paths; +static int reflink_config = REFLINK_OFF; static timestamp_t expire; static int git_worktree_config(const char *var, const char *value, @@ -141,6 +148,13 @@ static int git_worktree_config(const char *var, const char *value, } else if (!strcmp(var, "worktree.userelativepaths")) { use_relative_paths = git_config_bool(var, value); return 0; + } else if (!strcmp(var, "worktree.reflink")) { + if (value && !strcmp(value, "auto")) + reflink_config = REFLINK_AUTO; + else + reflink_config = git_config_bool(var, value) ? + REFLINK_ON : REFLINK_OFF; + return 0; } return git_default_config(var, value, ctx, cb); @@ -397,6 +411,182 @@ worktree_copy_cleanup: free(to_file); } +/* + * Probe whether the filesystem backing "dir" supports reflinks. We do this + * once up front so that, on filesystems without copy-on-write support (or on + * platforms such as Windows that lack a reflink primitive entirely), we can + * fall back to a normal checkout instead of byte-copying the whole source + * working tree -- which would include untracked files and be slower than the + * checkout we are trying to avoid. + */ +static int reflink_supported(const char *dir) +{ + struct strbuf src = STRBUF_INIT, dst = STRBUF_INIT; + int fd, ok = 0; + + strbuf_addf(&src, "%s/.git-reflink-probe-src", dir); + strbuf_addf(&dst, "%s/.git-reflink-probe-dst", dir); + + fd = open(src.buf, O_WRONLY | O_CREAT | O_TRUNC, 0600); + if (fd >= 0) { + write_in_full(fd, "x", 1); + close(fd); + if (!reflink_file(dst.buf, src.buf, 0600)) + ok = 1; + unlink(dst.buf); + } + unlink(src.buf); + + strbuf_release(&src); + strbuf_release(&dst); + return ok; +} + +/* + * Reflink a single regular file, falling back to a regular copy when the + * clone fails for this particular file (for example across mount points). + */ +static int reflink_or_copy(const char *dst, const char *src, int mode) +{ + if (!reflink_file(dst, src, mode)) + return 0; + return copy_file(dst, src, mode); +} + +/* + * Recursively copy the working-tree directory "src" into "dst" using reflinks + * for regular files. Directory entries that resolve to the destination + * worktree itself (identified by skip_dev/skip_ino, which matters when the new + * worktree lives inside the source one) and the top-level ".git" gitfile are + * skipped. + */ +static int reflink_tree(const char *src, const char *dst, + dev_t skip_dev, ino_t skip_ino, int top) +{ + struct strbuf s = STRBUF_INIT, d = STRBUF_INIT; + DIR *dir; + struct dirent *de; + int ret = 0; + + dir = opendir(src); + if (!dir) + return error_errno(_("could not open directory '%s'"), src); + + while (!ret && (de = readdir(dir))) { + struct stat st; + + if (is_dot_or_dotdot(de->d_name)) + continue; + if (top && !strcmp(de->d_name, ".git")) + continue; + + strbuf_reset(&s); + strbuf_addf(&s, "%s/%s", src, de->d_name); + strbuf_reset(&d); + strbuf_addf(&d, "%s/%s", dst, de->d_name); + + if (lstat(s.buf, &st)) { + ret = error_errno(_("could not stat '%s'"), s.buf); + break; + } + + if (S_ISDIR(st.st_mode)) { + /* never recurse into the new worktree itself */ + if (st.st_dev == skip_dev && st.st_ino == skip_ino) + continue; + if (mkdir(d.buf, st.st_mode & 07777) && errno != EEXIST) { + ret = error_errno(_("could not create directory '%s'"), d.buf); + break; + } + ret = reflink_tree(s.buf, d.buf, skip_dev, skip_ino, 0); + } else if (S_ISLNK(st.st_mode)) { + struct strbuf link = STRBUF_INIT; + + if (strbuf_readlink(&link, s.buf, st.st_size)) + ret = error_errno(_("could not read symlink '%s'"), s.buf); + else if (symlink(link.buf, d.buf)) + ret = error_errno(_("could not create symlink '%s'"), d.buf); + strbuf_release(&link); + } else if (S_ISREG(st.st_mode)) { + if (reflink_or_copy(d.buf, s.buf, st.st_mode)) + ret = error_errno(_("could not copy '%s' to '%s'"), + s.buf, d.buf); + } + /* silently skip fifos, sockets and device nodes */ + } + + closedir(dir); + strbuf_release(&s); + strbuf_release(&d); + return ret; +} + +/* + * Populate the new worktree at "path" by reflinking the current worktree's + * files and index. The subsequent "git reset --hard" then only has to rewrite + * the paths that actually differ between the source and <commit-ish>, leaving + * everything else sharing storage with the source. Returns 1 when reflinks are + * unavailable so the caller can fall back to a plain checkout. + */ +static int reflink_worktree(const char *path, const char *wt_git_dir, + const struct add_opts *opts, struct strvec *child_env) +{ + const char *src_wt = the_repository->worktree; + char *src_index = NULL, *dst_index = NULL; + struct stat dst_st; + int ret = 0; + + if (!src_wt) + return error(_("--reflink needs a source working tree, but this " + "repository does not have one")); + + if (!reflink_supported(path)) { + /* In auto mode the fallback is expected, so stay quiet. */ + if (!opts->quiet && opts->reflink != REFLINK_AUTO) + warning(_("the filesystem at '%s' does not support reflinks; " + "falling back to a regular checkout"), path); + return 1; + } + + if (stat(path, &dst_st)) + return error_errno(_("could not stat '%s'"), path); + + if ((ret = reflink_tree(src_wt, path, dst_st.st_dev, dst_st.st_ino, 1))) + return ret; + + /* + * Clone the source index so the following reset sees the source's + * state and only materializes the differences to <commit-ish>. + */ + src_index = repo_git_path(the_repository, "index"); + dst_index = xstrfmt("%s/index", wt_git_dir); + if (!access(src_index, F_OK) && + reflink_or_copy(dst_index, src_index, 0666)) { + ret = error_errno(_("could not copy index to '%s'"), dst_index); + goto out; + } + + /* + * The cloned index still carries the source files' stat information. + * Refresh it against the freshly reflinked files so that "git reset" + * recognizes unchanged paths as up to date and leaves them sharing + * storage instead of rewriting (and thus un-sharing) them. + */ + if (!access(dst_index, F_OK)) { + struct child_process cp = CHILD_PROCESS_INIT; + cp.git_cmd = 1; + strvec_pushl(&cp.args, "update-index", "-q", "--refresh", NULL); + strvec_pushv(&cp.env, child_env->v); + /* a dirty working tree is not an error here */ + run_command(&cp); + } + +out: + free(src_index); + free(dst_index); + return ret; +} + static int checkout_worktree(const struct add_opts *opts, struct strvec *child_env) { @@ -589,6 +779,20 @@ static int add_worktree(const char *path, const char *refname, (ret = make_worktree_orphan(refname, opts, &child_env))) goto done; + /* + * When --reflink is requested and the filesystem supports it, copy the + * current worktree (and its index) into the new one using copy-on-write + * clones. checkout_worktree() then only rewrites the paths that differ + * from <commit-ish>. reflink_worktree() returns 1 when reflinks are not + * available, in which case we just do an ordinary checkout below. + */ + if (opts->checkout && opts->reflink) { + ret = reflink_worktree(path, sb_repo.buf, opts, &child_env); + if (ret < 0) + goto done; + ret = 0; + } + if (opts->checkout && (ret = checkout_worktree(opts, &child_env))) goto done; @@ -801,6 +1005,7 @@ static int add(int ac, const char **av, const char *prefix, const char *lock_reason = NULL; int keep_locked = 0; int used_new_branch_options; + int reflink_cli = -1; struct option options[] = { OPT__FORCE(&opts.force, N_("checkout <branch> even if already checked out in other worktree"), @@ -823,6 +1028,8 @@ static int add(int ac, const char **av, const char *prefix, N_("try to match the new branch name with a remote-tracking branch")), OPT_BOOL(0, "relative-paths", &opts.relative_paths, N_("use relative paths for worktrees")), + OPT_BOOL(0, "reflink", &reflink_cli, + N_("populate the worktree using copy-on-write clones when supported")), OPT_END() }; int ret; @@ -842,6 +1049,24 @@ static int add(int ac, const char **av, const char *prefix, if (opts.orphan && !opts.checkout) die(_("options '%s' and '%s' cannot be used together"), "--orphan", "--no-checkout"); + + /* + * Resolve whether to reflink: an explicit --reflink/--no-reflink on + * the command line wins, otherwise fall back to the worktree.reflink + * configuration (which may select the "auto" mode). + */ + opts.reflink = (reflink_cli != -1) ? reflink_cli : reflink_config; + if (opts.reflink && (opts.orphan || !opts.checkout)) { + /* + * Reflinking is incompatible with these; only complain when it + * was explicitly requested, otherwise quietly do a plain + * checkout so a configured default does not break these modes. + */ + if (reflink_cli == REFLINK_ON) + die(_("options '%s' and '%s' cannot be used together"), + "--reflink", opts.orphan ? "--orphan" : "--no-checkout"); + opts.reflink = REFLINK_OFF; + } if (opts.orphan && ac == 2) die(_("option '%s' and commit-ish cannot be used together"), "--orphan"); diff --git a/copy.c b/copy.c index b668209b6c..d2c8ce5209 100644 --- a/copy.c +++ b/copy.c @@ -7,6 +7,21 @@ #include "strbuf.h" #include "abspath.h" +#if defined(__linux__) +#include <sys/ioctl.h> +/* + * FICLONE lives in <linux/fs.h>, but including that header tends to clash + * with the libc headers git already pulls in, so define it ourselves if it + * is missing. The value is part of the stable kernel uapi. + */ +#ifndef FICLONE +#define FICLONE _IOW(0x94, 9, int) +#endif +#elif defined(__APPLE__) +#include <sys/attr.h> +#include <sys/clonefile.h> +#endif + int copy_fd(int ifd, int ofd) { while (1) { @@ -72,3 +87,53 @@ int copy_file_with_time(const char *dst, const char *src, int mode) return copy_times(dst, src); return status; } + +int reflink_file(const char *dst, const char *src, int mode) +{ +#if defined(__APPLE__) + /* + * clonefile() refuses to operate when the destination exists and + * copies the source's permissions for us, so "mode" is unused here. + */ + (void)mode; + if (clonefile(src, dst, 0) < 0) + return -1; + if (adjust_shared_perm(the_repository, dst)) + return -1; + return 0; +#elif defined(__linux__) + int fdi, fdo, status; + + mode = (mode & 0111) ? 0777 : 0666; + if ((fdi = open(src, O_RDONLY)) < 0) + return -1; + if ((fdo = open(dst, O_WRONLY | O_CREAT | O_EXCL, mode)) < 0) { + int saved = errno; + close(fdi); + errno = saved; + return -1; + } + status = ioctl(fdo, FICLONE, fdi); + close(fdi); + if (status < 0) { + int saved = errno; + close(fdo); + /* we created an empty file above; do not leave it behind */ + unlink(dst); + errno = saved; + return -1; + } + if (close(fdo) != 0) + return -1; + if (adjust_shared_perm(the_repository, dst)) + return -1; + return 0; +#else + /* No reflink support on this platform (e.g. Windows). */ + (void)dst; + (void)src; + (void)mode; + errno = ENOSYS; + return -1; +#endif +} diff --git a/copy.h b/copy.h index 2af77cba86..a8646f7ff5 100644 --- a/copy.h +++ b/copy.h @@ -7,4 +7,17 @@ int copy_fd(int ifd, int ofd); int copy_file(const char *dst, const char *src, int mode); int copy_file_with_time(const char *dst, const char *src, int mode); +/* + * Create "dst" as a copy-on-write (reflink) clone of the regular file + * "src", so that the two files share their data blocks until one of + * them is modified. "dst" must not already exist. + * + * This only succeeds on filesystems that support block cloning (e.g. + * btrfs, XFS or bcachefs on Linux, APFS on macOS). When the platform or + * filesystem does not support reflinks, -1 is returned with errno set to + * ENOSYS (or the underlying error). Callers are expected to fall back to + * copy_file() in that case. Returns 0 on success. + */ +int reflink_file(const char *dst, const char *src, int mode); + #endif /* COPY_H */ diff --git a/t/t2400-worktree-add.sh b/t/t2400-worktree-add.sh index 58b4445cc4..56fb79179a 100755 --- a/t/t2400-worktree-add.sh +++ b/t/t2400-worktree-add.sh @@ -1248,4 +1248,92 @@ test_expect_success 'relative worktree sets extension config' ' test_cmp_config -C repo true extensions.relativeworktrees ' +test_expect_success '--reflink produces a correct checkout on any filesystem' ' + test_when_finished "rm -rf reflinkrepo" && + git init reflinkrepo && + ( + cd reflinkrepo && + test_commit base file.txt base && + test_commit next file.txt next && + git worktree add --reflink wt -b reflinkbr base && + echo base >expect && + test_cmp expect wt/file.txt && + git -C wt status --porcelain >status && + test_must_be_empty status + ) +' + +test_expect_success '--reflink rejects incompatible options' ' + test_must_fail git worktree add --reflink --no-checkout wt-bad HEAD 2>err && + test_grep "cannot be used together" err && + test_must_fail git worktree add --reflink --orphan wt-bad2 2>err && + test_grep "cannot be used together" err +' + +test_expect_success 'worktree.reflink config drives a correct checkout' ' + test_when_finished "rm -rf reflinkcfg" && + git init reflinkcfg && + ( + cd reflinkcfg && + test_commit cfg file.txt value && + git -c worktree.reflink=true worktree add wt HEAD && + echo value >expect && + test_cmp expect wt/file.txt + ) +' + +test_expect_success 'worktree.reflink=auto does not break --orphan' ' + test_when_finished "rm -rf reflinkorphan" && + git init reflinkorphan && + ( + cd reflinkorphan && + test_commit base file.txt value && + git -c worktree.reflink=auto worktree add --orphan wt && + git -C wt symbolic-ref --short HEAD >actual && + echo wt >expect && + test_cmp expect actual + ) +' + +test_expect_success '--no-reflink overrides worktree.reflink' ' + test_when_finished "rm -rf reflinkoff" && + git init reflinkoff && + ( + cd reflinkoff && + test_commit base file.txt value && + git -c worktree.reflink=true worktree add --no-reflink wt HEAD && + echo value >expect && + test_cmp expect wt/file.txt + ) +' + +# Reflinks are only available on copy-on-write filesystems (btrfs, XFS, +# bcachefs, APFS, ...). Where they are unavailable, --reflink transparently +# falls back to a regular checkout, which the test above already covers. +test_lazy_prereq REFLINK ' + echo probe >reflink-src && + cp --reflink=always reflink-src reflink-dst 2>/dev/null +' + +test_expect_success REFLINK '--reflink carries untracked files and reconciles changes' ' + test_when_finished "rm -rf cowrepo" && + git init cowrepo && + ( + cd cowrepo && + test_commit same same.txt content && + test_commit old change.txt old-value && + test_commit new change.txt new-value && + echo artifact >untracked.bin && + git worktree add --reflink wt -b cowbr old && + echo content >expect && + test_cmp expect wt/same.txt && + echo old-value >expect && + test_cmp expect wt/change.txt && + echo artifact >expect && + test_cmp expect wt/untracked.bin && + git -C wt status --porcelain >status && + grep "?? untracked.bin" status + ) +' + test_done -- gitgitgadget ^ permalink raw reply related [flat|nested] 21+ messages in thread
* [PATCH 2/2] worktree: allow sharing a checked-out branch across worktrees 2026-06-05 18:49 [PATCH 0/2] worktree: copy-on-write creation and shared-branch worktrees Jason Newton via GitGitGadget 2026-06-05 18:49 ` [PATCH 1/2] worktree: add --reflink for copy-on-write worktree creation Jason Newton via GitGitGadget @ 2026-06-05 18:49 ` Jason Newton via GitGitGadget 2026-06-05 19:59 ` [PATCH 0/2] worktree: copy-on-write creation and shared-branch worktrees brian m. carlson ` (3 subsequent siblings) 5 siblings, 0 replies; 21+ messages in thread From: Jason Newton via GitGitGadget @ 2026-06-05 18:49 UTC (permalink / raw) To: git; +Cc: Jason Newton, Jason Newton From: Jason Newton <nevion@gmail.com> When spinning up several worktrees on the same checkout for parallel work (for example a fleet of agents working from one branch), git's refusal to check out a branch that is already checked out elsewhere is just in the way. The restriction exists to stop two worktrees from moving the same branch underneath each other, but plain parallel checkouts do not need that protection. Drop the restriction: "git worktree add <branch>" now checks out a branch even if it is in use by another worktree. The genuinely dangerous case is kept -- a branch that another worktree is in the middle of rebasing or bisecting is still refused, because a second checkout could corrupt that operation. die_if_branch_busy() performs that narrower check in place of the old die_if_checked_out(). The separate guard against force-updating (e.g. with -B) a branch in use elsewhere is left untouched. Signed-off-by: Jason Newton <nevion@gmail.com> --- Documentation/git-worktree.adoc | 17 +++++++++-------- builtin/worktree.c | 30 +++++++++++++++++++++++++++++- t/t2400-worktree-add.sh | 31 ++++++++++++++++++++++++------- 3 files changed, 62 insertions(+), 16 deletions(-) diff --git a/Documentation/git-worktree.adoc b/Documentation/git-worktree.adoc index 1ca81718b7..cc4c91b787 100644 --- a/Documentation/git-worktree.adoc +++ b/Documentation/git-worktree.adoc @@ -93,8 +93,9 @@ then, as a convenience, the new worktree is associated with a branch (call it _<branch>_) named after `$(basename <path>)`. If _<branch>_ doesn't exist, a new branch based on `HEAD` is automatically created as if `-b <branch>` was given. If _<branch>_ does exist, it will be checked out -in the new worktree, if it's not checked out anywhere else, otherwise the -command will refuse to create the worktree (unless `--force` is used). +in the new worktree, even if it is already checked out in another worktree. +(A branch that another worktree is in the middle of rebasing or bisecting is +refused unless `--force` is used.) + If _<commit-ish>_ is omitted, neither `--detach`, or `--orphan` is used, and there are no valid local branches (or remote branches if @@ -177,12 +178,12 @@ OPTIONS `-f`:: `--force`:: - By default, `add` refuses to create a new worktree when - _<commit-ish>_ is a branch name and is already checked out by - another worktree, or if _<path>_ is already assigned to some - worktree but is missing (for instance, if _<path>_ was deleted - manually). This option overrides these safeguards. To add a missing but - locked worktree path, specify `--force` twice. + `add` refuses to create a new worktree when _<commit-ish>_ is a + branch that another worktree is in the middle of rebasing or + bisecting, or if _<path>_ is already assigned to some worktree but + is missing (for instance, if _<path>_ was deleted manually). This + option overrides these safeguards. To add a missing but locked + worktree path, specify `--force` twice. + `move` refuses to move a locked worktree unless `--force` is specified twice. If the destination is already assigned to some other worktree but is diff --git a/builtin/worktree.c b/builtin/worktree.c index 973da33051..b457b015d1 100644 --- a/builtin/worktree.c +++ b/builtin/worktree.c @@ -648,6 +648,34 @@ static void setup_alternate_ref_dir(struct worktree *wt, const char *wt_git_path strbuf_release(&sb); } +/* + * Checking out a branch that is already checked out in another worktree is + * fine -- it is exactly what you want when spinning up several worktrees on + * the same checkout for parallel work. The one case that is still unsafe is a + * branch that another worktree is in the middle of rebasing or bisecting, + * since a second checkout could corrupt that operation, so refuse only that. + */ +static void die_if_branch_busy(const char *branch) +{ + struct worktree **worktrees = get_worktrees(); + int i; + + for (i = 0; worktrees[i]; i++) { + const struct worktree *wt = worktrees[i]; + + if (is_worktree_being_rebased(wt, branch) || + is_worktree_being_bisected(wt, branch)) { + const char *shortname = branch; + + skip_prefix(branch, "refs/heads/", &shortname); + die(_("'%s' is already used by worktree at '%s'"), + shortname, wt->path); + } + } + + free_worktrees(worktrees); +} + static int add_worktree(const char *path, const char *refname, const struct add_opts *opts) { @@ -675,7 +703,7 @@ static int add_worktree(const char *path, const char *refname, refs_ref_exists(get_main_ref_store(the_repository), symref.buf)) { is_branch = 1; if (!opts->force) - die_if_checked_out(symref.buf, 0); + die_if_branch_busy(symref.buf); } commit = lookup_commit_reference_by_name(refname); if (!commit && !opts->orphan) diff --git a/t/t2400-worktree-add.sh b/t/t2400-worktree-add.sh index 56fb79179a..6a1eb72ac7 100755 --- a/t/t2400-worktree-add.sh +++ b/t/t2400-worktree-add.sh @@ -40,10 +40,24 @@ test_expect_success '"add" using - shorthand' ' test_cmp expect actual ' -test_expect_success '"add" refuses to checkout locked branch' ' - test_must_fail git worktree add zere main && - test_path_is_missing zere && - test_path_is_missing .git/worktrees/zere +test_expect_success '"add" can check out a branch in use by another worktree' ' + test_when_finished "git worktree remove -f zere || :" && + git worktree add zere main && + echo refs/heads/main >expect && + git -C zere symbolic-ref HEAD >actual && + test_cmp expect actual +' + +test_expect_success 'the same branch can be checked out in several worktrees' ' + test_when_finished "git worktree remove -f shared1 || :; git worktree remove -f shared2 || :" && + git branch -f sharedbr main && + git worktree add shared1 sharedbr && + git worktree add shared2 sharedbr && + echo refs/heads/sharedbr >expect && + git -C shared1 symbolic-ref HEAD >actual && + test_cmp expect actual && + git -C shared2 symbolic-ref HEAD >actual && + test_cmp expect actual ' test_expect_success 'checking out paths not complaining about linked checkouts' ' @@ -304,10 +318,13 @@ test_expect_success '"add" checks out existing branch of dwimd name' ' ) ' -test_expect_success '"add <path>" dwim fails with checked out branch' ' +test_expect_success '"add <path>" dwim shares a checked out branch' ' git checkout -b test-branch && - test_must_fail git worktree add test-branch && - test_path_is_missing test-branch + git worktree add test-branch && + echo refs/heads/test-branch >expect && + git -C test-branch symbolic-ref HEAD >actual && + test_cmp expect actual && + git worktree remove test-branch ' test_expect_success '"add --force" with existing dwimd name doesnt die' ' -- gitgitgadget ^ permalink raw reply related [flat|nested] 21+ messages in thread
* Re: [PATCH 0/2] worktree: copy-on-write creation and shared-branch worktrees 2026-06-05 18:49 [PATCH 0/2] worktree: copy-on-write creation and shared-branch worktrees Jason Newton via GitGitGadget 2026-06-05 18:49 ` [PATCH 1/2] worktree: add --reflink for copy-on-write worktree creation Jason Newton via GitGitGadget 2026-06-05 18:49 ` [PATCH 2/2] worktree: allow sharing a checked-out branch across worktrees Jason Newton via GitGitGadget @ 2026-06-05 19:59 ` brian m. carlson 2026-06-08 14:36 ` Junio C Hamano ` (2 subsequent siblings) 5 siblings, 0 replies; 21+ messages in thread From: brian m. carlson @ 2026-06-05 19:59 UTC (permalink / raw) To: Jason Newton via GitGitGadget; +Cc: git, Jason Newton [-- Attachment #1: Type: text/plain, Size: 2933 bytes --] On 2026-06-05 at 18:49:26, Jason Newton via GitGitGadget wrote: > When many worktrees share one repository -- e .g. a fleet of agents each > needing an isolated checkout -- "git worktree add" is costly at scale. > Objects are shared via the common dir, but the working tree is not: each add > rewrites every tracked file, so N worktrees cost N full checkouts of disk > and I/O. And a branch can only be checked out in one worktree. > > Patch 1 adds "git worktree add --reflink": on a copy-on-write filesystem it > populates the new worktree by reflinking the current worktree's files and > index, then "git reset --hard" rewrites only the paths that differ from . A > reflink_file() helper in copy.c uses FICLONE (Linux) and clonefile() > (macOS); elsewhere (other filesystems, Windows) it is probed up front and > falls back to a normal checkout. Defaulting is via the worktree.reflink > config (true/false/auto); --no-reflink overrides. Windows apparently has CoW functionality if you use ReFS. I believe Git LFS has code to do this and you may be interested in checking it out. Also, how does this work if worktree A is dirty (but `git update-index` and `git status` have not been run) when the reflink occurs? Does B have stale files from the working tree? If not, how do we plan on detecting that? (While I'm curious, this should also be explained in your commit message because we want to know that you have thought about this problem and have a good answer for it.) I was curious as to how this would work with containers, which typically use overlayfs, but some searching reveals that overlayfs does indeed support reflinks. Thanks for the opportunity to learn something new today. > Patch 2 lets a branch be checked out in several worktrees, for parallel work > on one checkout. A branch mid-rebase or mid-bisect elsewhere is still > refused. So how does this work if you have two worktrees for the same branch, A and B, and A commits, and then B does? What we don't want to happen is that because B's worktree is not up to date, it effectively reverts the changes that A made when adding objects to the index to commit. (Again, this is a good thing to explain in your commit message, since reviewers will be curious.) My personal approach, if I needed many worktrees of the same commit, would be to create many refs pointing to the same object ID and check those out. `git update-ref` can perform a single ref transaction with many refs, which is especially efficient with reftable. That would avoid the need for multiple checkout support, although I could still see the utility of reflinking if it can be done safely. If that's a solution that you think would be valuable, you could propose it as a FAQ entry or an edit to the manual page, since I'm sure there are other people with your use case. -- brian m. carlson (they/them) Toronto, Ontario, CA [-- Attachment #2: signature.asc --] [-- Type: application/pgp-signature, Size: 325 bytes --] ^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: [PATCH 0/2] worktree: copy-on-write creation and shared-branch worktrees 2026-06-05 18:49 [PATCH 0/2] worktree: copy-on-write creation and shared-branch worktrees Jason Newton via GitGitGadget ` (2 preceding siblings ...) 2026-06-05 19:59 ` [PATCH 0/2] worktree: copy-on-write creation and shared-branch worktrees brian m. carlson @ 2026-06-08 14:36 ` Junio C Hamano 2026-10-20 16:04 ` [PATCH v2 0/2] worktree: copy-on-write creation and ephemeral branches Jason Newton via GitGitGadget 2026-10-21 7:17 ` [PATCH v3 0/4] " Jason Newton via GitGitGadget 5 siblings, 0 replies; 21+ messages in thread From: Junio C Hamano @ 2026-06-08 14:36 UTC (permalink / raw) To: Jason Newton via GitGitGadget; +Cc: git, Jason Newton "Jason Newton via GitGitGadget" <gitgitgadget@gmail.com> writes: > When many worktrees share one repository -- e .g. a fleet of agents each > needing an isolated checkout -- "git worktree add" is costly at scale. > Objects are shared via the common dir, but the working tree is not: each add > rewrites every tracked file, so N worktrees cost N full checkouts of disk > and I/O. Are the "CoW" semantics offered by these underlying mechanisms, which may differ per operating system and possibly filesystem type, all meant as mere storage-space optimization, or do some of them trade potential space saving with some limitation of the features, i.e., what you can do in the CoW copy and original, or increased runtime cost, either at the clone time or the time of first modification? What I am trying to get at is why should this be even an opt-in feature. If "cp treeA treeB" at the shell level would make all the files in treeA under identical names and contents in treeB, and let you edit/update/delete copies in either tree without affecting the other tree, then in practice you would not even be able to _tell_ if CoW is in use, no? It may tilt the scale if there is a downside associated with the use of CoW, like at the first modification of one copy, the system may need to do real copies of other copies, but even such a cost should not be outrageously worse than the cost of copying everything once at the worktree creation time. So I would understand "whenever we say git_copy_file(A, B), we always use CoW facility under the hood if available, regardless of the purpose of the operation to copy one file to another location--- it may include, but does not have to be limited to, populating working file trees in a new worktree", and I think it is a welcome change. But I do not quite get "... only if the user gives --reflink option". Why is it even necessary to offer a choice? Especially since you seem to have auto-probe, we should be able to implement a low-level operation to materialize contents identified by a_hash at a_path in the working tree in two different ways, switching on the availablity of CoW, e.g., if (CoW available && we can find existing path with a_hash) { copy-cow the found path to a_path; } else { write object identified by a_hash to a_path; } > And a branch can only be checked out in one worktree. This is a safety feature that has nothing to do with shared files across worktrees, no? Your two worktrees may think they have a checkout of the same branch (thus the same commit), one worktree makes changes and commits, the other worktree suddenly starts seeing a totally different output from its "git diff HEAD" that mixes what it did relative to where it started (which is what we want) plus the reversion of what was done in the other worktree (which is definitely not what we want). ^ permalink raw reply [flat|nested] 21+ messages in thread
* [PATCH v2 0/2] worktree: copy-on-write creation and ephemeral branches 2026-06-05 18:49 [PATCH 0/2] worktree: copy-on-write creation and shared-branch worktrees Jason Newton via GitGitGadget ` (3 preceding siblings ...) 2026-06-08 14:36 ` Junio C Hamano @ 2026-10-20 16:04 ` Jason Newton via GitGitGadget 2026-10-20 16:04 ` [PATCH v2 1/2] worktree: add --reflink for copy-on-write worktree creation Jason Newton via GitGitGadget ` (2 more replies) 2026-10-21 7:17 ` [PATCH v3 0/4] " Jason Newton via GitGitGadget 5 siblings, 3 replies; 21+ messages in thread From: Jason Newton via GitGitGadget @ 2026-10-20 16:04 UTC (permalink / raw) To: git; +Cc: brian m. carlson, Jason Newton When many worktrees share one repository -- for example a fleet of agents each needing an isolated checkout -- "git worktree add" is costly at scale. Objects are shared via the common directory, but the working tree is not: each add rewrites every tracked file, so N worktrees cost N full checkouts of disk and I/O. A branch created only to back one of those worktrees is also left behind when the worktree is removed. Patch 1 adds "git worktree add --reflink". On a copy-on-write filesystem it populates the new worktree by reflinking the current worktree's files and index, then "git reset --hard" rewrites only the paths that differ from . reflink_file() in copy.c uses FICLONE (Linux), clonefile() (macOS), and ReFS block cloning (FSCTL_DUPLICATE_EXTENTS_TO_FILE, as Git LFS does). Elsewhere it is probed up front and falls back to a normal checkout. worktree.reflink (true/false/auto) sets the default; --no-reflink overrides. Uncommitted edits to tracked files in the source do not leak into the new worktree. The copied index is refreshed before reset, and because every reflinked file is a new inode the refresh re-validates by content. Only untracked files are carried over. The option stays opt-in for that reason: it copies the source's untracked and ignored files, which a plain checkout does not. For many worktrees at one commit, the series does not share a branch. git-worktree(1) shows a single update-ref transaction that creates a ref per worktree (or a detached HEAD). A shared branch still lets a stale index in one worktree silently revert a commit made in another. Patch 2 treats a branch that "git worktree add " created itself as owned by that worktree. "git worktree remove" deletes it. Branches named with -b/-B, or that already existed, are left alone. --keep-branch keeps an owned branch. Changes since v1: * Dropped the patch that allowed one branch to be checked out in several worktrees. Documented the per-ref / detached-HEAD pattern instead (brian m. carlson, Junio). * Explained, in the commit message and the manual, why a dirty source worktree does not leak tracked content (brian m. carlson). * Added the ReFS block-cloning arm (brian m. carlson). * Noted overlayfs as a supported setup when the lower filesystem supports reflinks (brian m. carlson). * Kept --reflink opt-in. A whole-tree reflink is not invisible: it carries untracked files, and cloning every file can be slower than checkout when the files are small (Junio; qub1n on the ReFS work). * Rebased onto 47fdf2208c ("The 7th batch"). reflink_file() now takes a repository argument, matching copy_file(). * Added ephemeral-branch reaping for the auto-named branch that "worktree add " creates. Jason Newton (2): worktree: add --reflink for copy-on-write worktree creation worktree: reap ephemeral branches created by "worktree add" Documentation/config/worktree.adoc | 10 + Documentation/git-worktree.adoc | 71 ++++++- builtin/branch.c | 28 +++ builtin/worktree.c | 301 ++++++++++++++++++++++++++++- copy.c | 190 ++++++++++++++++++ copy.h | 14 ++ t/t2400-worktree-add.sh | 143 ++++++++++++++ t/t3200-branch.sh | 2 +- t/t3400-rebase.sh | 2 +- t/t5516-fetch-push.sh | 2 +- 10 files changed, 754 insertions(+), 9 deletions(-) base-commit: 47fdf2208c5b94c8cc01c106b8f7c85efb91c926 Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-git-2317%2Fnevion%2Fworktree-reflink-cow-v2 Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-git-2317/nevion/worktree-reflink-cow-v2 Pull-Request: https://github.com/git/git/pull/2317 Range-diff vs v1: 1: 7dbd9ca94f ! 1: 83841020f4 worktree: add --reflink for copy-on-write worktree creation @@ Commit message via the common directory, but the working tree is not: N worktrees mean N full checkouts on disk and N times the file I/O. - Add a "--reflink" option that, on copy-on-write filesystems, populates + Add a "--reflink" option that, on a copy-on-write filesystem, populates the new worktree by reflinking the current worktree's files and index instead. The subsequent "git reset --hard" then only rewrites the paths that actually differ between the current worktree and <commit-ish>; everything else (including untracked files such as build - outputs) keeps sharing storage with the source until modified. Because - the cloned index still carries the source files' stat data, it is - refreshed against the reflinked files first so that reset recognizes - the unchanged paths as up to date and leaves them sharing extents - rather than rewriting them. + outputs) keeps sharing storage with the source until modified. - The clones are made by a new reflink_file() helper in copy.c, which - uses the FICLONE ioctl on Linux and clonefile() on macOS and reports - an error otherwise so callers fall back to a normal copy. Support is - probed up front; when unavailable -- including on filesystems without - copy-on-write and on platforms such as Windows that lack a reflink - primitive -- "--reflink" transparently falls back to an ordinary - checkout, so the worst case is no slower than today rather than a - byte-for-byte copy of the source tree. The directory walk skips the - new worktree itself when it lives inside the source one, and preserves - symlinks and modes. + A new reflink_file() helper in copy.c clones every regular file. There + is no size cutoff. + + On Linux it uses the FICLONE ioctl. That is the primitive btrfs, XFS + and bcachefs implement, and the one overlayfs forwards when the lower + filesystem can clone. This path has been tested on btrfs. + + On macOS it uses clonefile(), the APFS primitive. That path is + implemented and has not been run here. + + On Windows it uses ReFS block cloning, + FSCTL_DUPLICATE_EXTENTS_TO_FILE, as Git LFS does. The destination is + sized to the source, extents are cloned in chunks of at most 1GiB, and + the tail is rounded up to a cluster (64K, then 4K) and cloned; Windows + ignores the range past EOF. The tail is not byte-copied, and small + files are not skipped. That path is implemented and has not been run + on ReFS. A volume that rejects the control code, including NTFS, fails + the probe. + + Anywhere else there is no reflink primitive. Support is probed up front + by cloning a one-byte file, and where that fails "--reflink" falls back + to an ordinary checkout instead of byte-copying the source tree. The + directory walk skips the new worktree itself when it lives inside the + source one, and preserves symlinks and modes. + + Uncommitted changes in the source do not leak into the new worktree. + After reflinking the source's files we copy its index and run + "update-index --refresh" before "reset --hard": because every reflinked + file is a fresh inode, the cached stat never matches, so the refresh is + forced to re-validate by content rather than by stat. A tracked path is + therefore only left sharing extents when its content equals the indexed + object; any path whose content differs from <commit-ish> -- which + includes every uncommitted modification in the source -- is rewritten + by the reset. The new worktree is thus always a faithful checkout of + <commit-ish> for tracked paths, and only untracked files are carried + over. The behavior can be made the default with the worktree.reflink configuration ("true", "false" or "auto", the last suppressing the unsupported-filesystem warning), and turned off per-invocation with --no-reflink. A configured default degrades quietly in modes that cannot reflink (--orphan, --no-checkout) instead of erroring, so - enabling it never breaks those commands. The checkout step continues - to honor checkout.workers, so parallel checkout composes with - --reflink for the paths that do need rewriting. + enabling it never breaks those commands. The checkout step continues to + honor checkout.workers, so parallel checkout composes with --reflink for + the paths that do need rewriting. + + To check out many worktrees at the same commit safely, give each its + own ref (or a detached HEAD) rather than sharing one branch; a + git-worktree(1) example shows the one-shot "update-ref" transaction + that pairs with --reflink. Signed-off-by: Jason Newton <nevion@gmail.com> @@ Documentation/git-worktree.adoc: To remove a locked worktree, specify `--force` + worktrees from the same base. `--no-reflink` forces an ordinary + checkout, overriding the `worktree.reflink` configuration. ++ -+Reflinks require a copy-on-write filesystem (for example btrfs, XFS, -+bcachefs or ZFS on Linux, or APFS on macOS). On filesystems or platforms -+that do not support reflinks, `--reflink` transparently falls back to an -+ordinary checkout. Because the source worktree's untracked and ignored -+files are cloned as well, only use `--reflink` when that is acceptable. ++Reflinks require a copy-on-write filesystem: for example btrfs, XFS, ++bcachefs or ZFS on Linux (and overlayfs on top of one of these, as used by ++containers), APFS on macOS, or ReFS on Windows. On a filesystem that does ++not support reflinks, `--reflink` transparently falls back to an ordinary ++checkout. Because the source worktree's untracked and ignored files are ++cloned as well, only use `--reflink` when that is acceptable. ++ +This option cannot be combined with `--no-checkout` or `--orphan`. It can +be enabled by default with the `worktree.reflink` configuration; see @@ Documentation/git-worktree.adoc: To remove a locked worktree, specify `--force` `--guess-remote`:: `--no-guess-remote`:: With `worktree add <path>`, without _<commit-ish>_, instead +@@ Documentation/git-worktree.adoc: $ popd + $ git worktree remove ../temp + ------------ + ++To run several independent worktrees from the same starting commit -- for ++example a pool of automated agents -- give each one its own branch rather than ++sharing a single branch between them. A shared branch is unsafe here: if one ++worktree commits and another then commits from its now-stale index, the second ++commit silently drops the first one's changes. Create the branches in a single ++ref transaction and check each out in its own (optionally reflinked) worktree: ++ ++------------ ++$ oid=$(git rev-parse HEAD) ++$ printf 'create refs/heads/agent-%s %s\n' 1 "$oid" 2 "$oid" 3 "$oid" | ++ git update-ref --stdin ++$ for n in 1 2 3; do git worktree add --reflink ../agent-$n agent-$n; done ++------------ ++ ++The single linkgit:git-update-ref[1] transaction is cheap and is especially ++efficient with the `reftable` backend. If a worktree does not need to advance a ++branch at all, use a detached `HEAD` (`git worktree add --detach`) instead. ++ + CONFIGURATION + ------------- + ## builtin/worktree.c ## @@ @@ builtin/worktree.c: worktree_copy_cleanup: +/* + * Probe whether the filesystem backing "dir" supports reflinks. We do this -+ * once up front so that, on filesystems without copy-on-write support (or on -+ * platforms such as Windows that lack a reflink primitive entirely), we can ++ * once up front so that, on filesystems without copy-on-write support (and on ++ * platforms that lack a reflink primitive entirely), we can + * fall back to a normal checkout instead of byte-copying the whole source + * working tree -- which would include untracked files and be slower than the + * checkout we are trying to avoid. @@ builtin/worktree.c: worktree_copy_cleanup: + if (fd >= 0) { + write_in_full(fd, "x", 1); + close(fd); -+ if (!reflink_file(dst.buf, src.buf, 0600)) ++ if (!reflink_file(the_repository, dst.buf, src.buf, 0600)) + ok = 1; + unlink(dst.buf); + } @@ builtin/worktree.c: worktree_copy_cleanup: + */ +static int reflink_or_copy(const char *dst, const char *src, int mode) +{ -+ if (!reflink_file(dst, src, mode)) ++ if (!reflink_file(the_repository, dst, src, mode)) + return 0; -+ return copy_file(dst, src, mode); ++ return copy_file(the_repository, dst, src, mode); +} + +/* @@ copy.c +#elif defined(__APPLE__) +#include <sys/attr.h> +#include <sys/clonefile.h> ++#elif defined(GIT_WINDOWS_NATIVE) ++#include <winioctl.h> ++/* ++ * Block cloning on ReFS uses FSCTL_DUPLICATE_EXTENTS_TO_FILE. Older Windows ++ * SDK headers may lack the control code and its payload struct, so define ++ * them ourselves when missing (the values are part of the stable Win32 API). ++ */ ++#ifndef FSCTL_DUPLICATE_EXTENTS_TO_FILE ++#define FSCTL_DUPLICATE_EXTENTS_TO_FILE \ ++ CTL_CODE(FILE_DEVICE_FILE_SYSTEM, 209, METHOD_BUFFERED, FILE_WRITE_DATA) ++typedef struct _DUPLICATE_EXTENTS_DATA { ++ HANDLE FileHandle; ++ LARGE_INTEGER SourceFileOffset; ++ LARGE_INTEGER TargetFileOffset; ++ LARGE_INTEGER ByteCount; ++} DUPLICATE_EXTENTS_DATA, *PDUPLICATE_EXTENTS_DATA; ++#endif +#endif + int copy_fd(int ifd, int ofd) { while (1) { -@@ copy.c: int copy_file_with_time(const char *dst, const char *src, int mode) +@@ copy.c: int copy_file_with_time(struct repository *repo, return copy_times(dst, src); return status; } + -+int reflink_file(const char *dst, const char *src, int mode) ++#ifdef GIT_WINDOWS_NATIVE ++/* ++ * Clone [offset, offset+count) of "src" onto "dst" via ReFS block cloning. ++ * "count" is rounded up to a cluster boundary by the caller; a range that ++ * runs past the source's end of file is harmless and ignored by Windows. ++ */ ++static int clone_extent_windows(HANDLE dst, HANDLE src, ++ uint64_t offset, uint64_t count) ++{ ++ DUPLICATE_EXTENTS_DATA dup; ++ DWORD bytes; ++ ++ dup.FileHandle = src; ++ dup.SourceFileOffset.QuadPart = offset; ++ dup.TargetFileOffset.QuadPart = offset; ++ dup.ByteCount.QuadPart = count; ++ if (!DeviceIoControl(dst, FSCTL_DUPLICATE_EXTENTS_TO_FILE, ++ &dup, sizeof(dup), NULL, 0, &bytes, NULL)) ++ return -1; ++ return 0; ++} ++#endif ++ ++int reflink_file(struct repository *repo, ++ const char *dst, const char *src, int mode) +{ +#if defined(__APPLE__) + /* @@ copy.c: int copy_file_with_time(const char *dst, const char *src, int mode) + (void)mode; + if (clonefile(src, dst, 0) < 0) + return -1; -+ if (adjust_shared_perm(the_repository, dst)) ++ if (adjust_shared_perm(repo, dst)) + return -1; + return 0; +#elif defined(__linux__) @@ copy.c: int copy_file_with_time(const char *dst, const char *src, int mode) + } + if (close(fdo) != 0) + return -1; -+ if (adjust_shared_perm(the_repository, dst)) ++ if (adjust_shared_perm(repo, dst)) ++ return -1; ++ return 0; ++#elif defined(GIT_WINDOWS_NATIVE) ++ /* ++ * ReFS supports copy-on-write block cloning via ++ * FSCTL_DUPLICATE_EXTENTS_TO_FILE, the same mechanism Git LFS uses. ++ * Cloned ranges must start and end on a cluster boundary (ReFS uses ++ * 64K or 4K clusters) and span less than 4GiB, so clone in <=1GiB ++ * chunks and round the final chunk up to a cluster; the destination is ++ * sized to the source's exact length and an over-long final range past ++ * the source's EOF is ignored. ++ */ ++ const uint64_t GiB = (uint64_t)1 << 30; ++ const uint64_t clusters[] = { 64 * 1024, 4 * 1024 }; ++ int fdi, fdo, ret = -1; ++ HANDLE hi, ho; ++ LARGE_INTEGER size; ++ uint64_t off; ++ size_t i; ++ ++ mode = (mode & 0111) ? 0777 : 0666; ++ if ((fdi = open(src, O_RDONLY | O_BINARY)) < 0) ++ return -1; ++ if ((fdo = open(dst, O_RDWR | O_CREAT | O_EXCL | O_BINARY, mode)) < 0) { ++ int saved = errno; ++ close(fdi); ++ errno = saved; ++ return -1; ++ } ++ hi = (HANDLE)_get_osfhandle(fdi); ++ ho = (HANDLE)_get_osfhandle(fdo); ++ if (hi == INVALID_HANDLE_VALUE || ho == INVALID_HANDLE_VALUE || ++ !GetFileSizeEx(hi, &size)) { ++ errno = EINVAL; ++ goto cleanup; ++ } ++ ++ /* the destination must be the source's exact size before cloning */ ++ if (!SetFilePointerEx(ho, size, NULL, FILE_BEGIN) || !SetEndOfFile(ho)) { ++ errno = EIO; ++ goto cleanup; ++ } ++ ++ ret = 0; ++ /* clone whole 1GiB regions (already cluster aligned) */ ++ for (off = 0; (uint64_t)size.QuadPart > off && ++ (uint64_t)size.QuadPart - off > GiB; off += GiB) { ++ if (clone_extent_windows(ho, hi, off, GiB)) { ++ errno = ENOSYS; ++ ret = -1; ++ goto cleanup; ++ } ++ } ++ /* clone the tail, rounding its length up to a cluster (64K then 4K) */ ++ if (!ret && (uint64_t)size.QuadPart > off) { ++ uint64_t rem = (uint64_t)size.QuadPart - off; ++ ++ ret = -1; ++ for (i = 0; i < ARRAY_SIZE(clusters); i++) { ++ uint64_t count = (rem + clusters[i] - 1) & ++ ~(clusters[i] - 1); ++ if (!clone_extent_windows(ho, hi, off, count)) { ++ ret = 0; ++ break; ++ } ++ } ++ if (ret) ++ errno = ENOSYS; /* not a CoW volume */ ++ } ++ ++cleanup: ++ close(fdi); ++ if (close(fdo) && !ret) { ++ errno = EIO; ++ ret = -1; ++ } ++ if (ret) { ++ int saved = errno; ++ unlink(dst); ++ errno = saved; ++ return -1; ++ } ++ if (adjust_shared_perm(repo, dst)) + return -1; + return 0; +#else -+ /* No reflink support on this platform (e.g. Windows). */ ++ /* No reflink primitive on this platform. */ ++ (void)repo; + (void)dst; + (void)src; + (void)mode; @@ copy.c: int copy_file_with_time(const char *dst, const char *src, int mode) +} ## copy.h ## -@@ copy.h: int copy_fd(int ifd, int ofd); - int copy_file(const char *dst, const char *src, int mode); - int copy_file_with_time(const char *dst, const char *src, int mode); +@@ copy.h: int copy_file(struct repository *repo, + int copy_file_with_time(struct repository *repo, + const char *dst, const char *src, int mode); +/* + * Create "dst" as a copy-on-write (reflink) clone of the regular file @@ copy.h: int copy_fd(int ifd, int ofd); + * ENOSYS (or the underlying error). Callers are expected to fall back to + * copy_file() in that case. Returns 0 on success. + */ -+int reflink_file(const char *dst, const char *src, int mode); ++int reflink_file(struct repository *repo, ++ const char *dst, const char *src, int mode); + #endif /* COPY_H */ @@ t/t2400-worktree-add.sh: test_expect_success 'relative worktree sets extension c + echo artifact >expect && + test_cmp expect wt/untracked.bin && + git -C wt status --porcelain >status && -+ grep "?? untracked.bin" status ++ test_grep "?? untracked.bin" status + ) +' + 2: 97f0fdad9b < -: ---------- worktree: allow sharing a checked-out branch across worktrees -: ---------- > 2: 6e2acdc5e2 worktree: reap ephemeral branches created by "worktree add" -- gitgitgadget ^ permalink raw reply [flat|nested] 21+ messages in thread
* [PATCH v2 1/2] worktree: add --reflink for copy-on-write worktree creation 2026-10-20 16:04 ` [PATCH v2 0/2] worktree: copy-on-write creation and ephemeral branches Jason Newton via GitGitGadget @ 2026-10-20 16:04 ` Jason Newton via GitGitGadget 2026-10-20 21:20 ` brian m. carlson 2026-10-20 16:04 ` [PATCH v2 2/2] worktree: reap ephemeral branches created by "worktree add" Jason Newton via GitGitGadget 2026-10-21 7:08 ` [PATCH v2 0/2] worktree: copy-on-write creation and ephemeral branches Eric Sunshine 2 siblings, 1 reply; 21+ messages in thread From: Jason Newton via GitGitGadget @ 2026-10-20 16:04 UTC (permalink / raw) To: git; +Cc: brian m. carlson, Jason Newton, Jason Newton From: Jason Newton <nevion@gmail.com> Creating many worktrees from the same base -- for example to run a fleet of automated agents in parallel -- is expensive today: every "git worktree add" materializes the entire working tree by writing each tracked file out from the object store. The objects are shared via the common directory, but the working tree is not: N worktrees mean N full checkouts on disk and N times the file I/O. Add a "--reflink" option that, on a copy-on-write filesystem, populates the new worktree by reflinking the current worktree's files and index instead. The subsequent "git reset --hard" then only rewrites the paths that actually differ between the current worktree and <commit-ish>; everything else (including untracked files such as build outputs) keeps sharing storage with the source until modified. A new reflink_file() helper in copy.c clones every regular file. There is no size cutoff. On Linux it uses the FICLONE ioctl. That is the primitive btrfs, XFS and bcachefs implement, and the one overlayfs forwards when the lower filesystem can clone. This path has been tested on btrfs. On macOS it uses clonefile(), the APFS primitive. That path is implemented and has not been run here. On Windows it uses ReFS block cloning, FSCTL_DUPLICATE_EXTENTS_TO_FILE, as Git LFS does. The destination is sized to the source, extents are cloned in chunks of at most 1GiB, and the tail is rounded up to a cluster (64K, then 4K) and cloned; Windows ignores the range past EOF. The tail is not byte-copied, and small files are not skipped. That path is implemented and has not been run on ReFS. A volume that rejects the control code, including NTFS, fails the probe. Anywhere else there is no reflink primitive. Support is probed up front by cloning a one-byte file, and where that fails "--reflink" falls back to an ordinary checkout instead of byte-copying the source tree. The directory walk skips the new worktree itself when it lives inside the source one, and preserves symlinks and modes. Uncommitted changes in the source do not leak into the new worktree. After reflinking the source's files we copy its index and run "update-index --refresh" before "reset --hard": because every reflinked file is a fresh inode, the cached stat never matches, so the refresh is forced to re-validate by content rather than by stat. A tracked path is therefore only left sharing extents when its content equals the indexed object; any path whose content differs from <commit-ish> -- which includes every uncommitted modification in the source -- is rewritten by the reset. The new worktree is thus always a faithful checkout of <commit-ish> for tracked paths, and only untracked files are carried over. The behavior can be made the default with the worktree.reflink configuration ("true", "false" or "auto", the last suppressing the unsupported-filesystem warning), and turned off per-invocation with --no-reflink. A configured default degrades quietly in modes that cannot reflink (--orphan, --no-checkout) instead of erroring, so enabling it never breaks those commands. The checkout step continues to honor checkout.workers, so parallel checkout composes with --reflink for the paths that do need rewriting. To check out many worktrees at the same commit safely, give each its own ref (or a detached HEAD) rather than sharing one branch; a git-worktree(1) example shows the one-shot "update-ref" transaction that pairs with --reflink. Signed-off-by: Jason Newton <nevion@gmail.com> --- Documentation/config/worktree.adoc | 10 ++ Documentation/git-worktree.adoc | 49 ++++++- builtin/worktree.c | 227 ++++++++++++++++++++++++++++- copy.c | 190 ++++++++++++++++++++++++ copy.h | 14 ++ t/t2400-worktree-add.sh | 88 +++++++++++ 6 files changed, 576 insertions(+), 2 deletions(-) diff --git a/Documentation/config/worktree.adoc b/Documentation/config/worktree.adoc index 0930183b91..aa849c08d2 100644 --- a/Documentation/config/worktree.adoc +++ b/Documentation/config/worktree.adoc @@ -18,3 +18,13 @@ Note that setting `worktree.useRelativePaths` to "`true`" implies enabling the `extensions.relativeWorktrees` config (see linkgit:git-config[1]), thus making it incompatible with older versions of Git. + +`worktree.reflink`:: + Controls whether `git worktree add` populates new worktrees with + copy-on-write (reflink) clones, as if `--reflink` had been given + (see linkgit:git-worktree[1]). May be set to "`true`", "`false`" + (the default), or "`auto`". With "`true`", a filesystem that does + not support reflinks produces a warning before falling back to an + ordinary checkout; with "`auto`", the fallback is silent. An + explicit `--reflink` or `--no-reflink` on the command line + overrides this setting. diff --git a/Documentation/git-worktree.adoc b/Documentation/git-worktree.adoc index e6e77252ab..4264ee753a 100644 --- a/Documentation/git-worktree.adoc +++ b/Documentation/git-worktree.adoc @@ -10,7 +10,7 @@ SYNOPSIS -------- [synopsis] git worktree add [-f] [--detach] [--checkout] [--lock [--reason <string>]] - [--orphan] [(-b | -B) <new-branch>] <path> [<commit-ish>] + [--orphan] [--reflink] [(-b | -B) <new-branch>] <path> [<commit-ish>] git worktree list [-v | --porcelain [-z]] git worktree lock [--reason <string>] <worktree> git worktree move <worktree> <new-path> @@ -213,6 +213,35 @@ To remove a locked worktree, specify `--force` twice. such as configuring sparse-checkout. See "Sparse checkout" in linkgit:git-read-tree[1]. +`--reflink`:: +`--no-reflink`:: + Populate the new worktree by creating copy-on-write (reflink) + clones of the current worktree's files and index instead of + writing every file out from the object store. The checkout that + follows then only has to rewrite the paths that actually differ + between the current worktree and _<commit-ish>_; everything else + (including untracked files such as build outputs) keeps sharing + storage with the source worktree until modified. This makes + `add` much faster and far cheaper on disk when creating many + worktrees from the same base. `--no-reflink` forces an ordinary + checkout, overriding the `worktree.reflink` configuration. ++ +Reflinks require a copy-on-write filesystem: for example btrfs, XFS, +bcachefs or ZFS on Linux (and overlayfs on top of one of these, as used by +containers), APFS on macOS, or ReFS on Windows. On a filesystem that does +not support reflinks, `--reflink` transparently falls back to an ordinary +checkout. Because the source worktree's untracked and ignored files are +cloned as well, only use `--reflink` when that is acceptable. ++ +This option cannot be combined with `--no-checkout` or `--orphan`. It can +be enabled by default with the `worktree.reflink` configuration; see +linkgit:git-config[1]. ++ +The checkout that populates a new worktree also honors the +`checkout.workers` configuration (see linkgit:git-config[1]), so setting it +parallelizes the file writes and can further speed up `add`, with or +without `--reflink`. + `--guess-remote`:: `--no-guess-remote`:: With `worktree add <path>`, without _<commit-ish>_, instead @@ -531,6 +560,24 @@ $ popd $ git worktree remove ../temp ------------ +To run several independent worktrees from the same starting commit -- for +example a pool of automated agents -- give each one its own branch rather than +sharing a single branch between them. A shared branch is unsafe here: if one +worktree commits and another then commits from its now-stale index, the second +commit silently drops the first one's changes. Create the branches in a single +ref transaction and check each out in its own (optionally reflinked) worktree: + +------------ +$ oid=$(git rev-parse HEAD) +$ printf 'create refs/heads/agent-%s %s\n' 1 "$oid" 2 "$oid" 3 "$oid" | + git update-ref --stdin +$ for n in 1 2 3; do git worktree add --reflink ../agent-$n agent-$n; done +------------ + +The single linkgit:git-update-ref[1] transaction is cheap and is especially +efficient with the `reftable` backend. If a worktree does not need to advance a +branch at all, use a detached `HEAD` (`git worktree add --detach`) instead. + CONFIGURATION ------------- diff --git a/builtin/worktree.c b/builtin/worktree.c index 77ecd0f71f..9d4b51305e 100644 --- a/builtin/worktree.c +++ b/builtin/worktree.c @@ -30,7 +30,7 @@ #define BUILTIN_WORKTREE_ADD_USAGE \ N_("git worktree add [-f] [--detach] [--checkout] [--lock [--reason <string>]]\n" \ - " [--orphan] [(-b | -B) <new-branch>] <path> [<commit-ish>]") + " [--orphan] [--reflink] [(-b | -B) <new-branch>] <path> [<commit-ish>]") #define BUILTIN_WORKTREE_LIST_USAGE \ N_("git worktree list [-v | --porcelain [-z]]") @@ -47,6 +47,11 @@ #define BUILTIN_WORKTREE_UNLOCK_USAGE \ N_("git worktree unlock <worktree>") +/* values for add_opts.reflink and the worktree.reflink config */ +#define REFLINK_OFF 0 +#define REFLINK_ON 1 /* warn and fall back if unsupported */ +#define REFLINK_AUTO 2 /* silently fall back if unsupported */ + #define WORKTREE_ADD_DWIM_ORPHAN_INFER_TEXT \ _("No possible source branch, inferring '--orphan'") @@ -123,6 +128,7 @@ struct add_opts { int checkout; int orphan; int relative_paths; + int reflink; const char *keep_locked; }; @@ -130,6 +136,7 @@ static int show_only; static int verbose; static int guess_remote; static int use_relative_paths; +static int reflink_config = REFLINK_OFF; static timestamp_t expire; static int git_worktree_config(const char *var, const char *value, @@ -141,6 +148,13 @@ static int git_worktree_config(const char *var, const char *value, } else if (!strcmp(var, "worktree.userelativepaths")) { use_relative_paths = git_config_bool(var, value); return 0; + } else if (!strcmp(var, "worktree.reflink")) { + if (value && !strcmp(value, "auto")) + reflink_config = REFLINK_AUTO; + else + reflink_config = git_config_bool(var, value) ? + REFLINK_ON : REFLINK_OFF; + return 0; } return git_default_config(var, value, ctx, cb); @@ -394,6 +408,182 @@ worktree_copy_cleanup: free(to_file); } +/* + * Probe whether the filesystem backing "dir" supports reflinks. We do this + * once up front so that, on filesystems without copy-on-write support (and on + * platforms that lack a reflink primitive entirely), we can + * fall back to a normal checkout instead of byte-copying the whole source + * working tree -- which would include untracked files and be slower than the + * checkout we are trying to avoid. + */ +static int reflink_supported(const char *dir) +{ + struct strbuf src = STRBUF_INIT, dst = STRBUF_INIT; + int fd, ok = 0; + + strbuf_addf(&src, "%s/.git-reflink-probe-src", dir); + strbuf_addf(&dst, "%s/.git-reflink-probe-dst", dir); + + fd = open(src.buf, O_WRONLY | O_CREAT | O_TRUNC, 0600); + if (fd >= 0) { + write_in_full(fd, "x", 1); + close(fd); + if (!reflink_file(the_repository, dst.buf, src.buf, 0600)) + ok = 1; + unlink(dst.buf); + } + unlink(src.buf); + + strbuf_release(&src); + strbuf_release(&dst); + return ok; +} + +/* + * Reflink a single regular file, falling back to a regular copy when the + * clone fails for this particular file (for example across mount points). + */ +static int reflink_or_copy(const char *dst, const char *src, int mode) +{ + if (!reflink_file(the_repository, dst, src, mode)) + return 0; + return copy_file(the_repository, dst, src, mode); +} + +/* + * Recursively copy the working-tree directory "src" into "dst" using reflinks + * for regular files. Directory entries that resolve to the destination + * worktree itself (identified by skip_dev/skip_ino, which matters when the new + * worktree lives inside the source one) and the top-level ".git" gitfile are + * skipped. + */ +static int reflink_tree(const char *src, const char *dst, + dev_t skip_dev, ino_t skip_ino, int top) +{ + struct strbuf s = STRBUF_INIT, d = STRBUF_INIT; + DIR *dir; + struct dirent *de; + int ret = 0; + + dir = opendir(src); + if (!dir) + return error_errno(_("could not open directory '%s'"), src); + + while (!ret && (de = readdir(dir))) { + struct stat st; + + if (is_dot_or_dotdot(de->d_name)) + continue; + if (top && !strcmp(de->d_name, ".git")) + continue; + + strbuf_reset(&s); + strbuf_addf(&s, "%s/%s", src, de->d_name); + strbuf_reset(&d); + strbuf_addf(&d, "%s/%s", dst, de->d_name); + + if (lstat(s.buf, &st)) { + ret = error_errno(_("could not stat '%s'"), s.buf); + break; + } + + if (S_ISDIR(st.st_mode)) { + /* never recurse into the new worktree itself */ + if (st.st_dev == skip_dev && st.st_ino == skip_ino) + continue; + if (mkdir(d.buf, st.st_mode & 07777) && errno != EEXIST) { + ret = error_errno(_("could not create directory '%s'"), d.buf); + break; + } + ret = reflink_tree(s.buf, d.buf, skip_dev, skip_ino, 0); + } else if (S_ISLNK(st.st_mode)) { + struct strbuf link = STRBUF_INIT; + + if (strbuf_readlink(&link, s.buf, st.st_size)) + ret = error_errno(_("could not read symlink '%s'"), s.buf); + else if (symlink(link.buf, d.buf)) + ret = error_errno(_("could not create symlink '%s'"), d.buf); + strbuf_release(&link); + } else if (S_ISREG(st.st_mode)) { + if (reflink_or_copy(d.buf, s.buf, st.st_mode)) + ret = error_errno(_("could not copy '%s' to '%s'"), + s.buf, d.buf); + } + /* silently skip fifos, sockets and device nodes */ + } + + closedir(dir); + strbuf_release(&s); + strbuf_release(&d); + return ret; +} + +/* + * Populate the new worktree at "path" by reflinking the current worktree's + * files and index. The subsequent "git reset --hard" then only has to rewrite + * the paths that actually differ between the source and <commit-ish>, leaving + * everything else sharing storage with the source. Returns 1 when reflinks are + * unavailable so the caller can fall back to a plain checkout. + */ +static int reflink_worktree(const char *path, const char *wt_git_dir, + const struct add_opts *opts, struct strvec *child_env) +{ + const char *src_wt = the_repository->worktree; + char *src_index = NULL, *dst_index = NULL; + struct stat dst_st; + int ret = 0; + + if (!src_wt) + return error(_("--reflink needs a source working tree, but this " + "repository does not have one")); + + if (!reflink_supported(path)) { + /* In auto mode the fallback is expected, so stay quiet. */ + if (!opts->quiet && opts->reflink != REFLINK_AUTO) + warning(_("the filesystem at '%s' does not support reflinks; " + "falling back to a regular checkout"), path); + return 1; + } + + if (stat(path, &dst_st)) + return error_errno(_("could not stat '%s'"), path); + + if ((ret = reflink_tree(src_wt, path, dst_st.st_dev, dst_st.st_ino, 1))) + return ret; + + /* + * Clone the source index so the following reset sees the source's + * state and only materializes the differences to <commit-ish>. + */ + src_index = repo_git_path(the_repository, "index"); + dst_index = xstrfmt("%s/index", wt_git_dir); + if (!access(src_index, F_OK) && + reflink_or_copy(dst_index, src_index, 0666)) { + ret = error_errno(_("could not copy index to '%s'"), dst_index); + goto out; + } + + /* + * The cloned index still carries the source files' stat information. + * Refresh it against the freshly reflinked files so that "git reset" + * recognizes unchanged paths as up to date and leaves them sharing + * storage instead of rewriting (and thus un-sharing) them. + */ + if (!access(dst_index, F_OK)) { + struct child_process cp = CHILD_PROCESS_INIT; + cp.git_cmd = 1; + strvec_pushl(&cp.args, "update-index", "-q", "--refresh", NULL); + strvec_pushv(&cp.env, child_env->v); + /* a dirty working tree is not an error here */ + run_command(&cp); + } + +out: + free(src_index); + free(dst_index); + return ret; +} + static int checkout_worktree(const struct add_opts *opts, struct strvec *child_env) { @@ -588,6 +778,20 @@ static int add_worktree(const char *path, const char *refname, (ret = make_worktree_orphan(refname, opts, &child_env))) goto done; + /* + * When --reflink is requested and the filesystem supports it, copy the + * current worktree (and its index) into the new one using copy-on-write + * clones. checkout_worktree() then only rewrites the paths that differ + * from <commit-ish>. reflink_worktree() returns 1 when reflinks are not + * available, in which case we just do an ordinary checkout below. + */ + if (opts->checkout && opts->reflink) { + ret = reflink_worktree(path, sb_repo.buf, opts, &child_env); + if (ret < 0) + goto done; + ret = 0; + } + if (opts->checkout && (ret = checkout_worktree(opts, &child_env))) goto done; @@ -832,6 +1036,7 @@ static int add(int ac, const char **av, const char *prefix, const char *lock_reason = NULL; int keep_locked = 0; int used_new_branch_options; + int reflink_cli = -1; struct option options[] = { OPT__FORCE(&opts.force, N_("checkout <branch> even if already checked out in other worktree"), @@ -854,6 +1059,8 @@ static int add(int ac, const char **av, const char *prefix, N_("try to match the new branch name with a remote-tracking branch")), OPT_BOOL(0, "relative-paths", &opts.relative_paths, N_("use relative paths for worktrees")), + OPT_BOOL(0, "reflink", &reflink_cli, + N_("populate the worktree using copy-on-write clones when supported")), OPT_END() }; int ret; @@ -873,6 +1080,24 @@ static int add(int ac, const char **av, const char *prefix, if (opts.orphan && !opts.checkout) die(_("options '%s' and '%s' cannot be used together"), "--orphan", "--no-checkout"); + + /* + * Resolve whether to reflink: an explicit --reflink/--no-reflink on + * the command line wins, otherwise fall back to the worktree.reflink + * configuration (which may select the "auto" mode). + */ + opts.reflink = (reflink_cli != -1) ? reflink_cli : reflink_config; + if (opts.reflink && (opts.orphan || !opts.checkout)) { + /* + * Reflinking is incompatible with these; only complain when it + * was explicitly requested, otherwise quietly do a plain + * checkout so a configured default does not break these modes. + */ + if (reflink_cli == REFLINK_ON) + die(_("options '%s' and '%s' cannot be used together"), + "--reflink", opts.orphan ? "--orphan" : "--no-checkout"); + opts.reflink = REFLINK_OFF; + } if (opts.orphan && ac == 2) die(_("option '%s' and commit-ish cannot be used together"), "--orphan"); diff --git a/copy.c b/copy.c index 6074132050..e6dfe33f6a 100644 --- a/copy.c +++ b/copy.c @@ -5,6 +5,38 @@ #include "strbuf.h" #include "abspath.h" +#if defined(__linux__) +#include <sys/ioctl.h> +/* + * FICLONE lives in <linux/fs.h>, but including that header tends to clash + * with the libc headers git already pulls in, so define it ourselves if it + * is missing. The value is part of the stable kernel uapi. + */ +#ifndef FICLONE +#define FICLONE _IOW(0x94, 9, int) +#endif +#elif defined(__APPLE__) +#include <sys/attr.h> +#include <sys/clonefile.h> +#elif defined(GIT_WINDOWS_NATIVE) +#include <winioctl.h> +/* + * Block cloning on ReFS uses FSCTL_DUPLICATE_EXTENTS_TO_FILE. Older Windows + * SDK headers may lack the control code and its payload struct, so define + * them ourselves when missing (the values are part of the stable Win32 API). + */ +#ifndef FSCTL_DUPLICATE_EXTENTS_TO_FILE +#define FSCTL_DUPLICATE_EXTENTS_TO_FILE \ + CTL_CODE(FILE_DEVICE_FILE_SYSTEM, 209, METHOD_BUFFERED, FILE_WRITE_DATA) +typedef struct _DUPLICATE_EXTENTS_DATA { + HANDLE FileHandle; + LARGE_INTEGER SourceFileOffset; + LARGE_INTEGER TargetFileOffset; + LARGE_INTEGER ByteCount; +} DUPLICATE_EXTENTS_DATA, *PDUPLICATE_EXTENTS_DATA; +#endif +#endif + int copy_fd(int ifd, int ofd) { while (1) { @@ -72,3 +104,161 @@ int copy_file_with_time(struct repository *repo, return copy_times(dst, src); return status; } + +#ifdef GIT_WINDOWS_NATIVE +/* + * Clone [offset, offset+count) of "src" onto "dst" via ReFS block cloning. + * "count" is rounded up to a cluster boundary by the caller; a range that + * runs past the source's end of file is harmless and ignored by Windows. + */ +static int clone_extent_windows(HANDLE dst, HANDLE src, + uint64_t offset, uint64_t count) +{ + DUPLICATE_EXTENTS_DATA dup; + DWORD bytes; + + dup.FileHandle = src; + dup.SourceFileOffset.QuadPart = offset; + dup.TargetFileOffset.QuadPart = offset; + dup.ByteCount.QuadPart = count; + if (!DeviceIoControl(dst, FSCTL_DUPLICATE_EXTENTS_TO_FILE, + &dup, sizeof(dup), NULL, 0, &bytes, NULL)) + return -1; + return 0; +} +#endif + +int reflink_file(struct repository *repo, + const char *dst, const char *src, int mode) +{ +#if defined(__APPLE__) + /* + * clonefile() refuses to operate when the destination exists and + * copies the source's permissions for us, so "mode" is unused here. + */ + (void)mode; + if (clonefile(src, dst, 0) < 0) + return -1; + if (adjust_shared_perm(repo, dst)) + return -1; + return 0; +#elif defined(__linux__) + int fdi, fdo, status; + + mode = (mode & 0111) ? 0777 : 0666; + if ((fdi = open(src, O_RDONLY)) < 0) + return -1; + if ((fdo = open(dst, O_WRONLY | O_CREAT | O_EXCL, mode)) < 0) { + int saved = errno; + close(fdi); + errno = saved; + return -1; + } + status = ioctl(fdo, FICLONE, fdi); + close(fdi); + if (status < 0) { + int saved = errno; + close(fdo); + /* we created an empty file above; do not leave it behind */ + unlink(dst); + errno = saved; + return -1; + } + if (close(fdo) != 0) + return -1; + if (adjust_shared_perm(repo, dst)) + return -1; + return 0; +#elif defined(GIT_WINDOWS_NATIVE) + /* + * ReFS supports copy-on-write block cloning via + * FSCTL_DUPLICATE_EXTENTS_TO_FILE, the same mechanism Git LFS uses. + * Cloned ranges must start and end on a cluster boundary (ReFS uses + * 64K or 4K clusters) and span less than 4GiB, so clone in <=1GiB + * chunks and round the final chunk up to a cluster; the destination is + * sized to the source's exact length and an over-long final range past + * the source's EOF is ignored. + */ + const uint64_t GiB = (uint64_t)1 << 30; + const uint64_t clusters[] = { 64 * 1024, 4 * 1024 }; + int fdi, fdo, ret = -1; + HANDLE hi, ho; + LARGE_INTEGER size; + uint64_t off; + size_t i; + + mode = (mode & 0111) ? 0777 : 0666; + if ((fdi = open(src, O_RDONLY | O_BINARY)) < 0) + return -1; + if ((fdo = open(dst, O_RDWR | O_CREAT | O_EXCL | O_BINARY, mode)) < 0) { + int saved = errno; + close(fdi); + errno = saved; + return -1; + } + hi = (HANDLE)_get_osfhandle(fdi); + ho = (HANDLE)_get_osfhandle(fdo); + if (hi == INVALID_HANDLE_VALUE || ho == INVALID_HANDLE_VALUE || + !GetFileSizeEx(hi, &size)) { + errno = EINVAL; + goto cleanup; + } + + /* the destination must be the source's exact size before cloning */ + if (!SetFilePointerEx(ho, size, NULL, FILE_BEGIN) || !SetEndOfFile(ho)) { + errno = EIO; + goto cleanup; + } + + ret = 0; + /* clone whole 1GiB regions (already cluster aligned) */ + for (off = 0; (uint64_t)size.QuadPart > off && + (uint64_t)size.QuadPart - off > GiB; off += GiB) { + if (clone_extent_windows(ho, hi, off, GiB)) { + errno = ENOSYS; + ret = -1; + goto cleanup; + } + } + /* clone the tail, rounding its length up to a cluster (64K then 4K) */ + if (!ret && (uint64_t)size.QuadPart > off) { + uint64_t rem = (uint64_t)size.QuadPart - off; + + ret = -1; + for (i = 0; i < ARRAY_SIZE(clusters); i++) { + uint64_t count = (rem + clusters[i] - 1) & + ~(clusters[i] - 1); + if (!clone_extent_windows(ho, hi, off, count)) { + ret = 0; + break; + } + } + if (ret) + errno = ENOSYS; /* not a CoW volume */ + } + +cleanup: + close(fdi); + if (close(fdo) && !ret) { + errno = EIO; + ret = -1; + } + if (ret) { + int saved = errno; + unlink(dst); + errno = saved; + return -1; + } + if (adjust_shared_perm(repo, dst)) + return -1; + return 0; +#else + /* No reflink primitive on this platform. */ + (void)repo; + (void)dst; + (void)src; + (void)mode; + errno = ENOSYS; + return -1; +#endif +} diff --git a/copy.h b/copy.h index 1059b118d6..b9168a24f1 100644 --- a/copy.h +++ b/copy.h @@ -11,4 +11,18 @@ int copy_file(struct repository *repo, int copy_file_with_time(struct repository *repo, const char *dst, const char *src, int mode); +/* + * Create "dst" as a copy-on-write (reflink) clone of the regular file + * "src", so that the two files share their data blocks until one of + * them is modified. "dst" must not already exist. + * + * This only succeeds on filesystems that support block cloning (e.g. + * btrfs, XFS or bcachefs on Linux, APFS on macOS). When the platform or + * filesystem does not support reflinks, -1 is returned with errno set to + * ENOSYS (or the underlying error). Callers are expected to fall back to + * copy_file() in that case. Returns 0 on success. + */ +int reflink_file(struct repository *repo, + const char *dst, const char *src, int mode); + #endif /* COPY_H */ diff --git a/t/t2400-worktree-add.sh b/t/t2400-worktree-add.sh index bdcca97633..4f62360a78 100755 --- a/t/t2400-worktree-add.sh +++ b/t/t2400-worktree-add.sh @@ -1288,4 +1288,92 @@ test_expect_success 'relative worktree sets extension config' ' test_cmp_config -C repo true extensions.relativeworktrees ' +test_expect_success '--reflink produces a correct checkout on any filesystem' ' + test_when_finished "rm -rf reflinkrepo" && + git init reflinkrepo && + ( + cd reflinkrepo && + test_commit base file.txt base && + test_commit next file.txt next && + git worktree add --reflink wt -b reflinkbr base && + echo base >expect && + test_cmp expect wt/file.txt && + git -C wt status --porcelain >status && + test_must_be_empty status + ) +' + +test_expect_success '--reflink rejects incompatible options' ' + test_must_fail git worktree add --reflink --no-checkout wt-bad HEAD 2>err && + test_grep "cannot be used together" err && + test_must_fail git worktree add --reflink --orphan wt-bad2 2>err && + test_grep "cannot be used together" err +' + +test_expect_success 'worktree.reflink config drives a correct checkout' ' + test_when_finished "rm -rf reflinkcfg" && + git init reflinkcfg && + ( + cd reflinkcfg && + test_commit cfg file.txt value && + git -c worktree.reflink=true worktree add wt HEAD && + echo value >expect && + test_cmp expect wt/file.txt + ) +' + +test_expect_success 'worktree.reflink=auto does not break --orphan' ' + test_when_finished "rm -rf reflinkorphan" && + git init reflinkorphan && + ( + cd reflinkorphan && + test_commit base file.txt value && + git -c worktree.reflink=auto worktree add --orphan wt && + git -C wt symbolic-ref --short HEAD >actual && + echo wt >expect && + test_cmp expect actual + ) +' + +test_expect_success '--no-reflink overrides worktree.reflink' ' + test_when_finished "rm -rf reflinkoff" && + git init reflinkoff && + ( + cd reflinkoff && + test_commit base file.txt value && + git -c worktree.reflink=true worktree add --no-reflink wt HEAD && + echo value >expect && + test_cmp expect wt/file.txt + ) +' + +# Reflinks are only available on copy-on-write filesystems (btrfs, XFS, +# bcachefs, APFS, ...). Where they are unavailable, --reflink transparently +# falls back to a regular checkout, which the test above already covers. +test_lazy_prereq REFLINK ' + echo probe >reflink-src && + cp --reflink=always reflink-src reflink-dst 2>/dev/null +' + +test_expect_success REFLINK '--reflink carries untracked files and reconciles changes' ' + test_when_finished "rm -rf cowrepo" && + git init cowrepo && + ( + cd cowrepo && + test_commit same same.txt content && + test_commit old change.txt old-value && + test_commit new change.txt new-value && + echo artifact >untracked.bin && + git worktree add --reflink wt -b cowbr old && + echo content >expect && + test_cmp expect wt/same.txt && + echo old-value >expect && + test_cmp expect wt/change.txt && + echo artifact >expect && + test_cmp expect wt/untracked.bin && + git -C wt status --porcelain >status && + test_grep "?? untracked.bin" status + ) +' + test_done -- gitgitgadget ^ permalink raw reply related [flat|nested] 21+ messages in thread
* Re: [PATCH v2 1/2] worktree: add --reflink for copy-on-write worktree creation 2026-10-20 16:04 ` [PATCH v2 1/2] worktree: add --reflink for copy-on-write worktree creation Jason Newton via GitGitGadget @ 2026-10-20 21:20 ` brian m. carlson 2026-10-21 7:15 ` Jason Newton 0 siblings, 1 reply; 21+ messages in thread From: brian m. carlson @ 2026-10-20 21:20 UTC (permalink / raw) To: Jason Newton via GitGitGadget; +Cc: git, Jason Newton [-- Attachment #1: Type: text/plain, Size: 2989 bytes --] On 2026-10-20 at 16:04:08, Jason Newton via GitGitGadget wrote: > Creating many worktrees from the same base -- for example to run a > fleet of automated agents in parallel -- is expensive today: every > "git worktree add" materializes the entire working tree by writing > each tracked file out from the object store. The objects are shared > via the common directory, but the working tree is not: N worktrees > mean N full checkouts on disk and N times the file I/O. > > Add a "--reflink" option that, on a copy-on-write filesystem, populates > the new worktree by reflinking the current worktree's files and index > instead. The subsequent "git reset --hard" then only rewrites the > paths that actually differ between the current worktree and > <commit-ish>; everything else (including untracked files such as build > outputs) keeps sharing storage with the source until modified. > > A new reflink_file() helper in copy.c clones every regular file. There > is no size cutoff. > > On Linux it uses the FICLONE ioctl. That is the primitive btrfs, XFS > and bcachefs implement, and the one overlayfs forwards when the lower > filesystem can clone. This path has been tested on btrfs. > > On macOS it uses clonefile(), the APFS primitive. That path is > implemented and has not been run here. > > On Windows it uses ReFS block cloning, > FSCTL_DUPLICATE_EXTENTS_TO_FILE, as Git LFS does. The destination is > sized to the source, extents are cloned in chunks of at most 1GiB, and > the tail is rounded up to a cluster (64K, then 4K) and cloned; Windows > ignores the range past EOF. The tail is not byte-copied, and small > files are not skipped. That path is implemented and has not been run > on ReFS. A volume that rejects the control code, including NTFS, fails > the probe. That functionality is defective; see https://github.com/git-lfs/git-lfs/issues/6312. We should not implement this on Windows because there's no reasonable way to avoid corruption. It's quite shocking that that bug exists, but it does. Unless we can guarantee a fixed version of Windows (which does not exist yet), we should skip reflinking there. I'd also like to see us limit this to platforms where it _has_ been manually tested or is guaranteed to be automatically tested in CI. That means we should make sure this runs in CI on APFS for macOS (see below). > +# Reflinks are only available on copy-on-write filesystems (btrfs, XFS, > +# bcachefs, APFS, ...). Where they are unavailable, --reflink transparently > +# falls back to a regular checkout, which the test above already covers. > +test_lazy_prereq REFLINK ' > + echo probe >reflink-src && > + cp --reflink=always reflink-src reflink-dst 2>/dev/null > +' Does this work properly on macOS? That is, does the macOS `cp` binary contain support for `--reflink=always`? If not, we won't be testing this case properly on macOS. -- brian m. carlson (they/them) Toronto, Ontario, CA [-- Attachment #2: signature.asc --] [-- Type: application/pgp-signature, Size: 325 bytes --] ^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: [PATCH v2 1/2] worktree: add --reflink for copy-on-write worktree creation 2026-10-20 21:20 ` brian m. carlson @ 2026-10-21 7:15 ` Jason Newton 0 siblings, 0 replies; 21+ messages in thread From: Jason Newton @ 2026-10-21 7:15 UTC (permalink / raw) To: brian m. carlson, Jason Newton via GitGitGadget, git, Jason Newton Brian, macOS cp has no --reflink=always. The clone option is -c, which is clonefile(2). t/t2400-worktree-add.sh already probes that way. The REFLINK prerequisite runs "cp -c" when the MACOS prerequisite is set, and "cp --reflink=always" otherwise. The test that depends on REFLINK is "--reflink carries untracked files and reconciles changes". That is in 042485f0e9cd8cf1765b811ea7334247aec44971. The pull request CI for that commit is the APFS run: https://github.com/git/git/pull/2317 https://github.com/git/git/actions/runs/39502233266 All four macos-14 jobs succeeded. The runners were macOS 14.8.9 (23J631), image macos-14-arm64 20260831.0302.1. "print test failures" was skipped on each job, which is the step that runs only when a test script fails. osx-clang: https://github.com/git/git/actions/runs/39502233266/job/118902393415 t2400-worktree-add.sh ok, 55208 ms osx-gcc: https://github.com/git/git/actions/runs/39502233266/job/118902393588 t2400-worktree-add.sh ok, 45173 ms osx-meson: https://github.com/git/git/actions/runs/39502233266/job/118902393644 osx-reftable: https://github.com/git/git/actions/runs/39502233266/job/118902393664 The macOS evidence is those jobs. Reflink is disabled on Windows in "worktree: disable reflink on Windows" (91806975a6909265b1325206ef2f33a169393920). The pull request head is 876fb541a8e1d43731bb72ddbbf1c49da8760775, which keeps that disable and leaves the filesystem probe compiled so the mingw build does not fail with -Werror=unused-function. The CI run linked above is the previous tip, 042485f. The warning names the build from the git-lfs report, Windows 11 build 26200. A later note on that report reproduced the zero-fill on build 26200.9448 with git-lfs 3.6.1. No fixed build is identified, so every Windows version refuses the ioctl and falls back to a regular checkout. "git worktree add --reflink" and worktree.reflink=true warn with "Windows build 26200". worktree.reflink=auto stays silent. t2400 checks that warning under MINGW. On the git-lfs report, the failure mode is real and documented here: https://github.com/git-lfs/git-lfs/issues/6312 https://github.com/git-lfs/git-lfs/pull/6323 FSCTL_DUPLICATE_EXTENTS_TO_FILE can publish zeros when the source data is still cached. FlushFileBuffers on the read-only handle a checkout uses fails with ERROR_ACCESS_DENIED (win32 5). The change in 6323 reopens the source writable, flushes, and skips that flush when the volume does not advertise FILE_SUPPORTS_BLOCK_REFCOUNTING. The numbers below are from the ioctl path before that disable, on this machine, Windows build 26300.9457. The read-only flush failed with win32 5, the same error the git-lfs report describes. The clones kept the original bytes. NTFS rejects the ioctl: win32 1 on a USB NTFS partition, win32 50 on C:. The fallback there is a normal write. The USB disk would not format as ReFS. The ReFS volume was a 32 GiB Dev Drive, stored as a VHD on that disk, with block refcounting set. The clone path did not flush. Content check, every clone, unbuffered SHA-256: 4000 files, 8,388,608 bytes each (33,554,432,000 bytes, 31.25 GiB of clone names). Created with FSCTL_DUPLICATE_EXTENTS_TO_FILE only. Source before rewrite: 106640827aa82610080b1e4c676e2028230b841fa4642faaa292046833950714 Pass 1: old=4000 new=0 other=0 errors=0 Source rewritten to a different pattern, still no flush: e58408ac406d407257cd3a2474b8bba7b615e76bd4ce03e8ec3df9b42a02bc89 Pass 2: old=4000 new=0 other=0 errors=0 FlushFileBuffers on the read-only handle: FAIL win32=5 Reopen writable and flush: OK Source hash stayed on the rewritten pattern. Pass 3, after that flush: old=4000 new=0 other=0 errors=0 All 4000 clones still hashed to the original source. After they settled, free space was 30,462,386,176 bytes and the backing VHD was still 4,164,943,872 bytes. The volume stayed Healthy. Deleting the clones freed nothing further. This is one Windows build, alongside the version described in git-lfs #6312. On Tue, Oct 20, 2026 at 2:20 PM brian m. carlson <sandals@crustytoothpaste.net> wrote: > > On 2026-10-20 at 16:04:08, Jason Newton via GitGitGadget wrote: > > Creating many worktrees from the same base -- for example to run a > > fleet of automated agents in parallel -- is expensive today: every > > "git worktree add" materializes the entire working tree by writing > > each tracked file out from the object store. The objects are shared > > via the common directory, but the working tree is not: N worktrees > > mean N full checkouts on disk and N times the file I/O. > > > > Add a "--reflink" option that, on a copy-on-write filesystem, populates > > the new worktree by reflinking the current worktree's files and index > > instead. The subsequent "git reset --hard" then only rewrites the > > paths that actually differ between the current worktree and > > <commit-ish>; everything else (including untracked files such as build > > outputs) keeps sharing storage with the source until modified. > > > > A new reflink_file() helper in copy.c clones every regular file. There > > is no size cutoff. > > > > On Linux it uses the FICLONE ioctl. That is the primitive btrfs, XFS > > and bcachefs implement, and the one overlayfs forwards when the lower > > filesystem can clone. This path has been tested on btrfs. > > > > On macOS it uses clonefile(), the APFS primitive. That path is > > implemented and has not been run here. > > > > On Windows it uses ReFS block cloning, > > FSCTL_DUPLICATE_EXTENTS_TO_FILE, as Git LFS does. The destination is > > sized to the source, extents are cloned in chunks of at most 1GiB, and > > the tail is rounded up to a cluster (64K, then 4K) and cloned; Windows > > ignores the range past EOF. The tail is not byte-copied, and small > > files are not skipped. That path is implemented and has not been run > > on ReFS. A volume that rejects the control code, including NTFS, fails > > the probe. > > That functionality is defective; see > https://github.com/git-lfs/git-lfs/issues/6312. We should not implement > this on Windows because there's no reasonable way to avoid corruption. > It's quite shocking that that bug exists, but it does. Unless we can > guarantee a fixed version of Windows (which does not exist yet), we > should skip reflinking there. > > I'd also like to see us limit this to platforms where it _has_ been > manually tested or is guaranteed to be automatically tested in CI. That > means we should make sure this runs in CI on APFS for macOS (see below). > > > +# Reflinks are only available on copy-on-write filesystems (btrfs, XFS, > > +# bcachefs, APFS, ...). Where they are unavailable, --reflink transparently > > +# falls back to a regular checkout, which the test above already covers. > > +test_lazy_prereq REFLINK ' > > + echo probe >reflink-src && > > + cp --reflink=always reflink-src reflink-dst 2>/dev/null > > +' > > Does this work properly on macOS? That is, does the macOS `cp` binary > contain support for `--reflink=always`? If not, we won't be testing > this case properly on macOS. > -- > brian m. carlson (they/them) > Toronto, Ontario, CA ^ permalink raw reply [flat|nested] 21+ messages in thread
* [PATCH v2 2/2] worktree: reap ephemeral branches created by "worktree add" 2026-10-20 16:04 ` [PATCH v2 0/2] worktree: copy-on-write creation and ephemeral branches Jason Newton via GitGitGadget 2026-10-20 16:04 ` [PATCH v2 1/2] worktree: add --reflink for copy-on-write worktree creation Jason Newton via GitGitGadget @ 2026-10-20 16:04 ` Jason Newton via GitGitGadget 2026-10-21 6:52 ` Eric Sunshine 2026-10-21 7:08 ` [PATCH v2 0/2] worktree: copy-on-write creation and ephemeral branches Eric Sunshine 2 siblings, 1 reply; 21+ messages in thread From: Jason Newton via GitGitGadget @ 2026-10-20 16:04 UTC (permalink / raw) To: git; +Cc: brian m. carlson, Jason Newton, Jason Newton From: Jason Newton <nevion@gmail.com> When "git worktree add <path>" is given no branch to check out, it auto-creates one named after the path. Such a branch exists only to back the new worktree, yet it outlives it: removing the worktree leaves the branch behind, so a fleet of throwaway worktrees fills refs/heads/ with dead branches that have to be cleaned up by hand. Treat an auto-created branch as owned by the worktree. Record it in the worktree's administrative directory (worktrees/<id>/owned-branch) at creation time -- the one moment we can know for certain the branch is worktree-private, since we are the one that created it. "git worktree remove" then deletes the owned branch once the worktree is gone and it is no longer checked out. Ownership is recorded only when "worktree add" itself creates the branch via DWIM auto-naming; a branch named explicitly with -b/-B, or one that already existed, is never owned and never reaped. Deletion is unconditional (even for unmerged commits) because an ephemeral branch is disposable by contract: to keep the work, pass --keep-branch, or anchor the commits with an ordinary branch beforehand -- a normal branch can point at the same commits, after which reaping the ephemeral ref loses nothing. "git branch -m" updates the marker so ownership follows a rename, and the branch stays ephemeral. The marker is just another per-worktree administrative file, so older versions of Git simply ignore it (they keep the branch on removal, as today). Signed-off-by: Jason Newton <nevion@gmail.com> --- Documentation/git-worktree.adoc | 22 ++++++++-- builtin/branch.c | 28 +++++++++++++ builtin/worktree.c | 74 ++++++++++++++++++++++++++++++++- t/t2400-worktree-add.sh | 55 ++++++++++++++++++++++++ t/t3200-branch.sh | 2 +- t/t3400-rebase.sh | 2 +- t/t5516-fetch-push.sh | 2 +- 7 files changed, 178 insertions(+), 7 deletions(-) diff --git a/Documentation/git-worktree.adoc b/Documentation/git-worktree.adoc index 4264ee753a..99e6e55be4 100644 --- a/Documentation/git-worktree.adoc +++ b/Documentation/git-worktree.adoc @@ -15,7 +15,7 @@ git worktree list [-v | --porcelain [-z]] git worktree lock [--reason <string>] <worktree> git worktree move <worktree> <new-path> git worktree prune [-n] [-v] [--expire <expire>] -git worktree remove [-f] <worktree> +git worktree remove [-f] [--keep-branch] <worktree> git worktree repair [<path>...] git worktree unlock <worktree> @@ -40,8 +40,11 @@ In its simplest form, `git worktree add <path>` automatically creates a new branch whose name is the final component of _<path>_, which is convenient if you plan to work on a new topic. For instance, `git worktree add ../hotfix` creates new branch `hotfix` and checks it out at -path `../hotfix`. To instead work on an existing branch in a new worktree, -use `git worktree add <path> <branch>`. On the other hand, if you just +path `../hotfix`. Such an automatically-created branch is _ephemeral_: it +is owned by the worktree and is deleted again when the worktree is removed +with `git worktree remove` (see that command and `--keep-branch`). To +instead work on an existing branch in a new worktree, use +`git worktree add <path> <branch>`. On the other hand, if you just plan to make some experimental changes or do testing without disturbing existing development, it is often convenient to create a 'throwaway' worktree not associated with any branch. For instance, @@ -145,6 +148,14 @@ Remove a worktree. Only clean worktrees (no untracked files and no modification in tracked files) can be removed. Unclean worktrees or ones with submodules can be removed with `--force`. The main worktree cannot be removed. ++ +If the worktree owns an _ephemeral_ branch -- one that `git worktree add` +created itself because no branch was named (the auto-naming case above) -- +that branch is deleted along with the worktree, even if it has unmerged +commits, since it was created solely to back this worktree. To keep it, use +`--keep-branch`, or anchor the commits with an ordinary branch +(`git branch <name>`) beforehand. A branch you named explicitly (`-b`/`-B`) +or one that existed before the worktree is never deleted. `repair [<path>...]`:: @@ -192,6 +203,11 @@ allows the move to proceed; use `--force` twice if the destination is locked. `remove` refuses to remove an unclean worktree unless `--force` is used. To remove a locked worktree, specify `--force` twice. +`--keep-branch`:: + With `remove`, do not delete the worktree's ephemeral (owned) branch; + keep it as an ordinary branch. Has no effect on worktrees that do not + own a branch. + `-b <new-branch>`:: `-B <new-branch>`:: With `add`, create a new branch named _<new-branch>_ starting at diff --git a/builtin/branch.c b/builtin/branch.c index a613148fc7..ca2e25df03 100644 --- a/builtin/branch.c +++ b/builtin/branch.c @@ -607,6 +607,32 @@ static int replace_each_worktree_head_symref(struct worktree **worktrees, #define IS_HEAD 1 #define IS_ORPHAN 2 +/* + * When a branch that a worktree owns (its ephemeral branch, recorded in + * worktrees/<id>/owned-branch) is renamed, update the marker so ownership + * follows the new name -- it stays ephemeral. + */ +static void rename_owned_branch_markers(struct worktree **worktrees, + const char *oldref, const char *newref) +{ + for (int i = 0; worktrees[i]; i++) { + struct strbuf path = STRBUF_INIT, buf = STRBUF_INIT; + + if (!worktrees[i]->id) + continue; /* the main worktree keeps no marker */ + repo_common_path_append(the_repository, &path, + "worktrees/%s/owned-branch", + worktrees[i]->id); + if (strbuf_read_file(&buf, path.buf, 64) > 0) { + strbuf_trim(&buf); + if (!strcmp(buf.buf, oldref)) + write_file(path.buf, "%s", newref); + } + strbuf_release(&path); + strbuf_release(&buf); + } +} + static void copy_or_rename_branch(const char *oldname, const char *newname, int copy, int force) { struct strbuf oldref = STRBUF_INIT, newref = STRBUF_INIT, logmsg = STRBUF_INIT; @@ -675,6 +701,8 @@ static void copy_or_rename_branch(const char *oldname, const char *newname, int if (!copy && !(oldref_usage & IS_ORPHAN) && refs_rename_ref(get_main_ref_store(the_repository), oldref.buf, newref.buf, logmsg.buf)) die(_("branch rename failed")); + if (!copy && !(oldref_usage & IS_ORPHAN)) + rename_owned_branch_markers(worktrees, oldref.buf, newref.buf); if (copy && refs_copy_existing_ref(get_main_ref_store(the_repository), oldref.buf, newref.buf, logmsg.buf)) die(_("branch copy failed")); diff --git a/builtin/worktree.c b/builtin/worktree.c index 9d4b51305e..1059429cbb 100644 --- a/builtin/worktree.c +++ b/builtin/worktree.c @@ -41,7 +41,7 @@ #define BUILTIN_WORKTREE_PRUNE_USAGE \ N_("git worktree prune [-n] [-v] [--expire <expire>]") #define BUILTIN_WORKTREE_REMOVE_USAGE \ - N_("git worktree remove [-f] <worktree>") + N_("git worktree remove [-f] [--keep-branch] <worktree>") #define BUILTIN_WORKTREE_REPAIR_USAGE \ N_("git worktree repair [<path>...]") #define BUILTIN_WORKTREE_UNLOCK_USAGE \ @@ -130,8 +130,34 @@ struct add_opts { int relative_paths; int reflink; const char *keep_locked; + const char *owned_branch; }; +/* + * An "owned" (ephemeral) branch is one that "git worktree add" created itself + * because the user did not name a branch (the DWIM auto-name case). Its + * lifetime is bound to the worktree: it is recorded in the worktree's + * administrative directory at creation and reaped when the worktree is + * removed. A pre-existing branch, or one named explicitly with -b/-B, is never + * owned. + */ +static char *read_owned_branch(const char *id) +{ + struct strbuf path = STRBUF_INIT, buf = STRBUF_INIT; + char *ret = NULL; + + repo_common_path_append(the_repository, &path, + "worktrees/%s/owned-branch", id); + if (strbuf_read_file(&buf, path.buf, 64) > 0) { + strbuf_trim(&buf); + if (buf.len) + ret = strbuf_detach(&buf, NULL); + } + strbuf_release(&path); + strbuf_release(&buf); + return ret; +} + static int show_only; static int verbose; static int guess_remote; @@ -771,6 +797,16 @@ static int add_worktree(const char *path, const char *refname, if (the_repository->repository_format_worktree_config) copy_filtered_worktree_config(sb_repo.buf); + /* + * Record an owned (ephemeral) branch so that "git worktree remove" can + * reap it. Only DWIM-created branches are owned (see read_owned_branch()). + */ + if (opts->owned_branch) { + strbuf_reset(&sb); + strbuf_addf(&sb, "%s/owned-branch", sb_repo.buf); + write_file(sb.buf, "refs/heads/%s", opts->owned_branch); + } + strvec_pushf(&child_env, "%s=%s", GIT_DIR_ENVIRONMENT, sb_git.buf); strvec_pushf(&child_env, "%s=%s", GIT_WORK_TREE_ENVIRONMENT, path); @@ -1148,6 +1184,13 @@ static int add(int ac, const char **av, const char *prefix, branch = branch_to_free = s; new_branch = new_branch_to_free; + /* + * A branch auto-created from the path (the user named no + * branch) is owned by this worktree and reaped on removal. + */ + if (new_branch_to_free) + opts.owned_branch = new_branch_to_free; + /* DWIM: Infer --orphan when repo has no refs. */ opts.orphan = (!s) && dwim_orphan(&opts, !!opt_track, 1); } else if (ac == 2 && new_branch) { @@ -1651,15 +1694,19 @@ static int remove_worktree(int ac, const char **av, const char *prefix, struct repository *repo UNUSED) { int force = 0; + int keep_branch = 0; struct option options[] = { OPT__FORCE(&force, N_("force removal even if worktree is dirty or locked"), PARSE_OPT_NOCOMPLETE), + OPT_BOOL(0, "keep-branch", &keep_branch, + N_("keep the worktree's owned (ephemeral) branch")), OPT_END() }; struct worktree **worktrees, *wt; struct strbuf errmsg = STRBUF_INIT; const char *reason = NULL; + char *owned = NULL; int ret = 0; ac = parse_options(ac, av, prefix, options, git_worktree_remove_usage, 0); @@ -1685,6 +1732,10 @@ static int remove_worktree(int ac, const char **av, const char *prefix, errmsg.buf); strbuf_release(&errmsg); + /* read the owned-branch marker before the admin dir is deleted */ + if (!keep_branch) + owned = read_owned_branch(wt->id); + if (file_exists(wt->path)) { if (!force) check_clean_worktree(wt, av[0]); @@ -1698,6 +1749,27 @@ static int remove_worktree(int ac, const char **av, const char *prefix, ret |= delete_git_dir(wt->id); delete_worktrees_dir_if_empty(); + /* + * Reap the worktree's owned (ephemeral) branch now that the worktree + * is gone and the branch is no longer checked out. This is + * unconditional by design: the branch was created for this worktree + * and is disposable. To keep the work, anchor it with an ordinary + * branch (or pass --keep-branch) before removing the worktree. + */ + if (owned) { + struct ref_store *refs = get_main_ref_store(the_repository); + if (refs_ref_exists(refs, owned)) { + if (refs_delete_ref(refs, NULL, owned, NULL, 0)) + ret = error(_("failed to reap branch '%s'"), owned); + else { + const char *name = owned; + skip_prefix(owned, "refs/heads/", &name); + fprintf_ln(stderr, _("Removed branch '%s'"), name); + } + } + free(owned); + } + free_worktrees(worktrees); return ret; } diff --git a/t/t2400-worktree-add.sh b/t/t2400-worktree-add.sh index 4f62360a78..d083a871eb 100755 --- a/t/t2400-worktree-add.sh +++ b/t/t2400-worktree-add.sh @@ -1376,4 +1376,59 @@ test_expect_success REFLINK '--reflink carries untracked files and reconciles ch ) ' +test_expect_success 'ephemeral: DWIM-created branch is owned and reaped on remove' ' + test_when_finished "git worktree remove -f eph || :; git branch -D eph 2>/dev/null || :" && + git worktree add eph && + echo refs/heads/eph >expect && + test_cmp expect .git/worktrees/eph/owned-branch && + git rev-parse --verify refs/heads/eph && + git worktree remove eph && + test_must_fail git rev-parse --verify --quiet refs/heads/eph +' + +test_expect_success 'ephemeral: -b branch is not owned and is kept on remove' ' + test_when_finished "git worktree remove -f nb || :; git branch -D named 2>/dev/null || :" && + git worktree add -b named nb && + test_path_is_missing .git/worktrees/nb/owned-branch && + git worktree remove nb && + git rev-parse --verify refs/heads/named +' + +test_expect_success 'ephemeral: pre-existing branch is not owned' ' + test_when_finished "git worktree remove -f pe || :; git branch -D pre 2>/dev/null || :" && + git branch pre && + git worktree add pe pre && + test_path_is_missing .git/worktrees/pe/owned-branch && + git worktree remove pe && + git rev-parse --verify refs/heads/pre +' + +test_expect_success 'ephemeral: --keep-branch keeps an owned branch' ' + test_when_finished "git branch -D keepme 2>/dev/null || :" && + git worktree add keepme && + git worktree remove --keep-branch keepme && + git rev-parse --verify refs/heads/keepme +' + +test_expect_success 'ephemeral: branch -m follows the marker and stays ephemeral' ' + test_when_finished "git worktree remove -f mv || :; git branch -D mv2 2>/dev/null || :" && + git worktree add mv && + git -C mv branch -m mv mv2 && + echo refs/heads/mv2 >expect && + test_cmp expect .git/worktrees/mv/owned-branch && + git worktree remove mv && + test_must_fail git rev-parse --verify --quiet refs/heads/mv2 +' + +test_expect_success 'ephemeral: commits survive the reap when anchored by a normal branch' ' + test_when_finished "git worktree remove -f anc || :; git branch -D anchored 2>/dev/null || :" && + git worktree add anc && + test_commit -C anc agentwork && + tip=$(git -C anc rev-parse HEAD) && + git branch anchored "$tip" && + git worktree remove anc && + test_must_fail git rev-parse --verify --quiet refs/heads/anc && + test "$(git rev-parse anchored)" = "$tip" +' + test_done diff --git a/t/t3200-branch.sh b/t/t3200-branch.sh index cdb6c6a634..67c7c67853 100755 --- a/t/t3200-branch.sh +++ b/t/t3200-branch.sh @@ -190,7 +190,7 @@ test_expect_success 'git branch -M foo bar should fail when bar is checked out' test_expect_success 'git branch -M foo bar should fail when bar is checked out in worktree' ' git branch -f bar && - test_when_finished "git worktree remove wt && git branch -D wt" && + test_when_finished "git worktree remove --keep-branch wt && git branch -D wt" && git worktree add wt && test_must_fail git branch -M bar wt ' diff --git a/t/t3400-rebase.sh b/t/t3400-rebase.sh index accaaba4d4..9c0c80f113 100755 --- a/t/t3400-rebase.sh +++ b/t/t3400-rebase.sh @@ -471,7 +471,7 @@ test_expect_success 'git rebase --update-ref with core.commentChar and branch on test_when_finished git branch -D base topic2 && test_when_finished git checkout main && test_when_finished git branch -D wt-topic && - test_when_finished git worktree remove wt-topic && + test_when_finished git worktree remove --keep-branch wt-topic && git checkout main && git checkout -b base && git checkout -b topic2 && diff --git a/t/t5516-fetch-push.sh b/t/t5516-fetch-push.sh index b982b209bf..6ffeb064fa 100755 --- a/t/t5516-fetch-push.sh +++ b/t/t5516-fetch-push.sh @@ -1873,7 +1873,7 @@ test_expect_success 'updateInstead with bare repository worktree and unborn bare ' test_expect_success 'refuse fetch to current branch of worktree' ' - test_when_finished "git worktree remove --force wt && git branch -D wt" && + test_when_finished "git worktree remove --force --keep-branch wt && git branch -D wt" && git worktree add wt && test_commit apple && test_must_fail git fetch . HEAD:wt && -- gitgitgadget ^ permalink raw reply related [flat|nested] 21+ messages in thread
* Re: [PATCH v2 2/2] worktree: reap ephemeral branches created by "worktree add" 2026-10-20 16:04 ` [PATCH v2 2/2] worktree: reap ephemeral branches created by "worktree add" Jason Newton via GitGitGadget @ 2026-10-21 6:52 ` Eric Sunshine 2026-10-21 7:40 ` Jason Newton 0 siblings, 1 reply; 21+ messages in thread From: Eric Sunshine @ 2026-10-21 6:52 UTC (permalink / raw) To: Jason Newton via GitGitGadget; +Cc: git, brian m. carlson, Jason Newton On Tue, Oct 20, 2026 at 12:09 PM Jason Newton via GitGitGadget <gitgitgadget@gmail.com> wrote: > When "git worktree add <path>" is given no branch to check out, it > auto-creates one named after the path. Such a branch exists only to back > the new worktree, yet it outlives it: removing the worktree leaves the > branch behind, so a fleet of throwaway worktrees fills refs/heads/ with > dead branches that have to be cleaned up by hand. > > Treat an auto-created branch as owned by the worktree. Record it in the > worktree's administrative directory (worktrees/<id>/owned-branch) at > creation time -- the one moment we can know for certain the branch is > worktree-private, since we are the one that created it. "git worktree > remove" then deletes the owned branch once the worktree is gone and it > is no longer checked out. > > Ownership is recorded only when "worktree add" itself creates the branch > via DWIM auto-naming; a branch named explicitly with -b/-B, or one that > already existed, is never owned and never reaped. Deletion is > unconditional (even for unmerged commits) because an ephemeral branch is > disposable by contract: to keep the work, pass --keep-branch, or anchor > the commits with an ordinary branch beforehand -- a normal branch can > point at the same commits, after which reaping the ephemeral ref loses > nothing. "git branch -m" updates the marker so ownership follows a > rename, and the branch stays ephemeral. > > The marker is just another per-worktree administrative file, so older > versions of Git simply ignore it (they keep the branch on removal, as > today). > > Signed-off-by: Jason Newton <nevion@gmail.com> Sorry, I may be missing something obvious, but this message never explains why we should add all this additional complexity for a use-case already directly supported by `git worktree add -d <path>`. Specifically, worktree's --detach option is exactly what you want when creating throwaway worktrees, even "fleets" of them (using terminology from this apparently fully or partially LLM-generated message). If your tooling or agent is not using -d/--detach for throwaway worktrees, then the problem is fixable by modifying the tooling or by teaching the agent about --detach, without bulking up the git-worktree implementation unnecessarily. Moreover, a change like this which makes the new behavior unconditional is certainly going to break existing tooling and workflows, thus it will not go over well, even with the --keep-branch escape-hatch. At minimum, to be acceptable, assuming such new functionality is even desirable, to avoid breaking existing setups, it would have to be opt-in, not opt-out (via --keep-branch). But if it's going to be opt-in, then you may as well just use --detach to achieve the same effect. ^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: [PATCH v2 2/2] worktree: reap ephemeral branches created by "worktree add" 2026-10-21 6:52 ` Eric Sunshine @ 2026-10-21 7:40 ` Jason Newton 2026-10-21 8:06 ` Eric Sunshine 0 siblings, 1 reply; 21+ messages in thread From: Jason Newton @ 2026-10-21 7:40 UTC (permalink / raw) To: Eric Sunshine; +Cc: Jason Newton via GitGitGadget, git, brian m. carlson --detach and an ephemeral branch differ while the worktree exists. --detach leaves HEAD with no branch, so status, push, rename, upstream, and symbolic-ref have nothing to name. An ephemeral branch is a normal refs/heads/ ref for the life of the worktree. Commits go on it, and another ref can point at them before "worktree remove" deletes that branch. Deleting the branch that "worktree add <path>" creates today would change the default, so this has to be opt-in. That opt-in is not --detach: the worktree stays on a branch until removal. These worktrees might be minutes or they might go on for months; either way they should be easy to use and not require flags to assist in the majority of common case behaviors, moreover detached feels ike a degraded halfway state, frequently allowing a gotcha to strike or a loss of a safety like a gc based reap. "git worktree remove" does not delete whatever branch a worktree has checked out. It deletes a branch only when this "git worktree add" created it by DWIM (the final component of <path>) and recorded that in worktrees/<id>/owned-branch. A branch named with -b/-B, a branch that already existed, and a detached HEAD are not owned and are not removed. A worktree created by an older Git has no marker, so removing it still leaves its branch in place. "Unconditional" in the commit means that this one owned branch is deleted even if it has unmerged commits. --keep-branch turns that off. "git branch <name> <tip>" before removal keeps the commits on an ordinary branch; only the owned ref is deleted. The call that changes is "git worktree add <path>" with no branch, followed by "git worktree remove". A caller who wants that branch to outlive the worktree already has a spelling this series does not touch: "git worktree add -b <name> <path>", or "git worktree add <path> <existing>". --detach still creates no branch, so it is not the same state while the worktree exists. Ownership is written only when the worktree is created, so I'll confirm an older worktree still has no marker after an upgrade. The branch stays. Upgrading does not mark a worktree you already have so no upgrade headaches are expected. -Jason On Tue, Oct 20, 2026 at 11:52 PM Eric Sunshine <sunshine@sunshineco.com> wrote: > > On Tue, Oct 20, 2026 at 12:09 PM Jason Newton via GitGitGadget > <gitgitgadget@gmail.com> wrote: > > When "git worktree add <path>" is given no branch to check out, it > > auto-creates one named after the path. Such a branch exists only to back > > the new worktree, yet it outlives it: removing the worktree leaves the > > branch behind, so a fleet of throwaway worktrees fills refs/heads/ with > > dead branches that have to be cleaned up by hand. > > > > Treat an auto-created branch as owned by the worktree. Record it in the > > worktree's administrative directory (worktrees/<id>/owned-branch) at > > creation time -- the one moment we can know for certain the branch is > > worktree-private, since we are the one that created it. "git worktree > > remove" then deletes the owned branch once the worktree is gone and it > > is no longer checked out. > > > > Ownership is recorded only when "worktree add" itself creates the branch > > via DWIM auto-naming; a branch named explicitly with -b/-B, or one that > > already existed, is never owned and never reaped. Deletion is > > unconditional (even for unmerged commits) because an ephemeral branch is > > disposable by contract: to keep the work, pass --keep-branch, or anchor > > the commits with an ordinary branch beforehand -- a normal branch can > > point at the same commits, after which reaping the ephemeral ref loses > > nothing. "git branch -m" updates the marker so ownership follows a > > rename, and the branch stays ephemeral. > > > > The marker is just another per-worktree administrative file, so older > > versions of Git simply ignore it (they keep the branch on removal, as > > today). > > > > Signed-off-by: Jason Newton <nevion@gmail.com> > > Sorry, I may be missing something obvious, but this message never > explains why we should add all this additional complexity for a > use-case already directly supported by `git worktree add -d <path>`. > Specifically, worktree's --detach option is exactly what you want when > creating throwaway worktrees, even "fleets" of them (using terminology > from this apparently fully or partially LLM-generated message). If > your tooling or agent is not using -d/--detach for throwaway > worktrees, then the problem is fixable by modifying the tooling or by > teaching the agent about --detach, without bulking up the git-worktree > implementation unnecessarily. > > Moreover, a change like this which makes the new behavior > unconditional is certainly going to break existing tooling and > workflows, thus it will not go over well, even with the --keep-branch > escape-hatch. At minimum, to be acceptable, assuming such new > functionality is even desirable, to avoid breaking existing setups, it > would have to be opt-in, not opt-out (via --keep-branch). But if it's > going to be opt-in, then you may as well just use --detach to achieve > the same effect. ^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: [PATCH v2 2/2] worktree: reap ephemeral branches created by "worktree add" 2026-10-21 7:40 ` Jason Newton @ 2026-10-21 8:06 ` Eric Sunshine 2026-10-21 8:28 ` Jason Newton 0 siblings, 1 reply; 21+ messages in thread From: Eric Sunshine @ 2026-10-21 8:06 UTC (permalink / raw) To: Jason Newton; +Cc: Jason Newton via GitGitGadget, git, brian m. carlson On Wed, Oct 21, 2026 at 3:40 AM Jason Newton <nevion@gmail.com> wrote:> > --detach and an ephemeral branch differ while the worktree exists. > > --detach leaves HEAD with no branch, so status, push, rename, > upstream, and symbolic-ref have nothing to name. An ephemeral > branch is a normal refs/heads/ ref for the life of the worktree. > Commits go on it, and another ref can point at them before > "worktree remove" deletes that branch. > > Deleting the branch that "worktree add <path>" creates today > would change the default, so this has to be opt-in. That opt-in > is not --detach: the worktree stays on a branch until removal. > > These worktrees might be minutes or they might go on for months; > either way they should be easy to use and not require flags to assist > in the majority of common case behaviors, moreover detached feels > ike a degraded halfway state, frequently allowing a gotcha to strike or > a loss of a safety like a gc based reap. > > "git worktree remove" does not delete whatever branch a worktree > has checked out. It deletes a branch only when this "git worktree > add" created it by DWIM (the final component of <path>) and recorded > that in worktrees/<id>/owned-branch. > > A branch named with -b/-B, a branch that already existed, and a > detached HEAD are not owned and are not removed. A worktree created > by an older Git has no marker, so removing it still leaves its > branch in place. > > "Unconditional" in the commit means that this one owned branch is > deleted even if it has unmerged commits. --keep-branch turns that > off. "git branch <name> <tip>" before removal keeps the commits on > an ordinary branch; only the owned ref is deleted. > > The call that changes is "git worktree add <path>" with no branch, > followed by "git worktree remove". A caller who wants that branch > to outlive the worktree already has a spelling this series does not > touch: "git worktree add -b <name> <path>", or "git worktree add > <path> <existing>". --detach still creates no branch, so it is not > the same state while the worktree exists. > > Ownership is written only when the worktree is created, so I'll confirm > an older worktree still has no marker after an upgrade. The branch stays. > Upgrading does not mark a worktree you already have so no upgrade > headaches are expected. Feeding a reviewer's comments into an agent and pasting its LLM-generated response into an email is not a good way to convince project stakeholders that the submission has value. If you can't explain its value in concrete terms in your own words, then you're going to have a hard time eliciting any responses or additional reviewer feedback. ^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: [PATCH v2 2/2] worktree: reap ephemeral branches created by "worktree add" 2026-10-21 8:06 ` Eric Sunshine @ 2026-10-21 8:28 ` Jason Newton 2026-10-21 14:03 ` Phillip Wood 0 siblings, 1 reply; 21+ messages in thread From: Jason Newton @ 2026-10-21 8:28 UTC (permalink / raw) To: Eric Sunshine; +Cc: Jason Newton via GitGitGadget, git, brian m. carlson Please focus on reviewing, I use mixed but you shouldn't assume which parts are and are not llms. What you don't realize by working in a silo for a long time is your communication skills to external parties gets reduced with jargon and nitpicking ; I have the llms both assume you're wrong and right while seeing the undesirable behaviors your stances have to the one that I know is correct. You seem overly sensitive to the usage so far. I have no tollerance left to give - so stay on point. You should consider several harnesses chose not to even go through PRs and work around the functionality like grok build did, patching it in externally which is kind of nuts - and replicated. I think it's something you should think about the nature of quite a bit. As I mentioned, detached IOWs sucks and you shouldn't have bothered mentioning it because it fails hard, though it was noted and evaluated in earnest for a bad idea - want to go back to the long but more neutral form? -Jason On Wed, Oct 21, 2026 at 1:07 AM Eric Sunshine <sunshine@sunshineco.com> wrote: > > On Wed, Oct 21, 2026 at 3:40 AM Jason Newton <nevion@gmail.com> wrote:> > > --detach and an ephemeral branch differ while the worktree exists. > > > > --detach leaves HEAD with no branch, so status, push, rename, > > upstream, and symbolic-ref have nothing to name. An ephemeral > > branch is a normal refs/heads/ ref for the life of the worktree. > > Commits go on it, and another ref can point at them before > > "worktree remove" deletes that branch. > > > > Deleting the branch that "worktree add <path>" creates today > > would change the default, so this has to be opt-in. That opt-in > > is not --detach: the worktree stays on a branch until removal. > > > > These worktrees might be minutes or they might go on for months; > > either way they should be easy to use and not require flags to assist > > in the majority of common case behaviors, moreover detached feels > > ike a degraded halfway state, frequently allowing a gotcha to strike or > > a loss of a safety like a gc based reap. > > > > "git worktree remove" does not delete whatever branch a worktree > > has checked out. It deletes a branch only when this "git worktree > > add" created it by DWIM (the final component of <path>) and recorded > > that in worktrees/<id>/owned-branch. > > > > A branch named with -b/-B, a branch that already existed, and a > > detached HEAD are not owned and are not removed. A worktree created > > by an older Git has no marker, so removing it still leaves its > > branch in place. > > > > "Unconditional" in the commit means that this one owned branch is > > deleted even if it has unmerged commits. --keep-branch turns that > > off. "git branch <name> <tip>" before removal keeps the commits on > > an ordinary branch; only the owned ref is deleted. > > > > The call that changes is "git worktree add <path>" with no branch, > > followed by "git worktree remove". A caller who wants that branch > > to outlive the worktree already has a spelling this series does not > > touch: "git worktree add -b <name> <path>", or "git worktree add > > <path> <existing>". --detach still creates no branch, so it is not > > the same state while the worktree exists. > > > > Ownership is written only when the worktree is created, so I'll confirm > > an older worktree still has no marker after an upgrade. The branch stays. > > Upgrading does not mark a worktree you already have so no upgrade > > headaches are expected. > > Feeding a reviewer's comments into an agent and pasting its > LLM-generated response into an email is not a good way to convince > project stakeholders that the submission has value. If you can't > explain its value in concrete terms in your own words, then you're > going to have a hard time eliciting any responses or additional > reviewer feedback. ^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: [PATCH v2 2/2] worktree: reap ephemeral branches created by "worktree add" 2026-10-21 8:28 ` Jason Newton @ 2026-10-21 14:03 ` Phillip Wood 0 siblings, 0 replies; 21+ messages in thread From: Phillip Wood @ 2026-10-21 14:03 UTC (permalink / raw) To: Jason Newton, Eric Sunshine Cc: Jason Newton via GitGitGadget, git, brian m. carlson On 21/10/2026 09:28, Jason Newton wrote: > Please focus on reviewing, I use mixed but you shouldn't assume which > parts are and are not llms. > > What you don't realize by working in a silo for a long time is your > communication skills to external parties gets reduced with jargon and > nitpicking ; I have the llms both assume you're wrong and right while > seeing the undesirable behaviors your stances have to the one that I > know is correct. You seem overly sensitive to the usage so far. I > have no tollerance left to give - so stay on point. You should > consider several harnesses chose not to even go through PRs and work > around the functionality like grok build did, patching it in > externally which is kind of nuts - and replicated. I think it's > something you should think about the nature of quite a bit. > > As I mentioned, detached IOWs sucks and you shouldn't have bothered > mentioning it because it fails hard, though it was noted and evaluated > in earnest for a bad idea - want to go back to the long but more > neutral form? I'm afraid this messages falls far short of the standard of constructive discussion that is expected on this list. It is perfectly possible to disagree with someone without attacking them or telling them what they should or should not be doing. It is clear that you do not like the current behavior of "git worktree add", but replying with a generic list of the differences between a branch checkout and a detached HEAD and then attacking the reviewer does not explain what problem you are trying to solve and is not going to convince anyone that this patch is a good idea. Thanks Phillip > -Jason > > > On Wed, Oct 21, 2026 at 1:07 AM Eric Sunshine <sunshine@sunshineco.com> wrote: >> >> On Wed, Oct 21, 2026 at 3:40 AM Jason Newton <nevion@gmail.com> wrote:> >>> --detach and an ephemeral branch differ while the worktree exists. >>> >>> --detach leaves HEAD with no branch, so status, push, rename, >>> upstream, and symbolic-ref have nothing to name. An ephemeral >>> branch is a normal refs/heads/ ref for the life of the worktree. >>> Commits go on it, and another ref can point at them before >>> "worktree remove" deletes that branch. >>> >>> Deleting the branch that "worktree add <path>" creates today >>> would change the default, so this has to be opt-in. That opt-in >>> is not --detach: the worktree stays on a branch until removal. >>> >>> These worktrees might be minutes or they might go on for months; >>> either way they should be easy to use and not require flags to assist >>> in the majority of common case behaviors, moreover detached feels >>> ike a degraded halfway state, frequently allowing a gotcha to strike or >>> a loss of a safety like a gc based reap. >>> >>> "git worktree remove" does not delete whatever branch a worktree >>> has checked out. It deletes a branch only when this "git worktree >>> add" created it by DWIM (the final component of <path>) and recorded >>> that in worktrees/<id>/owned-branch. >>> >>> A branch named with -b/-B, a branch that already existed, and a >>> detached HEAD are not owned and are not removed. A worktree created >>> by an older Git has no marker, so removing it still leaves its >>> branch in place. >>> >>> "Unconditional" in the commit means that this one owned branch is >>> deleted even if it has unmerged commits. --keep-branch turns that >>> off. "git branch <name> <tip>" before removal keeps the commits on >>> an ordinary branch; only the owned ref is deleted. >>> >>> The call that changes is "git worktree add <path>" with no branch, >>> followed by "git worktree remove". A caller who wants that branch >>> to outlive the worktree already has a spelling this series does not >>> touch: "git worktree add -b <name> <path>", or "git worktree add >>> <path> <existing>". --detach still creates no branch, so it is not >>> the same state while the worktree exists. >>> >>> Ownership is written only when the worktree is created, so I'll confirm >>> an older worktree still has no marker after an upgrade. The branch stays. >>> Upgrading does not mark a worktree you already have so no upgrade >>> headaches are expected. >> >> Feeding a reviewer's comments into an agent and pasting its >> LLM-generated response into an email is not a good way to convince >> project stakeholders that the submission has value. If you can't >> explain its value in concrete terms in your own words, then you're >> going to have a hard time eliciting any responses or additional >> reviewer feedback. > ^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: [PATCH v2 0/2] worktree: copy-on-write creation and ephemeral branches 2026-10-20 16:04 ` [PATCH v2 0/2] worktree: copy-on-write creation and ephemeral branches Jason Newton via GitGitGadget 2026-10-20 16:04 ` [PATCH v2 1/2] worktree: add --reflink for copy-on-write worktree creation Jason Newton via GitGitGadget 2026-10-20 16:04 ` [PATCH v2 2/2] worktree: reap ephemeral branches created by "worktree add" Jason Newton via GitGitGadget @ 2026-10-21 7:08 ` Eric Sunshine 2 siblings, 0 replies; 21+ messages in thread From: Eric Sunshine @ 2026-10-21 7:08 UTC (permalink / raw) To: Jason Newton via GitGitGadget; +Cc: git, brian m. carlson, Jason Newton On Tue, Oct 20, 2026 at 12:08 PM Jason Newton via GitGitGadget <gitgitgadget@gmail.com> wrote: > When many worktrees share one repository -- for example a fleet of agents > each needing an isolated checkout -- "git worktree add" is costly at scale. > Objects are shared via the common directory, but the working tree is not: > each add rewrites every tracked file, so N worktrees cost N full checkouts > of disk and I/O. A branch created only to back one of those worktrees is > also left behind when the worktree is removed. > > Patch 1 adds "git worktree add --reflink". On a copy-on-write filesystem it > populates the new worktree by reflinking the current worktree's files and > index, then "git reset --hard" rewrites only the paths that differ from . > reflink_file() in copy.c uses FICLONE (Linux), clonefile() (macOS), and ReFS > block cloning (FSCTL_DUPLICATE_EXTENTS_TO_FILE, as Git LFS does). Elsewhere > it is probed up front and falls back to a normal checkout. worktree.reflink > (true/false/auto) sets the default; --no-reflink overrides. > > Changes since v1: > [...] > * Kept --reflink opt-in. A whole-tree reflink is not invisible: it carries > untracked files, and cloning every file can be slower than checkout when > the files are small (Junio; qub1n on the ReFS work). I may be missing something, but it's still not clear from this explanation why --reflink must be opt-in rather than default behavior. The above only seems to be touching on Junio's superficial "cp treeA treeB" example[*], but doesn't answer the deeper question he asked about why this can't be the default behavior for individual tracked resources in the worktree (his "a_hash at a_path" example). [*]: https://lore.kernel.org/git/xmqq8q8pxfny.fsf@gitster.g/ ^ permalink raw reply [flat|nested] 21+ messages in thread
* [PATCH v3 0/4] worktree: copy-on-write creation and ephemeral branches 2026-06-05 18:49 [PATCH 0/2] worktree: copy-on-write creation and shared-branch worktrees Jason Newton via GitGitGadget ` (4 preceding siblings ...) 2026-10-20 16:04 ` [PATCH v2 0/2] worktree: copy-on-write creation and ephemeral branches Jason Newton via GitGitGadget @ 2026-10-21 7:17 ` Jason Newton via GitGitGadget 2026-10-21 7:17 ` [PATCH v3 1/4] worktree: add --reflink for copy-on-write worktree creation Jason Newton via GitGitGadget ` (3 more replies) 5 siblings, 4 replies; 21+ messages in thread From: Jason Newton via GitGitGadget @ 2026-10-21 7:17 UTC (permalink / raw) To: git; +Cc: brian m. carlson, Jason Newton When many worktrees share one repository -- for example a fleet of agents each needing an isolated checkout -- "git worktree add" is costly at scale. Objects are shared via the common directory, but the working tree is not: each add rewrites every tracked file, so N worktrees cost N full checkouts of disk and I/O. A branch created only to back one of those worktrees is also left behind when the worktree is removed. Patch 1 adds "git worktree add --reflink". On a copy-on-write filesystem it populates the new worktree by reflinking the current worktree's files and index, then "git reset --hard" rewrites only the paths that differ from . reflink_file() in copy.c uses FICLONE (Linux), clonefile() (macOS), and ReFS block cloning (FSCTL_DUPLICATE_EXTENTS_TO_FILE, as Git LFS does). Elsewhere it is probed up front and falls back to a normal checkout. worktree.reflink (true/false/auto) sets the default; --no-reflink overrides. Uncommitted edits to tracked files in the source do not leak into the new worktree. The copied index is refreshed before reset, and because every reflinked file is a new inode the refresh re-validates by content. Only untracked files are carried over. The option stays opt-in for that reason: it copies the source's untracked and ignored files, which a plain checkout does not. For many worktrees at one commit, the series does not share a branch. git-worktree(1) shows a single update-ref transaction that creates a ref per worktree (or a detached HEAD). A shared branch still lets a stale index in one worktree silently revert a commit made in another. Patch 2 treats a branch that "git worktree add " created itself as owned by that worktree. "git worktree remove" deletes it. Branches named with -b/-B, or that already existed, are left alone. --keep-branch keeps an owned branch. Changes since v1: * Dropped the patch that allowed one branch to be checked out in several worktrees. Documented the per-ref / detached-HEAD pattern instead (brian m. carlson, Junio). * Explained, in the commit message and the manual, why a dirty source worktree does not leak tracked content (brian m. carlson). * Added the ReFS block-cloning arm (brian m. carlson). * Noted overlayfs as a supported setup when the lower filesystem supports reflinks (brian m. carlson). * Kept --reflink opt-in. A whole-tree reflink is not invisible: it carries untracked files, and cloning every file can be slower than checkout when the files are small (Junio; qub1n on the ReFS work). * Rebased onto 47fdf2208c ("The 7th batch"). reflink_file() now takes a repository argument, matching copy_file(). * Added ephemeral-branch reaping for the auto-named branch that "worktree add " creates. ReFS measurements (32 GiB Dev Drive, Windows build 26300). The USB disk itself would not format as ReFS, so the volume is a Dev Drive VHD on an NTFS partition of that disk. NTFS rejects the clone ioctl (win32 error 1 there, error 50 on C:) and falls back to a full write. The clone shares the source clusters and does not flush. Four thousand clones of one 8 MiB file, 31.3 GiB of logical names, were read back unbuffered and SHA-256'd in full: 4000 of 4000 matched the source. The source was then overwritten with a different pattern. A read-only FlushFileBuffers failed with win32 error 5; a writable reopen succeeded. After that flush, 4000 of 4000 clones still hashed to the original contents and none hashed to the new source. While the clones were created the free-space counter fell to about 1 GiB, then returned to 28.38 GiB with the clones still present. The backing VHD did not grow. Deleting the clones freed nothing further. Jason Newton (4): worktree: add --reflink for copy-on-write worktree creation worktree: reap ephemeral branches created by "worktree add" worktree: disable reflink on Windows worktree: keep the reflink probe referenced on Windows Documentation/config/worktree.adoc | 12 ++ Documentation/git-worktree.adoc | 75 ++++++- builtin/branch.c | 28 +++ builtin/worktree.c | 315 ++++++++++++++++++++++++++++- copy.c | 83 ++++++++ copy.h | 14 ++ t/t2400-worktree-add.sh | 166 +++++++++++++++ t/t3200-branch.sh | 2 +- t/t3400-rebase.sh | 2 +- t/t5516-fetch-push.sh | 2 +- 10 files changed, 690 insertions(+), 9 deletions(-) base-commit: 47fdf2208c5b94c8cc01c106b8f7c85efb91c926 Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-git-2317%2Fnevion%2Fworktree-reflink-cow-v3 Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-git-2317/nevion/worktree-reflink-cow-v3 Pull-Request: https://github.com/git/git/pull/2317 Range-diff vs v2: 1: 83841020f4 ! 1: a163f737af worktree: add --reflink for copy-on-write worktree creation @@ Commit message and bcachefs implement, and the one overlayfs forwards when the lower filesystem can clone. This path has been tested on btrfs. - On macOS it uses clonefile(), the APFS primitive. That path is - implemented and has not been run here. + On macOS it uses clonefile(), the APFS primitive. The t2400 + REFLINK prerequisite probes that with "cp -c", because BSD cp has no + --reflink=always, so the macOS CI jobs exercise this path on APFS. On Windows it uses ReFS block cloning, FSCTL_DUPLICATE_EXTENTS_TO_FILE, as Git LFS does. The destination is @@ t/t2400-worktree-add.sh: test_expect_success 'relative worktree sets extension c +# Reflinks are only available on copy-on-write filesystems (btrfs, XFS, +# bcachefs, APFS, ...). Where they are unavailable, --reflink transparently +# falls back to a regular checkout, which the test above already covers. ++# ++# GNU cp probes with --reflink=always. macOS cp has no such option; -c ++# is clonefile(2), which is what the APFS CI jobs exercise. +test_lazy_prereq REFLINK ' + echo probe >reflink-src && -+ cp --reflink=always reflink-src reflink-dst 2>/dev/null ++ if test_have_prereq MACOS ++ then ++ cp -c reflink-src reflink-dst 2>/dev/null ++ else ++ cp --reflink=always reflink-src reflink-dst 2>/dev/null ++ fi +' + +test_expect_success REFLINK '--reflink carries untracked files and reconciles changes' ' 2: 6e2acdc5e2 = 2: 042485f0e9 worktree: reap ephemeral branches created by "worktree add" -: ---------- > 3: 91806975a6 worktree: disable reflink on Windows -: ---------- > 4: 876fb541a8 worktree: keep the reflink probe referenced on Windows -- gitgitgadget ^ permalink raw reply [flat|nested] 21+ messages in thread
* [PATCH v3 1/4] worktree: add --reflink for copy-on-write worktree creation 2026-10-21 7:17 ` [PATCH v3 0/4] " Jason Newton via GitGitGadget @ 2026-10-21 7:17 ` Jason Newton via GitGitGadget 2026-10-21 7:17 ` [PATCH v3 2/4] worktree: reap ephemeral branches created by "worktree add" Jason Newton via GitGitGadget ` (2 subsequent siblings) 3 siblings, 0 replies; 21+ messages in thread From: Jason Newton via GitGitGadget @ 2026-10-21 7:17 UTC (permalink / raw) To: git; +Cc: brian m. carlson, Jason Newton, Jason Newton From: Jason Newton <nevion@gmail.com> Creating many worktrees from the same base -- for example to run a fleet of automated agents in parallel -- is expensive today: every "git worktree add" materializes the entire working tree by writing each tracked file out from the object store. The objects are shared via the common directory, but the working tree is not: N worktrees mean N full checkouts on disk and N times the file I/O. Add a "--reflink" option that, on a copy-on-write filesystem, populates the new worktree by reflinking the current worktree's files and index instead. The subsequent "git reset --hard" then only rewrites the paths that actually differ between the current worktree and <commit-ish>; everything else (including untracked files such as build outputs) keeps sharing storage with the source until modified. A new reflink_file() helper in copy.c clones every regular file. There is no size cutoff. On Linux it uses the FICLONE ioctl. That is the primitive btrfs, XFS and bcachefs implement, and the one overlayfs forwards when the lower filesystem can clone. This path has been tested on btrfs. On macOS it uses clonefile(), the APFS primitive. The t2400 REFLINK prerequisite probes that with "cp -c", because BSD cp has no --reflink=always, so the macOS CI jobs exercise this path on APFS. On Windows it uses ReFS block cloning, FSCTL_DUPLICATE_EXTENTS_TO_FILE, as Git LFS does. The destination is sized to the source, extents are cloned in chunks of at most 1GiB, and the tail is rounded up to a cluster (64K, then 4K) and cloned; Windows ignores the range past EOF. The tail is not byte-copied, and small files are not skipped. That path is implemented and has not been run on ReFS. A volume that rejects the control code, including NTFS, fails the probe. Anywhere else there is no reflink primitive. Support is probed up front by cloning a one-byte file, and where that fails "--reflink" falls back to an ordinary checkout instead of byte-copying the source tree. The directory walk skips the new worktree itself when it lives inside the source one, and preserves symlinks and modes. Uncommitted changes in the source do not leak into the new worktree. After reflinking the source's files we copy its index and run "update-index --refresh" before "reset --hard": because every reflinked file is a fresh inode, the cached stat never matches, so the refresh is forced to re-validate by content rather than by stat. A tracked path is therefore only left sharing extents when its content equals the indexed object; any path whose content differs from <commit-ish> -- which includes every uncommitted modification in the source -- is rewritten by the reset. The new worktree is thus always a faithful checkout of <commit-ish> for tracked paths, and only untracked files are carried over. The behavior can be made the default with the worktree.reflink configuration ("true", "false" or "auto", the last suppressing the unsupported-filesystem warning), and turned off per-invocation with --no-reflink. A configured default degrades quietly in modes that cannot reflink (--orphan, --no-checkout) instead of erroring, so enabling it never breaks those commands. The checkout step continues to honor checkout.workers, so parallel checkout composes with --reflink for the paths that do need rewriting. To check out many worktrees at the same commit safely, give each its own ref (or a detached HEAD) rather than sharing one branch; a git-worktree(1) example shows the one-shot "update-ref" transaction that pairs with --reflink. Signed-off-by: Jason Newton <nevion@gmail.com> --- Documentation/config/worktree.adoc | 10 ++ Documentation/git-worktree.adoc | 49 ++++++- builtin/worktree.c | 227 ++++++++++++++++++++++++++++- copy.c | 190 ++++++++++++++++++++++++ copy.h | 14 ++ t/t2400-worktree-add.sh | 96 ++++++++++++ 6 files changed, 584 insertions(+), 2 deletions(-) diff --git a/Documentation/config/worktree.adoc b/Documentation/config/worktree.adoc index 0930183b91..aa849c08d2 100644 --- a/Documentation/config/worktree.adoc +++ b/Documentation/config/worktree.adoc @@ -18,3 +18,13 @@ Note that setting `worktree.useRelativePaths` to "`true`" implies enabling the `extensions.relativeWorktrees` config (see linkgit:git-config[1]), thus making it incompatible with older versions of Git. + +`worktree.reflink`:: + Controls whether `git worktree add` populates new worktrees with + copy-on-write (reflink) clones, as if `--reflink` had been given + (see linkgit:git-worktree[1]). May be set to "`true`", "`false`" + (the default), or "`auto`". With "`true`", a filesystem that does + not support reflinks produces a warning before falling back to an + ordinary checkout; with "`auto`", the fallback is silent. An + explicit `--reflink` or `--no-reflink` on the command line + overrides this setting. diff --git a/Documentation/git-worktree.adoc b/Documentation/git-worktree.adoc index e6e77252ab..4264ee753a 100644 --- a/Documentation/git-worktree.adoc +++ b/Documentation/git-worktree.adoc @@ -10,7 +10,7 @@ SYNOPSIS -------- [synopsis] git worktree add [-f] [--detach] [--checkout] [--lock [--reason <string>]] - [--orphan] [(-b | -B) <new-branch>] <path> [<commit-ish>] + [--orphan] [--reflink] [(-b | -B) <new-branch>] <path> [<commit-ish>] git worktree list [-v | --porcelain [-z]] git worktree lock [--reason <string>] <worktree> git worktree move <worktree> <new-path> @@ -213,6 +213,35 @@ To remove a locked worktree, specify `--force` twice. such as configuring sparse-checkout. See "Sparse checkout" in linkgit:git-read-tree[1]. +`--reflink`:: +`--no-reflink`:: + Populate the new worktree by creating copy-on-write (reflink) + clones of the current worktree's files and index instead of + writing every file out from the object store. The checkout that + follows then only has to rewrite the paths that actually differ + between the current worktree and _<commit-ish>_; everything else + (including untracked files such as build outputs) keeps sharing + storage with the source worktree until modified. This makes + `add` much faster and far cheaper on disk when creating many + worktrees from the same base. `--no-reflink` forces an ordinary + checkout, overriding the `worktree.reflink` configuration. ++ +Reflinks require a copy-on-write filesystem: for example btrfs, XFS, +bcachefs or ZFS on Linux (and overlayfs on top of one of these, as used by +containers), APFS on macOS, or ReFS on Windows. On a filesystem that does +not support reflinks, `--reflink` transparently falls back to an ordinary +checkout. Because the source worktree's untracked and ignored files are +cloned as well, only use `--reflink` when that is acceptable. ++ +This option cannot be combined with `--no-checkout` or `--orphan`. It can +be enabled by default with the `worktree.reflink` configuration; see +linkgit:git-config[1]. ++ +The checkout that populates a new worktree also honors the +`checkout.workers` configuration (see linkgit:git-config[1]), so setting it +parallelizes the file writes and can further speed up `add`, with or +without `--reflink`. + `--guess-remote`:: `--no-guess-remote`:: With `worktree add <path>`, without _<commit-ish>_, instead @@ -531,6 +560,24 @@ $ popd $ git worktree remove ../temp ------------ +To run several independent worktrees from the same starting commit -- for +example a pool of automated agents -- give each one its own branch rather than +sharing a single branch between them. A shared branch is unsafe here: if one +worktree commits and another then commits from its now-stale index, the second +commit silently drops the first one's changes. Create the branches in a single +ref transaction and check each out in its own (optionally reflinked) worktree: + +------------ +$ oid=$(git rev-parse HEAD) +$ printf 'create refs/heads/agent-%s %s\n' 1 "$oid" 2 "$oid" 3 "$oid" | + git update-ref --stdin +$ for n in 1 2 3; do git worktree add --reflink ../agent-$n agent-$n; done +------------ + +The single linkgit:git-update-ref[1] transaction is cheap and is especially +efficient with the `reftable` backend. If a worktree does not need to advance a +branch at all, use a detached `HEAD` (`git worktree add --detach`) instead. + CONFIGURATION ------------- diff --git a/builtin/worktree.c b/builtin/worktree.c index 77ecd0f71f..9d4b51305e 100644 --- a/builtin/worktree.c +++ b/builtin/worktree.c @@ -30,7 +30,7 @@ #define BUILTIN_WORKTREE_ADD_USAGE \ N_("git worktree add [-f] [--detach] [--checkout] [--lock [--reason <string>]]\n" \ - " [--orphan] [(-b | -B) <new-branch>] <path> [<commit-ish>]") + " [--orphan] [--reflink] [(-b | -B) <new-branch>] <path> [<commit-ish>]") #define BUILTIN_WORKTREE_LIST_USAGE \ N_("git worktree list [-v | --porcelain [-z]]") @@ -47,6 +47,11 @@ #define BUILTIN_WORKTREE_UNLOCK_USAGE \ N_("git worktree unlock <worktree>") +/* values for add_opts.reflink and the worktree.reflink config */ +#define REFLINK_OFF 0 +#define REFLINK_ON 1 /* warn and fall back if unsupported */ +#define REFLINK_AUTO 2 /* silently fall back if unsupported */ + #define WORKTREE_ADD_DWIM_ORPHAN_INFER_TEXT \ _("No possible source branch, inferring '--orphan'") @@ -123,6 +128,7 @@ struct add_opts { int checkout; int orphan; int relative_paths; + int reflink; const char *keep_locked; }; @@ -130,6 +136,7 @@ static int show_only; static int verbose; static int guess_remote; static int use_relative_paths; +static int reflink_config = REFLINK_OFF; static timestamp_t expire; static int git_worktree_config(const char *var, const char *value, @@ -141,6 +148,13 @@ static int git_worktree_config(const char *var, const char *value, } else if (!strcmp(var, "worktree.userelativepaths")) { use_relative_paths = git_config_bool(var, value); return 0; + } else if (!strcmp(var, "worktree.reflink")) { + if (value && !strcmp(value, "auto")) + reflink_config = REFLINK_AUTO; + else + reflink_config = git_config_bool(var, value) ? + REFLINK_ON : REFLINK_OFF; + return 0; } return git_default_config(var, value, ctx, cb); @@ -394,6 +408,182 @@ worktree_copy_cleanup: free(to_file); } +/* + * Probe whether the filesystem backing "dir" supports reflinks. We do this + * once up front so that, on filesystems without copy-on-write support (and on + * platforms that lack a reflink primitive entirely), we can + * fall back to a normal checkout instead of byte-copying the whole source + * working tree -- which would include untracked files and be slower than the + * checkout we are trying to avoid. + */ +static int reflink_supported(const char *dir) +{ + struct strbuf src = STRBUF_INIT, dst = STRBUF_INIT; + int fd, ok = 0; + + strbuf_addf(&src, "%s/.git-reflink-probe-src", dir); + strbuf_addf(&dst, "%s/.git-reflink-probe-dst", dir); + + fd = open(src.buf, O_WRONLY | O_CREAT | O_TRUNC, 0600); + if (fd >= 0) { + write_in_full(fd, "x", 1); + close(fd); + if (!reflink_file(the_repository, dst.buf, src.buf, 0600)) + ok = 1; + unlink(dst.buf); + } + unlink(src.buf); + + strbuf_release(&src); + strbuf_release(&dst); + return ok; +} + +/* + * Reflink a single regular file, falling back to a regular copy when the + * clone fails for this particular file (for example across mount points). + */ +static int reflink_or_copy(const char *dst, const char *src, int mode) +{ + if (!reflink_file(the_repository, dst, src, mode)) + return 0; + return copy_file(the_repository, dst, src, mode); +} + +/* + * Recursively copy the working-tree directory "src" into "dst" using reflinks + * for regular files. Directory entries that resolve to the destination + * worktree itself (identified by skip_dev/skip_ino, which matters when the new + * worktree lives inside the source one) and the top-level ".git" gitfile are + * skipped. + */ +static int reflink_tree(const char *src, const char *dst, + dev_t skip_dev, ino_t skip_ino, int top) +{ + struct strbuf s = STRBUF_INIT, d = STRBUF_INIT; + DIR *dir; + struct dirent *de; + int ret = 0; + + dir = opendir(src); + if (!dir) + return error_errno(_("could not open directory '%s'"), src); + + while (!ret && (de = readdir(dir))) { + struct stat st; + + if (is_dot_or_dotdot(de->d_name)) + continue; + if (top && !strcmp(de->d_name, ".git")) + continue; + + strbuf_reset(&s); + strbuf_addf(&s, "%s/%s", src, de->d_name); + strbuf_reset(&d); + strbuf_addf(&d, "%s/%s", dst, de->d_name); + + if (lstat(s.buf, &st)) { + ret = error_errno(_("could not stat '%s'"), s.buf); + break; + } + + if (S_ISDIR(st.st_mode)) { + /* never recurse into the new worktree itself */ + if (st.st_dev == skip_dev && st.st_ino == skip_ino) + continue; + if (mkdir(d.buf, st.st_mode & 07777) && errno != EEXIST) { + ret = error_errno(_("could not create directory '%s'"), d.buf); + break; + } + ret = reflink_tree(s.buf, d.buf, skip_dev, skip_ino, 0); + } else if (S_ISLNK(st.st_mode)) { + struct strbuf link = STRBUF_INIT; + + if (strbuf_readlink(&link, s.buf, st.st_size)) + ret = error_errno(_("could not read symlink '%s'"), s.buf); + else if (symlink(link.buf, d.buf)) + ret = error_errno(_("could not create symlink '%s'"), d.buf); + strbuf_release(&link); + } else if (S_ISREG(st.st_mode)) { + if (reflink_or_copy(d.buf, s.buf, st.st_mode)) + ret = error_errno(_("could not copy '%s' to '%s'"), + s.buf, d.buf); + } + /* silently skip fifos, sockets and device nodes */ + } + + closedir(dir); + strbuf_release(&s); + strbuf_release(&d); + return ret; +} + +/* + * Populate the new worktree at "path" by reflinking the current worktree's + * files and index. The subsequent "git reset --hard" then only has to rewrite + * the paths that actually differ between the source and <commit-ish>, leaving + * everything else sharing storage with the source. Returns 1 when reflinks are + * unavailable so the caller can fall back to a plain checkout. + */ +static int reflink_worktree(const char *path, const char *wt_git_dir, + const struct add_opts *opts, struct strvec *child_env) +{ + const char *src_wt = the_repository->worktree; + char *src_index = NULL, *dst_index = NULL; + struct stat dst_st; + int ret = 0; + + if (!src_wt) + return error(_("--reflink needs a source working tree, but this " + "repository does not have one")); + + if (!reflink_supported(path)) { + /* In auto mode the fallback is expected, so stay quiet. */ + if (!opts->quiet && opts->reflink != REFLINK_AUTO) + warning(_("the filesystem at '%s' does not support reflinks; " + "falling back to a regular checkout"), path); + return 1; + } + + if (stat(path, &dst_st)) + return error_errno(_("could not stat '%s'"), path); + + if ((ret = reflink_tree(src_wt, path, dst_st.st_dev, dst_st.st_ino, 1))) + return ret; + + /* + * Clone the source index so the following reset sees the source's + * state and only materializes the differences to <commit-ish>. + */ + src_index = repo_git_path(the_repository, "index"); + dst_index = xstrfmt("%s/index", wt_git_dir); + if (!access(src_index, F_OK) && + reflink_or_copy(dst_index, src_index, 0666)) { + ret = error_errno(_("could not copy index to '%s'"), dst_index); + goto out; + } + + /* + * The cloned index still carries the source files' stat information. + * Refresh it against the freshly reflinked files so that "git reset" + * recognizes unchanged paths as up to date and leaves them sharing + * storage instead of rewriting (and thus un-sharing) them. + */ + if (!access(dst_index, F_OK)) { + struct child_process cp = CHILD_PROCESS_INIT; + cp.git_cmd = 1; + strvec_pushl(&cp.args, "update-index", "-q", "--refresh", NULL); + strvec_pushv(&cp.env, child_env->v); + /* a dirty working tree is not an error here */ + run_command(&cp); + } + +out: + free(src_index); + free(dst_index); + return ret; +} + static int checkout_worktree(const struct add_opts *opts, struct strvec *child_env) { @@ -588,6 +778,20 @@ static int add_worktree(const char *path, const char *refname, (ret = make_worktree_orphan(refname, opts, &child_env))) goto done; + /* + * When --reflink is requested and the filesystem supports it, copy the + * current worktree (and its index) into the new one using copy-on-write + * clones. checkout_worktree() then only rewrites the paths that differ + * from <commit-ish>. reflink_worktree() returns 1 when reflinks are not + * available, in which case we just do an ordinary checkout below. + */ + if (opts->checkout && opts->reflink) { + ret = reflink_worktree(path, sb_repo.buf, opts, &child_env); + if (ret < 0) + goto done; + ret = 0; + } + if (opts->checkout && (ret = checkout_worktree(opts, &child_env))) goto done; @@ -832,6 +1036,7 @@ static int add(int ac, const char **av, const char *prefix, const char *lock_reason = NULL; int keep_locked = 0; int used_new_branch_options; + int reflink_cli = -1; struct option options[] = { OPT__FORCE(&opts.force, N_("checkout <branch> even if already checked out in other worktree"), @@ -854,6 +1059,8 @@ static int add(int ac, const char **av, const char *prefix, N_("try to match the new branch name with a remote-tracking branch")), OPT_BOOL(0, "relative-paths", &opts.relative_paths, N_("use relative paths for worktrees")), + OPT_BOOL(0, "reflink", &reflink_cli, + N_("populate the worktree using copy-on-write clones when supported")), OPT_END() }; int ret; @@ -873,6 +1080,24 @@ static int add(int ac, const char **av, const char *prefix, if (opts.orphan && !opts.checkout) die(_("options '%s' and '%s' cannot be used together"), "--orphan", "--no-checkout"); + + /* + * Resolve whether to reflink: an explicit --reflink/--no-reflink on + * the command line wins, otherwise fall back to the worktree.reflink + * configuration (which may select the "auto" mode). + */ + opts.reflink = (reflink_cli != -1) ? reflink_cli : reflink_config; + if (opts.reflink && (opts.orphan || !opts.checkout)) { + /* + * Reflinking is incompatible with these; only complain when it + * was explicitly requested, otherwise quietly do a plain + * checkout so a configured default does not break these modes. + */ + if (reflink_cli == REFLINK_ON) + die(_("options '%s' and '%s' cannot be used together"), + "--reflink", opts.orphan ? "--orphan" : "--no-checkout"); + opts.reflink = REFLINK_OFF; + } if (opts.orphan && ac == 2) die(_("option '%s' and commit-ish cannot be used together"), "--orphan"); diff --git a/copy.c b/copy.c index 6074132050..e6dfe33f6a 100644 --- a/copy.c +++ b/copy.c @@ -5,6 +5,38 @@ #include "strbuf.h" #include "abspath.h" +#if defined(__linux__) +#include <sys/ioctl.h> +/* + * FICLONE lives in <linux/fs.h>, but including that header tends to clash + * with the libc headers git already pulls in, so define it ourselves if it + * is missing. The value is part of the stable kernel uapi. + */ +#ifndef FICLONE +#define FICLONE _IOW(0x94, 9, int) +#endif +#elif defined(__APPLE__) +#include <sys/attr.h> +#include <sys/clonefile.h> +#elif defined(GIT_WINDOWS_NATIVE) +#include <winioctl.h> +/* + * Block cloning on ReFS uses FSCTL_DUPLICATE_EXTENTS_TO_FILE. Older Windows + * SDK headers may lack the control code and its payload struct, so define + * them ourselves when missing (the values are part of the stable Win32 API). + */ +#ifndef FSCTL_DUPLICATE_EXTENTS_TO_FILE +#define FSCTL_DUPLICATE_EXTENTS_TO_FILE \ + CTL_CODE(FILE_DEVICE_FILE_SYSTEM, 209, METHOD_BUFFERED, FILE_WRITE_DATA) +typedef struct _DUPLICATE_EXTENTS_DATA { + HANDLE FileHandle; + LARGE_INTEGER SourceFileOffset; + LARGE_INTEGER TargetFileOffset; + LARGE_INTEGER ByteCount; +} DUPLICATE_EXTENTS_DATA, *PDUPLICATE_EXTENTS_DATA; +#endif +#endif + int copy_fd(int ifd, int ofd) { while (1) { @@ -72,3 +104,161 @@ int copy_file_with_time(struct repository *repo, return copy_times(dst, src); return status; } + +#ifdef GIT_WINDOWS_NATIVE +/* + * Clone [offset, offset+count) of "src" onto "dst" via ReFS block cloning. + * "count" is rounded up to a cluster boundary by the caller; a range that + * runs past the source's end of file is harmless and ignored by Windows. + */ +static int clone_extent_windows(HANDLE dst, HANDLE src, + uint64_t offset, uint64_t count) +{ + DUPLICATE_EXTENTS_DATA dup; + DWORD bytes; + + dup.FileHandle = src; + dup.SourceFileOffset.QuadPart = offset; + dup.TargetFileOffset.QuadPart = offset; + dup.ByteCount.QuadPart = count; + if (!DeviceIoControl(dst, FSCTL_DUPLICATE_EXTENTS_TO_FILE, + &dup, sizeof(dup), NULL, 0, &bytes, NULL)) + return -1; + return 0; +} +#endif + +int reflink_file(struct repository *repo, + const char *dst, const char *src, int mode) +{ +#if defined(__APPLE__) + /* + * clonefile() refuses to operate when the destination exists and + * copies the source's permissions for us, so "mode" is unused here. + */ + (void)mode; + if (clonefile(src, dst, 0) < 0) + return -1; + if (adjust_shared_perm(repo, dst)) + return -1; + return 0; +#elif defined(__linux__) + int fdi, fdo, status; + + mode = (mode & 0111) ? 0777 : 0666; + if ((fdi = open(src, O_RDONLY)) < 0) + return -1; + if ((fdo = open(dst, O_WRONLY | O_CREAT | O_EXCL, mode)) < 0) { + int saved = errno; + close(fdi); + errno = saved; + return -1; + } + status = ioctl(fdo, FICLONE, fdi); + close(fdi); + if (status < 0) { + int saved = errno; + close(fdo); + /* we created an empty file above; do not leave it behind */ + unlink(dst); + errno = saved; + return -1; + } + if (close(fdo) != 0) + return -1; + if (adjust_shared_perm(repo, dst)) + return -1; + return 0; +#elif defined(GIT_WINDOWS_NATIVE) + /* + * ReFS supports copy-on-write block cloning via + * FSCTL_DUPLICATE_EXTENTS_TO_FILE, the same mechanism Git LFS uses. + * Cloned ranges must start and end on a cluster boundary (ReFS uses + * 64K or 4K clusters) and span less than 4GiB, so clone in <=1GiB + * chunks and round the final chunk up to a cluster; the destination is + * sized to the source's exact length and an over-long final range past + * the source's EOF is ignored. + */ + const uint64_t GiB = (uint64_t)1 << 30; + const uint64_t clusters[] = { 64 * 1024, 4 * 1024 }; + int fdi, fdo, ret = -1; + HANDLE hi, ho; + LARGE_INTEGER size; + uint64_t off; + size_t i; + + mode = (mode & 0111) ? 0777 : 0666; + if ((fdi = open(src, O_RDONLY | O_BINARY)) < 0) + return -1; + if ((fdo = open(dst, O_RDWR | O_CREAT | O_EXCL | O_BINARY, mode)) < 0) { + int saved = errno; + close(fdi); + errno = saved; + return -1; + } + hi = (HANDLE)_get_osfhandle(fdi); + ho = (HANDLE)_get_osfhandle(fdo); + if (hi == INVALID_HANDLE_VALUE || ho == INVALID_HANDLE_VALUE || + !GetFileSizeEx(hi, &size)) { + errno = EINVAL; + goto cleanup; + } + + /* the destination must be the source's exact size before cloning */ + if (!SetFilePointerEx(ho, size, NULL, FILE_BEGIN) || !SetEndOfFile(ho)) { + errno = EIO; + goto cleanup; + } + + ret = 0; + /* clone whole 1GiB regions (already cluster aligned) */ + for (off = 0; (uint64_t)size.QuadPart > off && + (uint64_t)size.QuadPart - off > GiB; off += GiB) { + if (clone_extent_windows(ho, hi, off, GiB)) { + errno = ENOSYS; + ret = -1; + goto cleanup; + } + } + /* clone the tail, rounding its length up to a cluster (64K then 4K) */ + if (!ret && (uint64_t)size.QuadPart > off) { + uint64_t rem = (uint64_t)size.QuadPart - off; + + ret = -1; + for (i = 0; i < ARRAY_SIZE(clusters); i++) { + uint64_t count = (rem + clusters[i] - 1) & + ~(clusters[i] - 1); + if (!clone_extent_windows(ho, hi, off, count)) { + ret = 0; + break; + } + } + if (ret) + errno = ENOSYS; /* not a CoW volume */ + } + +cleanup: + close(fdi); + if (close(fdo) && !ret) { + errno = EIO; + ret = -1; + } + if (ret) { + int saved = errno; + unlink(dst); + errno = saved; + return -1; + } + if (adjust_shared_perm(repo, dst)) + return -1; + return 0; +#else + /* No reflink primitive on this platform. */ + (void)repo; + (void)dst; + (void)src; + (void)mode; + errno = ENOSYS; + return -1; +#endif +} diff --git a/copy.h b/copy.h index 1059b118d6..b9168a24f1 100644 --- a/copy.h +++ b/copy.h @@ -11,4 +11,18 @@ int copy_file(struct repository *repo, int copy_file_with_time(struct repository *repo, const char *dst, const char *src, int mode); +/* + * Create "dst" as a copy-on-write (reflink) clone of the regular file + * "src", so that the two files share their data blocks until one of + * them is modified. "dst" must not already exist. + * + * This only succeeds on filesystems that support block cloning (e.g. + * btrfs, XFS or bcachefs on Linux, APFS on macOS). When the platform or + * filesystem does not support reflinks, -1 is returned with errno set to + * ENOSYS (or the underlying error). Callers are expected to fall back to + * copy_file() in that case. Returns 0 on success. + */ +int reflink_file(struct repository *repo, + const char *dst, const char *src, int mode); + #endif /* COPY_H */ diff --git a/t/t2400-worktree-add.sh b/t/t2400-worktree-add.sh index bdcca97633..4aa63c5d3f 100755 --- a/t/t2400-worktree-add.sh +++ b/t/t2400-worktree-add.sh @@ -1288,4 +1288,100 @@ test_expect_success 'relative worktree sets extension config' ' test_cmp_config -C repo true extensions.relativeworktrees ' +test_expect_success '--reflink produces a correct checkout on any filesystem' ' + test_when_finished "rm -rf reflinkrepo" && + git init reflinkrepo && + ( + cd reflinkrepo && + test_commit base file.txt base && + test_commit next file.txt next && + git worktree add --reflink wt -b reflinkbr base && + echo base >expect && + test_cmp expect wt/file.txt && + git -C wt status --porcelain >status && + test_must_be_empty status + ) +' + +test_expect_success '--reflink rejects incompatible options' ' + test_must_fail git worktree add --reflink --no-checkout wt-bad HEAD 2>err && + test_grep "cannot be used together" err && + test_must_fail git worktree add --reflink --orphan wt-bad2 2>err && + test_grep "cannot be used together" err +' + +test_expect_success 'worktree.reflink config drives a correct checkout' ' + test_when_finished "rm -rf reflinkcfg" && + git init reflinkcfg && + ( + cd reflinkcfg && + test_commit cfg file.txt value && + git -c worktree.reflink=true worktree add wt HEAD && + echo value >expect && + test_cmp expect wt/file.txt + ) +' + +test_expect_success 'worktree.reflink=auto does not break --orphan' ' + test_when_finished "rm -rf reflinkorphan" && + git init reflinkorphan && + ( + cd reflinkorphan && + test_commit base file.txt value && + git -c worktree.reflink=auto worktree add --orphan wt && + git -C wt symbolic-ref --short HEAD >actual && + echo wt >expect && + test_cmp expect actual + ) +' + +test_expect_success '--no-reflink overrides worktree.reflink' ' + test_when_finished "rm -rf reflinkoff" && + git init reflinkoff && + ( + cd reflinkoff && + test_commit base file.txt value && + git -c worktree.reflink=true worktree add --no-reflink wt HEAD && + echo value >expect && + test_cmp expect wt/file.txt + ) +' + +# Reflinks are only available on copy-on-write filesystems (btrfs, XFS, +# bcachefs, APFS, ...). Where they are unavailable, --reflink transparently +# falls back to a regular checkout, which the test above already covers. +# +# GNU cp probes with --reflink=always. macOS cp has no such option; -c +# is clonefile(2), which is what the APFS CI jobs exercise. +test_lazy_prereq REFLINK ' + echo probe >reflink-src && + if test_have_prereq MACOS + then + cp -c reflink-src reflink-dst 2>/dev/null + else + cp --reflink=always reflink-src reflink-dst 2>/dev/null + fi +' + +test_expect_success REFLINK '--reflink carries untracked files and reconciles changes' ' + test_when_finished "rm -rf cowrepo" && + git init cowrepo && + ( + cd cowrepo && + test_commit same same.txt content && + test_commit old change.txt old-value && + test_commit new change.txt new-value && + echo artifact >untracked.bin && + git worktree add --reflink wt -b cowbr old && + echo content >expect && + test_cmp expect wt/same.txt && + echo old-value >expect && + test_cmp expect wt/change.txt && + echo artifact >expect && + test_cmp expect wt/untracked.bin && + git -C wt status --porcelain >status && + test_grep "?? untracked.bin" status + ) +' + test_done -- gitgitgadget ^ permalink raw reply related [flat|nested] 21+ messages in thread
* [PATCH v3 2/4] worktree: reap ephemeral branches created by "worktree add" 2026-10-21 7:17 ` [PATCH v3 0/4] " Jason Newton via GitGitGadget 2026-10-21 7:17 ` [PATCH v3 1/4] worktree: add --reflink for copy-on-write worktree creation Jason Newton via GitGitGadget @ 2026-10-21 7:17 ` Jason Newton via GitGitGadget 2026-10-21 7:17 ` [PATCH v3 3/4] worktree: disable reflink on Windows Jason Newton via GitGitGadget 2026-10-21 7:17 ` [PATCH v3 4/4] worktree: keep the reflink probe referenced " Jason Newton via GitGitGadget 3 siblings, 0 replies; 21+ messages in thread From: Jason Newton via GitGitGadget @ 2026-10-21 7:17 UTC (permalink / raw) To: git; +Cc: brian m. carlson, Jason Newton, Jason Newton From: Jason Newton <nevion@gmail.com> When "git worktree add <path>" is given no branch to check out, it auto-creates one named after the path. Such a branch exists only to back the new worktree, yet it outlives it: removing the worktree leaves the branch behind, so a fleet of throwaway worktrees fills refs/heads/ with dead branches that have to be cleaned up by hand. Treat an auto-created branch as owned by the worktree. Record it in the worktree's administrative directory (worktrees/<id>/owned-branch) at creation time -- the one moment we can know for certain the branch is worktree-private, since we are the one that created it. "git worktree remove" then deletes the owned branch once the worktree is gone and it is no longer checked out. Ownership is recorded only when "worktree add" itself creates the branch via DWIM auto-naming; a branch named explicitly with -b/-B, or one that already existed, is never owned and never reaped. Deletion is unconditional (even for unmerged commits) because an ephemeral branch is disposable by contract: to keep the work, pass --keep-branch, or anchor the commits with an ordinary branch beforehand -- a normal branch can point at the same commits, after which reaping the ephemeral ref loses nothing. "git branch -m" updates the marker so ownership follows a rename, and the branch stays ephemeral. The marker is just another per-worktree administrative file, so older versions of Git simply ignore it (they keep the branch on removal, as today). Signed-off-by: Jason Newton <nevion@gmail.com> --- Documentation/git-worktree.adoc | 22 ++++++++-- builtin/branch.c | 28 +++++++++++++ builtin/worktree.c | 74 ++++++++++++++++++++++++++++++++- t/t2400-worktree-add.sh | 55 ++++++++++++++++++++++++ t/t3200-branch.sh | 2 +- t/t3400-rebase.sh | 2 +- t/t5516-fetch-push.sh | 2 +- 7 files changed, 178 insertions(+), 7 deletions(-) diff --git a/Documentation/git-worktree.adoc b/Documentation/git-worktree.adoc index 4264ee753a..99e6e55be4 100644 --- a/Documentation/git-worktree.adoc +++ b/Documentation/git-worktree.adoc @@ -15,7 +15,7 @@ git worktree list [-v | --porcelain [-z]] git worktree lock [--reason <string>] <worktree> git worktree move <worktree> <new-path> git worktree prune [-n] [-v] [--expire <expire>] -git worktree remove [-f] <worktree> +git worktree remove [-f] [--keep-branch] <worktree> git worktree repair [<path>...] git worktree unlock <worktree> @@ -40,8 +40,11 @@ In its simplest form, `git worktree add <path>` automatically creates a new branch whose name is the final component of _<path>_, which is convenient if you plan to work on a new topic. For instance, `git worktree add ../hotfix` creates new branch `hotfix` and checks it out at -path `../hotfix`. To instead work on an existing branch in a new worktree, -use `git worktree add <path> <branch>`. On the other hand, if you just +path `../hotfix`. Such an automatically-created branch is _ephemeral_: it +is owned by the worktree and is deleted again when the worktree is removed +with `git worktree remove` (see that command and `--keep-branch`). To +instead work on an existing branch in a new worktree, use +`git worktree add <path> <branch>`. On the other hand, if you just plan to make some experimental changes or do testing without disturbing existing development, it is often convenient to create a 'throwaway' worktree not associated with any branch. For instance, @@ -145,6 +148,14 @@ Remove a worktree. Only clean worktrees (no untracked files and no modification in tracked files) can be removed. Unclean worktrees or ones with submodules can be removed with `--force`. The main worktree cannot be removed. ++ +If the worktree owns an _ephemeral_ branch -- one that `git worktree add` +created itself because no branch was named (the auto-naming case above) -- +that branch is deleted along with the worktree, even if it has unmerged +commits, since it was created solely to back this worktree. To keep it, use +`--keep-branch`, or anchor the commits with an ordinary branch +(`git branch <name>`) beforehand. A branch you named explicitly (`-b`/`-B`) +or one that existed before the worktree is never deleted. `repair [<path>...]`:: @@ -192,6 +203,11 @@ allows the move to proceed; use `--force` twice if the destination is locked. `remove` refuses to remove an unclean worktree unless `--force` is used. To remove a locked worktree, specify `--force` twice. +`--keep-branch`:: + With `remove`, do not delete the worktree's ephemeral (owned) branch; + keep it as an ordinary branch. Has no effect on worktrees that do not + own a branch. + `-b <new-branch>`:: `-B <new-branch>`:: With `add`, create a new branch named _<new-branch>_ starting at diff --git a/builtin/branch.c b/builtin/branch.c index a613148fc7..ca2e25df03 100644 --- a/builtin/branch.c +++ b/builtin/branch.c @@ -607,6 +607,32 @@ static int replace_each_worktree_head_symref(struct worktree **worktrees, #define IS_HEAD 1 #define IS_ORPHAN 2 +/* + * When a branch that a worktree owns (its ephemeral branch, recorded in + * worktrees/<id>/owned-branch) is renamed, update the marker so ownership + * follows the new name -- it stays ephemeral. + */ +static void rename_owned_branch_markers(struct worktree **worktrees, + const char *oldref, const char *newref) +{ + for (int i = 0; worktrees[i]; i++) { + struct strbuf path = STRBUF_INIT, buf = STRBUF_INIT; + + if (!worktrees[i]->id) + continue; /* the main worktree keeps no marker */ + repo_common_path_append(the_repository, &path, + "worktrees/%s/owned-branch", + worktrees[i]->id); + if (strbuf_read_file(&buf, path.buf, 64) > 0) { + strbuf_trim(&buf); + if (!strcmp(buf.buf, oldref)) + write_file(path.buf, "%s", newref); + } + strbuf_release(&path); + strbuf_release(&buf); + } +} + static void copy_or_rename_branch(const char *oldname, const char *newname, int copy, int force) { struct strbuf oldref = STRBUF_INIT, newref = STRBUF_INIT, logmsg = STRBUF_INIT; @@ -675,6 +701,8 @@ static void copy_or_rename_branch(const char *oldname, const char *newname, int if (!copy && !(oldref_usage & IS_ORPHAN) && refs_rename_ref(get_main_ref_store(the_repository), oldref.buf, newref.buf, logmsg.buf)) die(_("branch rename failed")); + if (!copy && !(oldref_usage & IS_ORPHAN)) + rename_owned_branch_markers(worktrees, oldref.buf, newref.buf); if (copy && refs_copy_existing_ref(get_main_ref_store(the_repository), oldref.buf, newref.buf, logmsg.buf)) die(_("branch copy failed")); diff --git a/builtin/worktree.c b/builtin/worktree.c index 9d4b51305e..1059429cbb 100644 --- a/builtin/worktree.c +++ b/builtin/worktree.c @@ -41,7 +41,7 @@ #define BUILTIN_WORKTREE_PRUNE_USAGE \ N_("git worktree prune [-n] [-v] [--expire <expire>]") #define BUILTIN_WORKTREE_REMOVE_USAGE \ - N_("git worktree remove [-f] <worktree>") + N_("git worktree remove [-f] [--keep-branch] <worktree>") #define BUILTIN_WORKTREE_REPAIR_USAGE \ N_("git worktree repair [<path>...]") #define BUILTIN_WORKTREE_UNLOCK_USAGE \ @@ -130,8 +130,34 @@ struct add_opts { int relative_paths; int reflink; const char *keep_locked; + const char *owned_branch; }; +/* + * An "owned" (ephemeral) branch is one that "git worktree add" created itself + * because the user did not name a branch (the DWIM auto-name case). Its + * lifetime is bound to the worktree: it is recorded in the worktree's + * administrative directory at creation and reaped when the worktree is + * removed. A pre-existing branch, or one named explicitly with -b/-B, is never + * owned. + */ +static char *read_owned_branch(const char *id) +{ + struct strbuf path = STRBUF_INIT, buf = STRBUF_INIT; + char *ret = NULL; + + repo_common_path_append(the_repository, &path, + "worktrees/%s/owned-branch", id); + if (strbuf_read_file(&buf, path.buf, 64) > 0) { + strbuf_trim(&buf); + if (buf.len) + ret = strbuf_detach(&buf, NULL); + } + strbuf_release(&path); + strbuf_release(&buf); + return ret; +} + static int show_only; static int verbose; static int guess_remote; @@ -771,6 +797,16 @@ static int add_worktree(const char *path, const char *refname, if (the_repository->repository_format_worktree_config) copy_filtered_worktree_config(sb_repo.buf); + /* + * Record an owned (ephemeral) branch so that "git worktree remove" can + * reap it. Only DWIM-created branches are owned (see read_owned_branch()). + */ + if (opts->owned_branch) { + strbuf_reset(&sb); + strbuf_addf(&sb, "%s/owned-branch", sb_repo.buf); + write_file(sb.buf, "refs/heads/%s", opts->owned_branch); + } + strvec_pushf(&child_env, "%s=%s", GIT_DIR_ENVIRONMENT, sb_git.buf); strvec_pushf(&child_env, "%s=%s", GIT_WORK_TREE_ENVIRONMENT, path); @@ -1148,6 +1184,13 @@ static int add(int ac, const char **av, const char *prefix, branch = branch_to_free = s; new_branch = new_branch_to_free; + /* + * A branch auto-created from the path (the user named no + * branch) is owned by this worktree and reaped on removal. + */ + if (new_branch_to_free) + opts.owned_branch = new_branch_to_free; + /* DWIM: Infer --orphan when repo has no refs. */ opts.orphan = (!s) && dwim_orphan(&opts, !!opt_track, 1); } else if (ac == 2 && new_branch) { @@ -1651,15 +1694,19 @@ static int remove_worktree(int ac, const char **av, const char *prefix, struct repository *repo UNUSED) { int force = 0; + int keep_branch = 0; struct option options[] = { OPT__FORCE(&force, N_("force removal even if worktree is dirty or locked"), PARSE_OPT_NOCOMPLETE), + OPT_BOOL(0, "keep-branch", &keep_branch, + N_("keep the worktree's owned (ephemeral) branch")), OPT_END() }; struct worktree **worktrees, *wt; struct strbuf errmsg = STRBUF_INIT; const char *reason = NULL; + char *owned = NULL; int ret = 0; ac = parse_options(ac, av, prefix, options, git_worktree_remove_usage, 0); @@ -1685,6 +1732,10 @@ static int remove_worktree(int ac, const char **av, const char *prefix, errmsg.buf); strbuf_release(&errmsg); + /* read the owned-branch marker before the admin dir is deleted */ + if (!keep_branch) + owned = read_owned_branch(wt->id); + if (file_exists(wt->path)) { if (!force) check_clean_worktree(wt, av[0]); @@ -1698,6 +1749,27 @@ static int remove_worktree(int ac, const char **av, const char *prefix, ret |= delete_git_dir(wt->id); delete_worktrees_dir_if_empty(); + /* + * Reap the worktree's owned (ephemeral) branch now that the worktree + * is gone and the branch is no longer checked out. This is + * unconditional by design: the branch was created for this worktree + * and is disposable. To keep the work, anchor it with an ordinary + * branch (or pass --keep-branch) before removing the worktree. + */ + if (owned) { + struct ref_store *refs = get_main_ref_store(the_repository); + if (refs_ref_exists(refs, owned)) { + if (refs_delete_ref(refs, NULL, owned, NULL, 0)) + ret = error(_("failed to reap branch '%s'"), owned); + else { + const char *name = owned; + skip_prefix(owned, "refs/heads/", &name); + fprintf_ln(stderr, _("Removed branch '%s'"), name); + } + } + free(owned); + } + free_worktrees(worktrees); return ret; } diff --git a/t/t2400-worktree-add.sh b/t/t2400-worktree-add.sh index 4aa63c5d3f..14bf455736 100755 --- a/t/t2400-worktree-add.sh +++ b/t/t2400-worktree-add.sh @@ -1384,4 +1384,59 @@ test_expect_success REFLINK '--reflink carries untracked files and reconciles ch ) ' +test_expect_success 'ephemeral: DWIM-created branch is owned and reaped on remove' ' + test_when_finished "git worktree remove -f eph || :; git branch -D eph 2>/dev/null || :" && + git worktree add eph && + echo refs/heads/eph >expect && + test_cmp expect .git/worktrees/eph/owned-branch && + git rev-parse --verify refs/heads/eph && + git worktree remove eph && + test_must_fail git rev-parse --verify --quiet refs/heads/eph +' + +test_expect_success 'ephemeral: -b branch is not owned and is kept on remove' ' + test_when_finished "git worktree remove -f nb || :; git branch -D named 2>/dev/null || :" && + git worktree add -b named nb && + test_path_is_missing .git/worktrees/nb/owned-branch && + git worktree remove nb && + git rev-parse --verify refs/heads/named +' + +test_expect_success 'ephemeral: pre-existing branch is not owned' ' + test_when_finished "git worktree remove -f pe || :; git branch -D pre 2>/dev/null || :" && + git branch pre && + git worktree add pe pre && + test_path_is_missing .git/worktrees/pe/owned-branch && + git worktree remove pe && + git rev-parse --verify refs/heads/pre +' + +test_expect_success 'ephemeral: --keep-branch keeps an owned branch' ' + test_when_finished "git branch -D keepme 2>/dev/null || :" && + git worktree add keepme && + git worktree remove --keep-branch keepme && + git rev-parse --verify refs/heads/keepme +' + +test_expect_success 'ephemeral: branch -m follows the marker and stays ephemeral' ' + test_when_finished "git worktree remove -f mv || :; git branch -D mv2 2>/dev/null || :" && + git worktree add mv && + git -C mv branch -m mv mv2 && + echo refs/heads/mv2 >expect && + test_cmp expect .git/worktrees/mv/owned-branch && + git worktree remove mv && + test_must_fail git rev-parse --verify --quiet refs/heads/mv2 +' + +test_expect_success 'ephemeral: commits survive the reap when anchored by a normal branch' ' + test_when_finished "git worktree remove -f anc || :; git branch -D anchored 2>/dev/null || :" && + git worktree add anc && + test_commit -C anc agentwork && + tip=$(git -C anc rev-parse HEAD) && + git branch anchored "$tip" && + git worktree remove anc && + test_must_fail git rev-parse --verify --quiet refs/heads/anc && + test "$(git rev-parse anchored)" = "$tip" +' + test_done diff --git a/t/t3200-branch.sh b/t/t3200-branch.sh index cdb6c6a634..67c7c67853 100755 --- a/t/t3200-branch.sh +++ b/t/t3200-branch.sh @@ -190,7 +190,7 @@ test_expect_success 'git branch -M foo bar should fail when bar is checked out' test_expect_success 'git branch -M foo bar should fail when bar is checked out in worktree' ' git branch -f bar && - test_when_finished "git worktree remove wt && git branch -D wt" && + test_when_finished "git worktree remove --keep-branch wt && git branch -D wt" && git worktree add wt && test_must_fail git branch -M bar wt ' diff --git a/t/t3400-rebase.sh b/t/t3400-rebase.sh index accaaba4d4..9c0c80f113 100755 --- a/t/t3400-rebase.sh +++ b/t/t3400-rebase.sh @@ -471,7 +471,7 @@ test_expect_success 'git rebase --update-ref with core.commentChar and branch on test_when_finished git branch -D base topic2 && test_when_finished git checkout main && test_when_finished git branch -D wt-topic && - test_when_finished git worktree remove wt-topic && + test_when_finished git worktree remove --keep-branch wt-topic && git checkout main && git checkout -b base && git checkout -b topic2 && diff --git a/t/t5516-fetch-push.sh b/t/t5516-fetch-push.sh index b982b209bf..6ffeb064fa 100755 --- a/t/t5516-fetch-push.sh +++ b/t/t5516-fetch-push.sh @@ -1873,7 +1873,7 @@ test_expect_success 'updateInstead with bare repository worktree and unborn bare ' test_expect_success 'refuse fetch to current branch of worktree' ' - test_when_finished "git worktree remove --force wt && git branch -D wt" && + test_when_finished "git worktree remove --force --keep-branch wt && git branch -D wt" && git worktree add wt && test_commit apple && test_must_fail git fetch . HEAD:wt && -- gitgitgadget ^ permalink raw reply related [flat|nested] 21+ messages in thread
* [PATCH v3 3/4] worktree: disable reflink on Windows 2026-10-21 7:17 ` [PATCH v3 0/4] " Jason Newton via GitGitGadget 2026-10-21 7:17 ` [PATCH v3 1/4] worktree: add --reflink for copy-on-write worktree creation Jason Newton via GitGitGadget 2026-10-21 7:17 ` [PATCH v3 2/4] worktree: reap ephemeral branches created by "worktree add" Jason Newton via GitGitGadget @ 2026-10-21 7:17 ` Jason Newton via GitGitGadget 2026-10-21 7:17 ` [PATCH v3 4/4] worktree: keep the reflink probe referenced " Jason Newton via GitGitGadget 3 siblings, 0 replies; 21+ messages in thread From: Jason Newton via GitGitGadget @ 2026-10-21 7:17 UTC (permalink / raw) To: git; +Cc: brian m. carlson, Jason Newton, Jason Newton From: Jason Newton <nevion@gmail.com> FSCTL_DUPLICATE_EXTENTS_TO_FILE can publish zeros when the source data is still cached. git-lfs issue 6312 reports that on Windows 11 build 26200, and a later reproduction hit build 26200.9448. No fixed Windows build is identified, so reflink_file() refuses on every Windows version. git worktree add --reflink and worktree.reflink=true warn with that build number and fall back to a regular checkout. worktree.reflink=auto stays silent, as it does when the filesystem cannot reflink. Signed-off-by: Jason Newton <nevion@gmail.com> --- Documentation/config/worktree.adoc | 6 +- Documentation/git-worktree.adoc | 12 ++- builtin/worktree.c | 13 +++ copy.c | 133 +++-------------------------- t/t2400-worktree-add.sh | 15 ++++ 5 files changed, 53 insertions(+), 126 deletions(-) diff --git a/Documentation/config/worktree.adoc b/Documentation/config/worktree.adoc index aa849c08d2..e221efc615 100644 --- a/Documentation/config/worktree.adoc +++ b/Documentation/config/worktree.adoc @@ -25,6 +25,8 @@ thus making it incompatible with older versions of Git. (see linkgit:git-worktree[1]). May be set to "`true`", "`false`" (the default), or "`auto`". With "`true`", a filesystem that does not support reflinks produces a warning before falling back to an - ordinary checkout; with "`auto`", the fallback is silent. An - explicit `--reflink` or `--no-reflink` on the command line + ordinary checkout; with "`auto`", the fallback is silent. On Windows + the request always falls back. An explicit request warns that block + cloning can zero-fill files on Windows build 26200; "`auto`" stays + silent. An explicit `--reflink` or `--no-reflink` on the command line overrides this setting. diff --git a/Documentation/git-worktree.adoc b/Documentation/git-worktree.adoc index 99e6e55be4..6cc7c1f27a 100644 --- a/Documentation/git-worktree.adoc +++ b/Documentation/git-worktree.adoc @@ -244,10 +244,14 @@ To remove a locked worktree, specify `--force` twice. + Reflinks require a copy-on-write filesystem: for example btrfs, XFS, bcachefs or ZFS on Linux (and overlayfs on top of one of these, as used by -containers), APFS on macOS, or ReFS on Windows. On a filesystem that does -not support reflinks, `--reflink` transparently falls back to an ordinary -checkout. Because the source worktree's untracked and ignored files are -cloned as well, only use `--reflink` when that is acceptable. +containers), or APFS on macOS. Reflink is disabled on Windows. ReFS block +cloning can zero-fill files on Windows build 26200 (git-lfs issue 6312), +and no fixed build is known. On Windows, `--reflink` falls back to an +ordinary checkout and, unless `worktree.reflink` is `auto`, warns with +that build number. On a filesystem that does not support reflinks, +`--reflink` transparently falls back to an ordinary checkout. Because the +source worktree's untracked and ignored files are cloned as well, only use +`--reflink` when that is acceptable. + This option cannot be combined with `--no-checkout` or `--orphan`. It can be enabled by default with the `worktree.reflink` configuration; see diff --git a/builtin/worktree.c b/builtin/worktree.c index 1059429cbb..93fcaa5ecd 100644 --- a/builtin/worktree.c +++ b/builtin/worktree.c @@ -563,6 +563,18 @@ static int reflink_worktree(const char *path, const char *wt_git_dir, return error(_("--reflink needs a source working tree, but this " "repository does not have one")); +#ifdef GIT_WINDOWS_NATIVE + /* + * reflink_file() refuses on Windows. Build 26200 is the version + * named in git-lfs issue 6312; no later build is known to be fixed. + * Auto mode stays quiet, matching the non-Windows fallback. + */ + if (!opts->quiet && opts->reflink != REFLINK_AUTO) + warning(_("reflink is disabled on Windows; block cloning " + "can zero-fill files on Windows build 26200; " + "falling back to a regular checkout")); + return 1; +#else if (!reflink_supported(path)) { /* In auto mode the fallback is expected, so stay quiet. */ if (!opts->quiet && opts->reflink != REFLINK_AUTO) @@ -570,6 +582,7 @@ static int reflink_worktree(const char *path, const char *wt_git_dir, "falling back to a regular checkout"), path); return 1; } +#endif if (stat(path, &dst_st)) return error_errno(_("could not stat '%s'"), path); diff --git a/copy.c b/copy.c index e6dfe33f6a..2eb4c7477e 100644 --- a/copy.c +++ b/copy.c @@ -18,23 +18,6 @@ #elif defined(__APPLE__) #include <sys/attr.h> #include <sys/clonefile.h> -#elif defined(GIT_WINDOWS_NATIVE) -#include <winioctl.h> -/* - * Block cloning on ReFS uses FSCTL_DUPLICATE_EXTENTS_TO_FILE. Older Windows - * SDK headers may lack the control code and its payload struct, so define - * them ourselves when missing (the values are part of the stable Win32 API). - */ -#ifndef FSCTL_DUPLICATE_EXTENTS_TO_FILE -#define FSCTL_DUPLICATE_EXTENTS_TO_FILE \ - CTL_CODE(FILE_DEVICE_FILE_SYSTEM, 209, METHOD_BUFFERED, FILE_WRITE_DATA) -typedef struct _DUPLICATE_EXTENTS_DATA { - HANDLE FileHandle; - LARGE_INTEGER SourceFileOffset; - LARGE_INTEGER TargetFileOffset; - LARGE_INTEGER ByteCount; -} DUPLICATE_EXTENTS_DATA, *PDUPLICATE_EXTENTS_DATA; -#endif #endif int copy_fd(int ifd, int ofd) @@ -105,29 +88,6 @@ int copy_file_with_time(struct repository *repo, return status; } -#ifdef GIT_WINDOWS_NATIVE -/* - * Clone [offset, offset+count) of "src" onto "dst" via ReFS block cloning. - * "count" is rounded up to a cluster boundary by the caller; a range that - * runs past the source's end of file is harmless and ignored by Windows. - */ -static int clone_extent_windows(HANDLE dst, HANDLE src, - uint64_t offset, uint64_t count) -{ - DUPLICATE_EXTENTS_DATA dup; - DWORD bytes; - - dup.FileHandle = src; - dup.SourceFileOffset.QuadPart = offset; - dup.TargetFileOffset.QuadPart = offset; - dup.ByteCount.QuadPart = count; - if (!DeviceIoControl(dst, FSCTL_DUPLICATE_EXTENTS_TO_FILE, - &dup, sizeof(dup), NULL, 0, &bytes, NULL)) - return -1; - return 0; -} -#endif - int reflink_file(struct repository *repo, const char *dst, const char *src, int mode) { @@ -171,87 +131,20 @@ int reflink_file(struct repository *repo, return 0; #elif defined(GIT_WINDOWS_NATIVE) /* - * ReFS supports copy-on-write block cloning via - * FSCTL_DUPLICATE_EXTENTS_TO_FILE, the same mechanism Git LFS uses. - * Cloned ranges must start and end on a cluster boundary (ReFS uses - * 64K or 4K clusters) and span less than 4GiB, so clone in <=1GiB - * chunks and round the final chunk up to a cluster; the destination is - * sized to the source's exact length and an over-long final range past - * the source's EOF is ignored. + * ReFS block cloning (FSCTL_DUPLICATE_EXTENTS_TO_FILE) is disabled. + * On Windows 11 build 26200 the ioctl remaps on-disk clusters, so a + * clone of data still sitting in the cache captures unwritten zeros. + * The later flush of the source does not update the clone, and the + * zeros are permanent. That is git-lfs issue 6312, reproduced on + * build 26200.9448. No fixed Windows build is identified, so refuse + * on every Windows version and let the caller fall back to a copy. */ - const uint64_t GiB = (uint64_t)1 << 30; - const uint64_t clusters[] = { 64 * 1024, 4 * 1024 }; - int fdi, fdo, ret = -1; - HANDLE hi, ho; - LARGE_INTEGER size; - uint64_t off; - size_t i; - - mode = (mode & 0111) ? 0777 : 0666; - if ((fdi = open(src, O_RDONLY | O_BINARY)) < 0) - return -1; - if ((fdo = open(dst, O_RDWR | O_CREAT | O_EXCL | O_BINARY, mode)) < 0) { - int saved = errno; - close(fdi); - errno = saved; - return -1; - } - hi = (HANDLE)_get_osfhandle(fdi); - ho = (HANDLE)_get_osfhandle(fdo); - if (hi == INVALID_HANDLE_VALUE || ho == INVALID_HANDLE_VALUE || - !GetFileSizeEx(hi, &size)) { - errno = EINVAL; - goto cleanup; - } - - /* the destination must be the source's exact size before cloning */ - if (!SetFilePointerEx(ho, size, NULL, FILE_BEGIN) || !SetEndOfFile(ho)) { - errno = EIO; - goto cleanup; - } - - ret = 0; - /* clone whole 1GiB regions (already cluster aligned) */ - for (off = 0; (uint64_t)size.QuadPart > off && - (uint64_t)size.QuadPart - off > GiB; off += GiB) { - if (clone_extent_windows(ho, hi, off, GiB)) { - errno = ENOSYS; - ret = -1; - goto cleanup; - } - } - /* clone the tail, rounding its length up to a cluster (64K then 4K) */ - if (!ret && (uint64_t)size.QuadPart > off) { - uint64_t rem = (uint64_t)size.QuadPart - off; - - ret = -1; - for (i = 0; i < ARRAY_SIZE(clusters); i++) { - uint64_t count = (rem + clusters[i] - 1) & - ~(clusters[i] - 1); - if (!clone_extent_windows(ho, hi, off, count)) { - ret = 0; - break; - } - } - if (ret) - errno = ENOSYS; /* not a CoW volume */ - } - -cleanup: - close(fdi); - if (close(fdo) && !ret) { - errno = EIO; - ret = -1; - } - if (ret) { - int saved = errno; - unlink(dst); - errno = saved; - return -1; - } - if (adjust_shared_perm(repo, dst)) - return -1; - return 0; + (void)repo; + (void)dst; + (void)src; + (void)mode; + errno = ENOSYS; + return -1; #else /* No reflink primitive on this platform. */ (void)repo; diff --git a/t/t2400-worktree-add.sh b/t/t2400-worktree-add.sh index 14bf455736..592f414955 100755 --- a/t/t2400-worktree-add.sh +++ b/t/t2400-worktree-add.sh @@ -1288,6 +1288,21 @@ test_expect_success 'relative worktree sets extension config' ' test_cmp_config -C repo true extensions.relativeworktrees ' +test_expect_success MINGW 'reflink is disabled on Windows build 26200' ' + test_when_finished "rm -rf winreflink" && + git init winreflink && + ( + cd winreflink && + test_commit base file.txt base && + git worktree add --reflink wt -b winbr base 2>err && + test_grep "build 26200" err && + echo base >expect && + test_cmp expect wt/file.txt && + git -C wt status --porcelain >status && + test_must_be_empty status + ) +' + test_expect_success '--reflink produces a correct checkout on any filesystem' ' test_when_finished "rm -rf reflinkrepo" && git init reflinkrepo && -- gitgitgadget ^ permalink raw reply related [flat|nested] 21+ messages in thread
* [PATCH v3 4/4] worktree: keep the reflink probe referenced on Windows 2026-10-21 7:17 ` [PATCH v3 0/4] " Jason Newton via GitGitGadget ` (2 preceding siblings ...) 2026-10-21 7:17 ` [PATCH v3 3/4] worktree: disable reflink on Windows Jason Newton via GitGitGadget @ 2026-10-21 7:17 ` Jason Newton via GitGitGadget 3 siblings, 0 replies; 21+ messages in thread From: Jason Newton via GitGitGadget @ 2026-10-21 7:17 UTC (permalink / raw) To: git; +Cc: brian m. carlson, Jason Newton, Jason Newton From: Jason Newton <nevion@gmail.com> The mingw build compiles with -Werror=unused-function. Putting the reflink_supported() call in the non-Windows branch left that function unused, so "win build" failed at builtin/worktree.c:445. Leave the call in the common path. The Windows return still happens first. Signed-off-by: Jason Newton <nevion@gmail.com> --- builtin/worktree.c | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/builtin/worktree.c b/builtin/worktree.c index 93fcaa5ecd..07adf6bb7f 100644 --- a/builtin/worktree.c +++ b/builtin/worktree.c @@ -568,13 +568,15 @@ static int reflink_worktree(const char *path, const char *wt_git_dir, * reflink_file() refuses on Windows. Build 26200 is the version * named in git-lfs issue 6312; no later build is known to be fixed. * Auto mode stays quiet, matching the non-Windows fallback. + * Return before the probe, but leave the call compiled: an #else + * around it makes reflink_supported() unused under -Werror. */ if (!opts->quiet && opts->reflink != REFLINK_AUTO) warning(_("reflink is disabled on Windows; block cloning " "can zero-fill files on Windows build 26200; " "falling back to a regular checkout")); return 1; -#else +#endif if (!reflink_supported(path)) { /* In auto mode the fallback is expected, so stay quiet. */ if (!opts->quiet && opts->reflink != REFLINK_AUTO) @@ -582,7 +584,6 @@ static int reflink_worktree(const char *path, const char *wt_git_dir, "falling back to a regular checkout"), path); return 1; } -#endif if (stat(path, &dst_st)) return error_errno(_("could not stat '%s'"), path); -- gitgitgadget ^ permalink raw reply related [flat|nested] 21+ messages in thread
end of thread, other threads:[~2026-10-21 14:03 UTC | newest] Thread overview: 21+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2026-06-05 18:49 [PATCH 0/2] worktree: copy-on-write creation and shared-branch worktrees Jason Newton via GitGitGadget 2026-06-05 18:49 ` [PATCH 1/2] worktree: add --reflink for copy-on-write worktree creation Jason Newton via GitGitGadget 2026-06-05 18:49 ` [PATCH 2/2] worktree: allow sharing a checked-out branch across worktrees Jason Newton via GitGitGadget 2026-06-05 19:59 ` [PATCH 0/2] worktree: copy-on-write creation and shared-branch worktrees brian m. carlson 2026-06-08 14:36 ` Junio C Hamano 2026-10-20 16:04 ` [PATCH v2 0/2] worktree: copy-on-write creation and ephemeral branches Jason Newton via GitGitGadget 2026-10-20 16:04 ` [PATCH v2 1/2] worktree: add --reflink for copy-on-write worktree creation Jason Newton via GitGitGadget 2026-10-20 21:20 ` brian m. carlson 2026-10-21 7:15 ` Jason Newton 2026-10-20 16:04 ` [PATCH v2 2/2] worktree: reap ephemeral branches created by "worktree add" Jason Newton via GitGitGadget 2026-10-21 6:52 ` Eric Sunshine 2026-10-21 7:40 ` Jason Newton 2026-10-21 8:06 ` Eric Sunshine 2026-10-21 8:28 ` Jason Newton 2026-10-21 14:03 ` Phillip Wood 2026-10-21 7:08 ` [PATCH v2 0/2] worktree: copy-on-write creation and ephemeral branches Eric Sunshine 2026-10-21 7:17 ` [PATCH v3 0/4] " Jason Newton via GitGitGadget 2026-10-21 7:17 ` [PATCH v3 1/4] worktree: add --reflink for copy-on-write worktree creation Jason Newton via GitGitGadget 2026-10-21 7:17 ` [PATCH v3 2/4] worktree: reap ephemeral branches created by "worktree add" Jason Newton via GitGitGadget 2026-10-21 7:17 ` [PATCH v3 3/4] worktree: disable reflink on Windows Jason Newton via GitGitGadget 2026-10-21 7:17 ` [PATCH v3 4/4] worktree: keep the reflink probe referenced " Jason Newton via GitGitGadget
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox