* Re: [ANNOUNCE] Git for Windows 2.10.0
From: Johannes Schindelin @ 2016-09-08 18:19 UTC (permalink / raw)
To: stefan.naewe; +Cc: git-for-windows, git
In-Reply-To: <3ed6f32f-6330-3453-1581-3a2d4008bbfa@atlas-elektronik.com>
[-- Attachment #1: Type: text/plain, Size: 522 bytes --]
Hi Stefan,
On Thu, 8 Sep 2016, stefan.naewe@atlas-elektronik.com wrote:
> Am 03.09.2016 um 15:17 schrieb Johannes Schindelin:
>
> > New Features
> >
> > • Comes with Git v2.10.0.
> > • The git rebase -i command was made faster by reimplementing large
> > parts in C.
>
> I finally had the chance to do a "bigger" rebase and what shall I say...
> F***k, has this thing become fast, or what!
>
> Thank you so much for doing this!!!!
I *love* that kind of feedback! *beams*
Ciao,
Dscho
^ permalink raw reply
* Re: [PATCH] gpg-interface: reflect stderr to stderr
From: Junio C Hamano @ 2016-09-08 18:20 UTC (permalink / raw)
To: Jeff King; +Cc: Michael J Gruber, Johannes Schindelin, git
In-Reply-To: <20160907083947.b7q7ebe62xsr6447@sigill.intra.peff.net>
Jeff King <peff@peff.net> writes:
> On Wed, Sep 07, 2016 at 10:27:34AM +0200, Michael J Gruber wrote:
>
>> Now, I can't reproduce C on Linux[*], so there is more involved. It
>> could be that my patch just exposes a problem in our start_command()
>> etc.: run-command.c contains a lot of ifdefing, so possibly quite
>> different code is run on different platforms.
>
> Maybe, though my blind guess is that it is simply that on Linux we can
> open /dev/tty directly, and console-IO on Windows is a bit more
> complicated.
True.
Even though this patch is fixing only one of the two issues, I am
tempted to say that we should queue it for now, as it does so
without breaking a bigger gain made by the original, i.e. we learn
the status of verification in a way the authors of GPG wants us to,
while somebody figuires out what the best way is to show the prompt
to the console on Windows.
^ permalink raw reply
* [PATCH v7 00/10] Git filter protocol
From: larsxschneider @ 2016-09-08 18:21 UTC (permalink / raw)
To: git
Cc: peff, gitster, sbeller, Johannes.Schindelin, jnareb, mlbright,
tboegi, jacob.keller, Lars Schneider
From: Lars Schneider <larsxschneider@gmail.com>
The goal of this series is to avoid launching a new clean/smudge filter
process for each file that is filtered.
A short summary about v1 to v5 can be found here:
https://git.github.io/rev_news/2016/08/17/edition-18/
This series is also published on web:
https://github.com/larsxschneider/git/pull/11
Thanks a lot to
Stefan, Junio, Jakub (Narebski and Keller), Peff, and Torsten
for very helpful reviews,
Lars
## Major changes since v6
* series rebased to Git v2.10.0
* simplified and improved multi-packet test
* v2 filter takes precedence over v1 filter
* rename "error-all" to "abort"
* patch 07/13 "pack-protocol: fix maximum pkt-line size" submitted separately:
http://public-inbox.org/git/20160829175509.69050-1-larsxschneider@gmail.com/
* removed "10/13 convert: generate large test files only once"
* patch 13/13 "read-cache: make sure file handles are not inherited by child
processes" moved to an own series:
http://public-inbox.org/git/20160905211111.72956-1-larsxschneider@gmail.com/
## All changes since v6
### Stefan
* http://public-inbox.org/git/CAGZ79kaJn-yf28+Ls2JyqSs6Jt-Oj2JW-sAXQn-Oe5xaxSyjMQ@mail.gmail.com/
* submit patch "pack-protocol: fix maximum pkt-line size" separately:
http://public-inbox.org/git/20160829175509.69050-1-larsxschneider@gmail.com/
* http://public-inbox.org/git/CAGZ79kbEoYVM_+MWkajT2pHN1OKZXATe5d+Dv_uSJ7dyPGxUeg@mail.gmail.com/
* trace failures in new *_gently() pkt-line functions
### Junio
* http://public-inbox.org/git/xmqq8tvkle6o.fsf@gitster.mtv.corp.google.com/
* share code between packet_write_fmt() and packet_write_fmt_gently()
* http://public-inbox.org/git/xmqq4m68ldrg.fsf@gitster.mtv.corp.google.com/
* remove "Second, it will..." from "pkt-line: add packet_write_gently()"
commit message
* use separate buffer in write_packetized_from_fd()
* fix write order in packet_write_gently()
* http://public-inbox.org/git/xmqqzio0jxh7.fsf@gitster.mtv.corp.google.com/
* rename read/write flush terminated packet streams functions
* remove meaningless "bytes_to_write > ..." check
* reuse packet_read() function in read_packetized_to_buf()
* http://public-inbox.org/git/xmqqk2f3fgbd.fsf@gitster.mtv.corp.google.com/
* revert test_config replacement in test setup
* http://public-inbox.org/git/xmqq37lncvj6.fsf@gitster.mtv.corp.google.com/
* improve multi-packet test
* http://public-inbox.org/git/xmqqzinv6wtg.fsf@gitster.mtv.corp.google.com/
* give v2 filter precedence over v1 filter
* http://public-inbox.org/git/xmqqmvjt3nht.fsf@gitster.mtv.corp.google.com/
* explain that "error" behavior mimics v1 filter
* explain that Git closes the pipe on exit to allow the filter a graceful
exit
### Jakub
* http://public-inbox.org/git/7fbd02a1-ad62-7391-df2a-835aae8a62a7@gmail.com/
* rename "error-all" to "abort"
* add a simple Perl v2 filter example in contrib
## Interdiff (v6..v7)
diff --git a/Documentation/gitattributes.txt b/Documentation/gitattributes.txt
index df059e9..ac000ea 100644
--- a/Documentation/gitattributes.txt
+++ b/Documentation/gitattributes.txt
@@ -470,20 +470,11 @@ packet: git< status=error\n
packet: git< 0000
------------------------
-In case the filter cannot or does not want to process the content
-as well as any future content for the lifetime of the Git process,
-it is expected to respond with an "error-all" status. Depending on
-the `filter.<driver>.required` flag Git will interpret that as error
-but it will not stop or restart the filter process.
-------------------------
-packet: git< status=error-all\n
-packet: git< 0000
-------------------------
-
If the filter experiences an error during processing, then it can
-send the status "error". Depending on the `filter.<driver>.required`
-flag Git will interpret that as error but it will not stop or restart
-the filter process.
+send the status "error" after the content was (partially or
+completely) sent. Depending on the `filter.<driver>.required` flag
+Git will interpret that as error but it will not stop or restart the
+filter process.
------------------------
packet: git< status=success\n
packet: git< 0000
@@ -495,21 +486,38 @@ packet: git< 0000
If the filter dies during the communication or does not adhere to
the protocol then Git will stop the filter process and restart it
-with the next file that needs to be processed.
+with the next file that needs to be processed. Depending on the
+`filter.<driver>.required` flag Git will interpret that as error.
+
+The error handling for all cases above mimic the behavior of
+the `filter.<driver>.clean` / `filter.<driver>.smudge` error
+handling.
+
+In case the filter cannot or does not want to process the content
+as well as any future content for the lifetime of the Git process,
+it is expected to respond with an "abort" status. Depending on
+the `filter.<driver>.required` flag Git will interpret that as error
+for the content as well as any future content for the lifetime of the
+Git process but it will not stop or restart the filter process.
+------------------------
+packet: git< status=abort\n
+packet: git< 0000
+------------------------
After the filter has processed a blob it is expected to wait for
-the next "key=value" list containing a command. When the Git process
-terminates, it will send a kill signal to the filter in that stage.
+the next "key=value" list containing a command. Git will close
+the command pipe on exit. The filter is expected to detect EOF
+and exit gracefully on its own.
A long running filter demo implementation can be found in
-`t/t0021/rot13-filter.pl` located in the Git core repository.
-If you develop your own long running filter process then the
-`GIT_TRACE_PACKET` environment variables can be very helpful
-for debugging (see linkgit:git[1]).
-
-If a `filter.<driver>.clean` or `filter.<driver>.smudge` command
-is configured then these commands always take precedence over
-a configured `filter.<driver>.process` command.
+`contrib/long-running-filter/example.pl` located in the Git
+core repository. If you develop your own long running filter
+process then the `GIT_TRACE_PACKET` environment variables can be
+very helpful for debugging (see linkgit:git[1]).
+
+If a `filter.<driver>.process` command is configured then it
+always takes precedence over a configured `filter.<driver>.clean`
+or `filter.<driver>.smudge` command.
Please note that you cannot use an existing `filter.<driver>.clean`
or `filter.<driver>.smudge` command with `filter.<driver>.process`
diff --git a/contrib/long-running-filter/example.pl b/contrib/long-running-filter/example.pl
new file mode 100755
index 0000000..279fbfb
--- /dev/null
+++ b/contrib/long-running-filter/example.pl
@@ -0,0 +1,111 @@
+#!/usr/bin/perl
+#
+# Example implementation for the Git filter protocol version 2
+# See Documentation/gitattributes.txt, section "Filter Protocol"
+#
+
+use strict;
+use warnings;
+
+my $MAX_PACKET_CONTENT_SIZE = 65516;
+
+sub packet_read {
+ my $buffer;
+ my $bytes_read = read STDIN, $buffer, 4;
+ if ( $bytes_read == 0 ) {
+
+ # EOF - Git stopped talking to us!
+ exit();
+ }
+ elsif ( $bytes_read != 4 ) {
+ die "invalid packet size '$bytes_read' field";
+ }
+ my $pkt_size = hex($buffer);
+ if ( $pkt_size == 0 ) {
+ return ( 1, "" );
+ }
+ elsif ( $pkt_size > 4 ) {
+ my $content_size = $pkt_size - 4;
+ $bytes_read = read STDIN, $buffer, $content_size;
+ if ( $bytes_read != $content_size ) {
+ die "invalid packet ($content_size expected; $bytes_read read)";
+ }
+ return ( 0, $buffer );
+ }
+ else {
+ die "invalid packet size";
+ }
+}
+
+sub packet_write {
+ my ($packet) = @_;
+ print STDOUT sprintf( "%04x", length($packet) + 4 );
+ print STDOUT $packet;
+ STDOUT->flush();
+}
+
+sub packet_flush {
+ print STDOUT sprintf( "%04x", 0 );
+ STDOUT->flush();
+}
+
+( packet_read() eq ( 0, "git-filter-client" ) ) || die "bad initialization";
+( packet_read() eq ( 0, "version=2" ) ) || die "bad version";
+( packet_read() eq ( 1, "" ) ) || die "bad version end";
+
+packet_write("git-filter-server\n");
+packet_write("version=2\n");
+
+( packet_read() eq ( 0, "clean=true" ) ) || die "bad capability";
+( packet_read() eq ( 0, "smudge=true" ) ) || die "bad capability";
+( packet_read() eq ( 1, "" ) ) || die "bad capability end";
+
+packet_write( "clean=true\n" );
+packet_write( "smudge=true\n" );
+packet_flush();
+
+while (1) {
+ my ($command) = packet_read() =~ /^command=([^=]+)\n$/;
+ my ($pathname) = packet_read() =~ /^pathname=([^=]+)\n$/;
+
+ packet_read();
+
+ my $input = "";
+ {
+ binmode(STDIN);
+ my $buffer;
+ my $done = 0;
+ while ( !$done ) {
+ ( $done, $buffer ) = packet_read();
+ $input .= $buffer;
+ }
+ }
+
+ my $output;
+ if ( $command eq "clean" ) {
+ ### Perform clean here ###
+ $output = $input;
+ }
+ elsif ( $command eq "smudge" ) {
+ ### Perform smudge here ###
+ $output = $input;
+ }
+ else {
+ die "bad command '$command'";
+ }
+
+ packet_write("status=success\n");
+ packet_flush();
+ while ( length($output) > 0 ) {
+ my $packet = substr( $output, 0, $MAX_PACKET_CONTENT_SIZE );
+ packet_write($packet);
+ if ( length($output) > $MAX_PACKET_CONTENT_SIZE ) {
+ $output = substr( $output, $MAX_PACKET_CONTENT_SIZE );
+ }
+ else {
+ $output = "";
+ }
+ }
+ packet_flush(); # flush content!
+ packet_flush(); # empty list!
+}
diff --git a/convert.c b/convert.c
index 0269a49..0ed48ed 100644
--- a/convert.c
+++ b/convert.c
@@ -723,9 +723,9 @@ static int apply_multi_file_filter(const char *path, const char *src, size_t len
goto done;
if (fd >= 0)
- err = packet_write_stream_with_flush_from_fd(fd, process->in);
+ err = write_packetized_from_fd(fd, process->in);
else
- err = packet_write_stream_with_flush_from_buf(src, len, process->in);
+ err = write_packetized_from_buf(src, len, process->in);
if (err)
goto done;
@@ -734,7 +734,7 @@ static int apply_multi_file_filter(const char *path, const char *src, size_t len
if (err)
goto done;
- err = packet_read_till_flush(process->out, &nbuf) < 0;
+ err = read_packetized_to_buf(process->out, &nbuf) < 0;
if (err)
goto done;
@@ -747,7 +747,7 @@ static int apply_multi_file_filter(const char *path, const char *src, size_t len
if (err || errno == EPIPE) {
if (!strcmp(filter_status.buf, "error")) {
/* The filter signaled a problem with the file. */
- } else if (!strcmp(filter_status.buf, "error-all")) {
+ } else if (!strcmp(filter_status.buf, "abort")) {
/*
* The filter signaled a permanent problem. Don't try to filter
* files with the same command for the lifetime of the current
@@ -790,9 +790,9 @@ static int apply_filter(const char *path, const char *src, size_t len,
if (!dst)
return 1;
- if ((CAP_CLEAN & wanted_capability) && drv->clean)
+ if (!drv->process && (CAP_CLEAN & wanted_capability) && drv->clean)
cmd = drv->clean;
- else if ((CAP_SMUDGE & wanted_capability) && drv->smudge)
+ else if (!drv->process && (CAP_SMUDGE & wanted_capability) && drv->smudge)
cmd = drv->smudge;
if (cmd && *cmd)
diff --git a/pkt-line.c b/pkt-line.c
index 3033aa3..5001a07 100644
--- a/pkt-line.c
+++ b/pkt-line.c
@@ -3,7 +3,6 @@
#include "run-command.h"
char packet_buffer[LARGE_PACKET_MAX];
-static char packet_write_buffer[LARGE_PACKET_MAX];
static const char *packet_trace_prefix = "git";
static struct trace_key trace_packet = TRACE_KEY_INIT(PACKET);
static struct trace_key trace_pack = TRACE_KEY_INIT(PACKFILE);
@@ -95,7 +94,10 @@ void packet_flush(int fd)
int packet_flush_gently(int fd)
{
packet_trace("0000", 4, 1);
- return (write_in_full(fd, "0000", 4) == 4 ? 0 : -1);
+ if (write_in_full(fd, "0000", 4) == 4)
+ return 0;
+ error("flush packet write failed");
+ return -1;
}
void packet_buf_flush(struct strbuf *buf)
@@ -132,39 +134,70 @@ static void format_packet(struct strbuf *out, const char *fmt, va_list args)
packet_trace(out->buf + orig_len + 4, n - 4, 1);
}
+static int packet_write_fmt_1(int fd, int gently,
+ const char *fmt, va_list args)
+{
+ struct strbuf buf = STRBUF_INIT;
+ size_t count;
+
+ format_packet(&buf, fmt, args);
+ count = write_in_full(fd, buf.buf, buf.len);
+ if (count == buf.len)
+ return 0;
+
+ if (!gently) {
+ if (errno == EPIPE) {
+ if (in_async())
+ async_exit(141);
+
+ signal(SIGPIPE, SIG_DFL);
+ raise(SIGPIPE);
+ /* Should never happen, but just in case... */
+ exit(141);
+ }
+ die_errno("packet write error");
+ }
+ error("packet write failed");
+ return -1;
+}
+
void packet_write_fmt(int fd, const char *fmt, ...)
{
- static struct strbuf buf = STRBUF_INIT;
va_list args;
- strbuf_reset(&buf);
va_start(args, fmt);
- format_packet(&buf, fmt, args);
+ packet_write_fmt_1(fd, 0, fmt, args);
va_end(args);
- write_or_die(fd, buf.buf, buf.len);
}
int packet_write_fmt_gently(int fd, const char *fmt, ...)
{
- static struct strbuf buf = STRBUF_INIT;
+ int status;
va_list args;
- strbuf_reset(&buf);
va_start(args, fmt);
- format_packet(&buf, fmt, args);
+ status = packet_write_fmt_1(fd, 1, fmt, args);
va_end(args);
- return (write_in_full(fd, buf.buf, buf.len) == buf.len ? 0 : -1);
+ return status;
}
int packet_write_gently(const int fd_out, const char *buf, size_t size)
{
- if (size > sizeof(packet_write_buffer) - 4)
+ static char packet_write_buffer[LARGE_PACKET_MAX];
+
+ if (size > sizeof(packet_write_buffer) - 4) {
+ error("packet write failed");
return -1;
+ }
packet_trace(buf, size, 1);
- memmove(packet_write_buffer + 4, buf, size);
size += 4;
set_packet_header(packet_write_buffer, size);
- return (write_in_full(fd_out, packet_write_buffer, size) == size ? 0 : -1);
+ memcpy(packet_write_buffer + 4, buf, size - 4);
+ if (write_in_full(fd_out, packet_write_buffer, size) == size)
+ return 0;
+
+ error("packet write failed");
+ return -1;
}
void packet_buf_write(struct strbuf *buf, const char *fmt, ...)
@@ -176,35 +209,35 @@ void packet_buf_write(struct strbuf *buf, const char *fmt, ...)
va_end(args);
}
-int packet_write_stream_with_flush_from_fd(int fd_in, int fd_out)
+int write_packetized_from_fd(int fd_in, int fd_out)
{
+ static char buf[PKTLINE_DATA_MAXLEN];
int err = 0;
ssize_t bytes_to_write;
while (!err) {
- bytes_to_write = xread(fd_in, packet_write_buffer, sizeof(packet_write_buffer) - 4);
+ bytes_to_write = xread(fd_in, buf, sizeof(buf));
if (bytes_to_write < 0)
return COPY_READ_ERROR;
if (bytes_to_write == 0)
break;
- if (bytes_to_write > sizeof(packet_write_buffer) - 4)
- return COPY_WRITE_ERROR;
- err = packet_write_gently(fd_out, packet_write_buffer, bytes_to_write);
+ err = packet_write_gently(fd_out, buf, bytes_to_write);
}
if (!err)
err = packet_flush_gently(fd_out);
return err;
}
-int packet_write_stream_with_flush_from_buf(const char *src_in, size_t len, int fd_out)
+int write_packetized_from_buf(const char *src_in, size_t len, int fd_out)
{
+ static char buf[PKTLINE_DATA_MAXLEN];
int err = 0;
size_t bytes_written = 0;
size_t bytes_to_write;
while (!err) {
- if ((len - bytes_written) > sizeof(packet_write_buffer) - 4)
- bytes_to_write = sizeof(packet_write_buffer) - 4;
+ if ((len - bytes_written) > sizeof(buf))
+ bytes_to_write = sizeof(buf);
else
bytes_to_write = len - bytes_written;
if (bytes_to_write == 0)
@@ -327,52 +360,29 @@ char *packet_read_line_buf(char **src, size_t *src_len, int *dst_len)
return packet_read_line_generic(-1, src, src_len, dst_len);
}
-ssize_t packet_read_till_flush(int fd_in, struct strbuf *sb_out)
+ssize_t read_packetized_to_buf(int fd_in, struct strbuf *sb_out)
{
- int len, ret;
+ int paket_len;
int options = PACKET_READ_GENTLE_ON_EOF;
- char linelen[4];
size_t oldlen = sb_out->len;
size_t oldalloc = sb_out->alloc;
for (;;) {
- /* Read packet header */
- ret = get_packet_data(fd_in, NULL, NULL, linelen, 4, options);
- if (ret < 0)
- goto done;
- len = packet_length(linelen);
- if (len < 0)
- die("protocol error: bad line length character: %.4s", linelen);
- if (!len) {
- /* Found a flush packet - Done! */
- packet_trace("0000", 4, 0);
+ strbuf_grow(sb_out, PKTLINE_DATA_MAXLEN+1);
+ paket_len = packet_read(fd_in, NULL, NULL,
+ sb_out->buf + sb_out->len, PKTLINE_DATA_MAXLEN+1, options);
+ if (paket_len <= 0)
break;
- }
- len -= 4;
-
- /* Read packet content */
- strbuf_grow(sb_out, len);
- ret = get_packet_data(fd_in, NULL, NULL, sb_out->buf + sb_out->len, len, options);
- if (ret < 0)
- goto done;
-
- if (ret != len) {
- error("protocol error: incomplete read (expected %d, got %d)", len, ret);
- goto done;
- }
-
- packet_trace(sb_out->buf + sb_out->len, len, 0);
- sb_out->len += len;
+ sb_out->len += paket_len;
}
-done:
- if (ret < 0) {
+ if (paket_len < 0) {
if (oldalloc == 0)
strbuf_release(sb_out);
else
strbuf_setlen(sb_out, oldlen);
- return ret; /* unexpected EOF */
+ return paket_len;
}
return sb_out->len - oldlen;
}
diff --git a/pkt-line.h b/pkt-line.h
index 6a3b7cf..3d873f3 100644
--- a/pkt-line.h
+++ b/pkt-line.h
@@ -25,8 +25,8 @@ void packet_buf_flush(struct strbuf *buf);
void packet_buf_write(struct strbuf *buf, const char *fmt, ...) __attribute__((format (printf, 2, 3)));
int packet_flush_gently(int fd);
int packet_write_fmt_gently(int fd, const char *fmt, ...) __attribute__((format (printf, 2, 3)));
-int packet_write_stream_with_flush_from_fd(int fd_in, int fd_out);
-int packet_write_stream_with_flush_from_buf(const char *src_in, size_t len, int fd_out);
+int write_packetized_from_fd(int fd_in, int fd_out);
+int write_packetized_from_buf(const char *src_in, size_t len, int fd_out);
/*
* Read a packetized line into the buffer, which must be at least size bytes
@@ -82,7 +82,7 @@ char *packet_read_line_buf(char **src_buf, size_t *src_len, int *size);
/*
* Reads a stream of variable sized packets until a flush packet is detected.
*/
-ssize_t packet_read_till_flush(int fd_in, struct strbuf *sb_out);
+ssize_t read_packetized_to_buf(int fd_in, struct strbuf *sb_out);
#define DEFAULT_PACKET_MAX 1000
#define LARGE_PACKET_MAX 65520
diff --git a/read-cache.c b/read-cache.c
index 02f74d3..491e52d 100644
--- a/read-cache.c
+++ b/read-cache.c
@@ -156,7 +156,7 @@ void fill_stat_cache_info(struct cache_entry *ce, struct stat *st)
static int ce_compare_data(const struct cache_entry *ce, struct stat *st)
{
int match = -1;
- int fd = open(ce->name, O_RDONLY | O_CLOEXEC);
+ int fd = open(ce->name, O_RDONLY);
if (fd >= 0) {
unsigned char sha1[20];
diff --git a/t/t0021-conversion.sh b/t/t0021-conversion.sh
index 94b0bf3..1c98ac3 100755
--- a/t/t0021-conversion.sh
+++ b/t/t0021-conversion.sh
@@ -4,15 +4,6 @@ test_description='blob conversion via gitattributes'
. ./test-lib.sh
-if test_have_prereq EXPENSIVE
-then
- T0021_LARGE_FILE_SIZE=2048
- T0021_LARGISH_FILE_SIZE=100
-else
- T0021_LARGE_FILE_SIZE=30
- T0021_LARGISH_FILE_SIZE=2
-fi
-
cat <<EOF >rot13.sh
#!$SHELL_PATH
tr \
@@ -22,8 +13,8 @@ EOF
chmod +x rot13.sh
test_expect_success setup '
- test_config filter.rot13.smudge ./rot13.sh &&
- test_config filter.rot13.clean ./rot13.sh &&
+ git config filter.rot13.smudge ./rot13.sh &&
+ git config filter.rot13.clean ./rot13.sh &&
{
echo "*.t filter=rot13"
@@ -43,26 +34,7 @@ test_expect_success setup '
git checkout -- test test.t test.i &&
echo "content-test2" >test2.o &&
- echo "content-test3-subdir" >test3-subdir.o &&
-
- mkdir generated-test-data &&
- for i in $(test_seq 1 $T0021_LARGE_FILE_SIZE)
- do
- RANDOM_STRING="$(test-genrandom end $i | tr -dc "A-Za-z0-9" )"
- ROT_RANDOM_STRING="$(echo $RANDOM_STRING | ./rot13.sh )"
- # Generate 1MB of empty data and 100 bytes of random characters
- # printf "$(test-genrandom start $i)"
- printf "%1048576d" 1 >>generated-test-data/large.file &&
- printf "$RANDOM_STRING" >>generated-test-data/large.file &&
- printf "%1048576d" 1 >>generated-test-data/large.file.rot13 &&
- printf "$ROT_RANDOM_STRING" >>generated-test-data/large.file.rot13 &&
-
- if test $i = $T0021_LARGISH_FILE_SIZE
- then
- cat generated-test-data/large.file >generated-test-data/largish.file &&
- cat generated-test-data/large.file.rot13 >generated-test-data/largish.file.rot13
- fi
- done
+ echo "content-test3-subdir" >test3-subdir.o
'
script='s/^\$Id: \([0-9a-f]*\) \$/\1/p'
@@ -230,9 +202,9 @@ test_expect_success 'required filter clean failure' '
test_expect_success 'filtering large input to small output should use little memory' '
test_config filter.devnull.clean "cat >/dev/null" &&
test_config filter.devnull.required true &&
- cp generated-test-data/large.file large.file &&
- echo "large.file filter=devnull" >.gitattributes &&
- GIT_MMAP_LIMIT=1m GIT_ALLOC_LIMIT=1m git add large.file
+ for i in $(test_seq 1 30); do printf "%1048576d" 1; done >30MB &&
+ echo "30MB filter=devnull" >.gitattributes &&
+ GIT_MMAP_LIMIT=1m GIT_ALLOC_LIMIT=1m git add 30MB
'
test_expect_success 'filter that does not read is fine' '
@@ -245,15 +217,15 @@ test_expect_success 'filter that does not read is fine' '
test_cmp expect actual
'
-test_expect_success 'filter large file' '
+test_expect_success EXPENSIVE 'filter large file' '
test_config filter.largefile.smudge cat &&
test_config filter.largefile.clean cat &&
- echo "large.file filter=largefile" >.gitattributes &&
- cp generated-test-data/large.file large.file &&
- git add large.file 2>err &&
+ for i in $(test_seq 1 2048); do printf "%1048576d" 1; done >2GB &&
+ echo "2GB filter=largefile" >.gitattributes &&
+ git add 2GB 2>err &&
test_must_be_empty err &&
- rm -f large.file &&
- git checkout -- large.file 2>err &&
+ rm -f 2GB &&
+ git checkout -- 2GB 2>err &&
test_must_be_empty err
'
@@ -374,22 +346,24 @@ test_expect_success PERL 'required process filter should filter data' '
check_filter \
git add . \
<<-\EOF &&
- 1 IN: clean test.r 57 [OK] -- OUT: 57 [OK]
- 1 IN: clean test2.r 14 [OK] -- OUT: 14 [OK]
- 1 IN: clean test4-empty.r 0 [OK] -- OUT: 0 [OK]
- 1 IN: clean testsubdir/test3-subdir.r 21 [OK] -- OUT: 21 [OK]
- 1 start
+ 1 IN: clean test.r 57 [OK] -- OUT: 57 . [OK]
+ 1 IN: clean test2.r 14 [OK] -- OUT: 14 . [OK]
+ 1 IN: clean test4-empty.r 0 [OK] -- OUT: 0 [OK]
+ 1 IN: clean testsubdir/test3-subdir.r 21 [OK] -- OUT: 21 . [OK]
+ 1 START
+ 1 STOP
1 wrote filter header
EOF
check_filter_count_clean \
git commit . -m "test commit" \
<<-\EOF &&
- x IN: clean test.r 57 [OK] -- OUT: 57 [OK]
- x IN: clean test2.r 14 [OK] -- OUT: 14 [OK]
- x IN: clean test4-empty.r 0 [OK] -- OUT: 0 [OK]
- x IN: clean testsubdir/test3-subdir.r 21 [OK] -- OUT: 21 [OK]
- 1 start
+ x IN: clean test.r 57 [OK] -- OUT: 57 . [OK]
+ x IN: clean test2.r 14 [OK] -- OUT: 14 . [OK]
+ x IN: clean test4-empty.r 0 [OK] -- OUT: 0 [OK]
+ x IN: clean testsubdir/test3-subdir.r 21 [OK] -- OUT: 21 . [OK]
+ 1 START
+ 1 STOP
1 wrote filter header
EOF
@@ -398,28 +372,31 @@ test_expect_success PERL 'required process filter should filter data' '
check_filter_ignore_clean \
git checkout . \
<<-\EOF &&
- start
+ START
wrote filter header
- IN: smudge test2.r 14 [OK] -- OUT: 14 [OK]
- IN: smudge testsubdir/test3-subdir.r 21 [OK] -- OUT: 21 [OK]
+ IN: smudge test2.r 14 [OK] -- OUT: 14 . [OK]
+ IN: smudge testsubdir/test3-subdir.r 21 [OK] -- OUT: 21 . [OK]
+ STOP
EOF
check_filter_ignore_clean \
git checkout empty \
<<-\EOF &&
- start
+ START
wrote filter header
+ STOP
EOF
check_filter_ignore_clean \
git checkout master \
<<-\EOF &&
- start
+ START
wrote filter header
- IN: smudge test.r 57 [OK] -- OUT: 57 [OK]
- IN: smudge test2.r 14 [OK] -- OUT: 14 [OK]
- IN: smudge test4-empty.r 0 [OK] -- OUT: 0 [OK]
- IN: smudge testsubdir/test3-subdir.r 21 [OK] -- OUT: 21 [OK]
+ IN: smudge test.r 57 [OK] -- OUT: 57 . [OK]
+ IN: smudge test2.r 14 [OK] -- OUT: 14 . [OK]
+ IN: smudge test4-empty.r 0 [OK] -- OUT: 0 [OK]
+ IN: smudge testsubdir/test3-subdir.r 21 [OK] -- OUT: 21 . [OK]
+ STOP
EOF
check_rot13 ../test.o test.r &&
@@ -428,57 +405,8 @@ test_expect_success PERL 'required process filter should filter data' '
)
'
-test_expect_success PERL 'required process filter should filter smudge data and one-shot filter should clean' '
+test_expect_success PERL 'required process filter should clean only and take precedence' '
test_config_global filter.protocol.clean ./../rot13.sh &&
- test_config_global filter.protocol.process "$TEST_DIRECTORY/t0021/rot13-filter.pl smudge" &&
- test_config_global filter.protocol.required true &&
- rm -rf repo &&
- mkdir repo &&
- (
- cd repo &&
- git init &&
-
- echo "*.r filter=protocol" >.gitattributes &&
- git add . &&
- git commit . -m "test commit" &&
- git branch empty &&
-
- cp ../test.o test.r &&
- cp ../test2.o test2.r &&
-
- check_filter_no_call \
- git add . &&
-
- check_filter_no_call \
- git commit . -m "test commit" &&
-
- rm -f test?.r testsubdir/test3-subdir.r &&
-
- check_filter_ignore_clean \
- git checkout . \
- <<-\EOF &&
- start
- wrote filter header
- IN: smudge test2.r 14 [OK] -- OUT: 14 [OK]
- EOF
-
- git checkout empty &&
-
- check_filter_ignore_clean \
- git checkout master\
- <<-\EOF &&
- start
- wrote filter header
- IN: smudge test.r 57 [OK] -- OUT: 57 [OK]
- IN: smudge test2.r 14 [OK] -- OUT: 14 [OK]
- EOF
-
- check_rot13 ../test.o test.r &&
- check_rot13 ../test2.o test2.r
- )
-'
-
-test_expect_success PERL 'required process filter should clean only' '
test_config_global filter.protocol.process "$TEST_DIRECTORY/t0021/rot13-filter.pl clean" &&
test_config_global filter.protocol.required true &&
rm -rf repo &&
@@ -497,42 +425,74 @@ test_expect_success PERL 'required process filter should clean only' '
check_filter \
git add . \
<<-\EOF &&
- 1 IN: clean test.r 57 [OK] -- OUT: 57 [OK]
- 1 start
+ 1 IN: clean test.r 57 [OK] -- OUT: 57 . [OK]
+ 1 START
+ 1 STOP
1 wrote filter header
EOF
check_filter_count_clean \
git commit . -m "test commit" \
<<-\EOF
- x IN: clean test.r 57 [OK] -- OUT: 57 [OK]
- 1 start
+ x IN: clean test.r 57 [OK] -- OUT: 57 . [OK]
+ 1 START
+ 1 STOP
1 wrote filter header
EOF
)
'
-test_expect_success PERL 'required process filter should process binary files larger LARGE_PACKET_MAX' '
+generate_test_data () {
+ LEN=$1
+ NAME=$2
+ test-genrandom end $LEN |
+ perl -pe "s/./chr((ord($&) % 26) + 97)/sge" >../$NAME.file &&
+ cp ../$NAME.file . &&
+ ./../rot13.sh <../$NAME.file >../$NAME.file.rot13
+}
+
+test_expect_success PERL 'required process filter should process multiple packets' '
test_config_global filter.protocol.process "$TEST_DIRECTORY/t0021/rot13-filter.pl clean smudge" &&
test_config_global filter.protocol.required true &&
+
rm -rf repo &&
mkdir repo &&
(
cd repo &&
git init &&
- echo "*.file filter=protocol" >.gitattributes &&
- cat ../generated-test-data/largish.file.rot13 >large.rot13 &&
- cat ../generated-test-data/largish.file >large.file &&
- cat large.file >large.original &&
+ # Generate data that requires 3 packets
+ PKTLINE_DATA_MAXLEN=65516 &&
+
+ generate_test_data $(($PKTLINE_DATA_MAXLEN )) 1pkt_1__ &&
+ generate_test_data $(($PKTLINE_DATA_MAXLEN + 1)) 2pkt_1+1 &&
+ generate_test_data $(($PKTLINE_DATA_MAXLEN * 2 - 1)) 2pkt_2-1 &&
+ generate_test_data $(($PKTLINE_DATA_MAXLEN * 2 )) 2pkt_2__ &&
+ generate_test_data $(($PKTLINE_DATA_MAXLEN * 2 + 1)) 3pkt_2+1 &&
- git add large.file .gitattributes &&
+ echo "*.file filter=protocol" >.gitattributes &&
+ check_filter \
+ git add *.file .gitattributes \
+ <<-\EOF &&
+ 1 IN: clean 1pkt_1__.file 65516 [OK] -- OUT: 65516 . [OK]
+ 1 IN: clean 2pkt_1+1.file 65517 [OK] -- OUT: 65517 .. [OK]
+ 1 IN: clean 2pkt_2-1.file 131031 [OK] -- OUT: 131031 .. [OK]
+ 1 IN: clean 2pkt_2__.file 131032 [OK] -- OUT: 131032 .. [OK]
+ 1 IN: clean 3pkt_2+1.file 131033 [OK] -- OUT: 131033 ... [OK]
+ 1 START
+ 1 STOP
+ 1 wrote filter header
+ EOF
git commit . -m "test commit" &&
- rm -f large.file &&
- git checkout -- large.file &&
- git cat-file blob :large.file >actual &&
- test_cmp large.rot13 actual
+ rm -f *.file &&
+ git checkout -- *.file &&
+
+ for f in *.file
+ do
+ git cat-file blob :$f >actual &&
+ test_cmp ../$f.rot13 actual
+ done
)
'
@@ -577,13 +537,14 @@ test_expect_success PERL 'process filter should restart after unexpected write f
check_filter_ignore_clean \
git checkout . \
<<-\EOF &&
- start
+ START
wrote filter header
IN: smudge smudge-write-fail.r 22 [OK] -- OUT: 22 [WRITE FAIL]
- start
+ START
wrote filter header
- IN: smudge test.r 57 [OK] -- OUT: 57 [OK]
- IN: smudge test2.r 14 [OK] -- OUT: 14 [OK]
+ IN: smudge test.r 57 [OK] -- OUT: 57 . [OK]
+ IN: smudge test2.r 14 [OK] -- OUT: 14 . [OK]
+ STOP
EOF
check_rot13 ../test.o test.r &&
@@ -617,11 +578,12 @@ test_expect_success PERL 'process filter should not restart in case of an error'
check_filter_ignore_clean \
git checkout . \
<<-\EOF &&
- start
+ START
wrote filter header
IN: smudge error.r 25 [OK] -- OUT: 0 [ERROR]
- IN: smudge test.r 57 [OK] -- OUT: 57 [OK]
- IN: smudge test2.r 14 [OK] -- OUT: 14 [OK]
+ IN: smudge test.r 57 [OK] -- OUT: 57 . [OK]
+ IN: smudge test2.r 14 [OK] -- OUT: 14 . [OK]
+ STOP
EOF
check_rot13 ../test.o test.r &&
@@ -642,8 +604,8 @@ test_expect_success PERL 'process filter should be able to signal an error for a
cp ../test.o test.r &&
cp ../test2.o test2.r &&
- echo "error this blob and all future blobs" >error-all.o &&
- cp error-all.o error-all.r &&
+ echo "error this blob and all future blobs" >abort.o &&
+ cp abort.o abort.r &&
git add . &&
git commit . -m "test commit" &&
rm -f *.r &&
@@ -651,14 +613,15 @@ test_expect_success PERL 'process filter should be able to signal an error for a
check_filter_ignore_clean \
git checkout . \
<<-\EOF &&
- start
+ START
wrote filter header
- IN: smudge error-all.r 37 [OK] -- OUT: 0 [ERROR-ALL]
+ IN: smudge abort.r 37 [OK] -- OUT: 0 [ABORT]
+ STOP
EOF
test_cmp ../test.o test.r &&
test_cmp ../test2.o test2.r &&
- test_cmp error-all.o error-all.r
+ test_cmp abort.o abort.r
)
'
diff --git a/t/t0021/rot13-filter.pl b/t/t0021/rot13-filter.pl
index 0e1181d..8e27877 100755
--- a/t/t0021/rot13-filter.pl
+++ b/t/t0021/rot13-filter.pl
@@ -14,7 +14,7 @@
# (3) If data with the pathname "error.r" is processed with any
# operation then the filter signals that it cannot or does not want
# to process the file.
-# (4) If data with the pathname "error-all.r" is processed with any
+# (4) If data with the pathname "abort.r" is processed with any
# operation then the filter signals that it cannot or does not want
# to process the file and any file after that is processed with the
# same command.
@@ -26,6 +26,8 @@ use warnings;
my $MAX_PACKET_CONTENT_SIZE = 65516;
my @capabilities = @ARGV;
+open my $debug, ">>", "rot13-filter.log";
+
sub rot13 {
my ($str) = @_;
$str =~ y/A-Za-z/N-ZA-Mn-za-m/;
@@ -38,6 +40,7 @@ sub packet_read {
if ( $bytes_read == 0 ) {
# EOF - Git stopped talking to us!
+ print $debug "STOP\n";
exit();
}
elsif ( $bytes_read != 4 ) {
@@ -72,8 +75,7 @@ sub packet_flush {
STDOUT->flush();
}
-open my $debug, ">>", "rot13-filter.log";
-print $debug "start\n";
+print $debug "START\n";
$debug->flush();
( packet_read() eq ( 0, "git-filter-client" ) ) || die "bad initialization";
@@ -120,7 +122,7 @@ while (1) {
}
my $output;
- if ( $pathname eq "error.r" or $pathname eq "error-all.r" ) {
+ if ( $pathname eq "error.r" or $pathname eq "abort.r" ) {
$output = "";
}
elsif ( $command eq "clean" and grep( /^clean$/, @capabilities ) ) {
@@ -142,10 +144,10 @@ while (1) {
packet_write("status=error\n");
packet_flush();
}
- elsif ( $pathname eq "error-all.r" ) {
- print $debug "[ERROR-ALL]\n";
+ elsif ( $pathname eq "abort.r" ) {
+ print $debug "[ABORT]\n";
$debug->flush();
- packet_write("status=error-all\n");
+ packet_write("status=abort\n");
packet_flush();
}
else {
@@ -161,6 +163,7 @@ while (1) {
while ( length($output) > 0 ) {
my $packet = substr( $output, 0, $MAX_PACKET_CONTENT_SIZE );
packet_write($packet);
+ print $debug ".";
if ( length($output) > $MAX_PACKET_CONTENT_SIZE ) {
$output = substr( $output, $MAX_PACKET_CONTENT_SIZE );
}
@@ -169,7 +172,7 @@ while (1) {
}
}
packet_flush();
- print $debug "[OK]\n";
+ print $debug " [OK]\n";
$debug->flush();
packet_flush();
}
Lars Schneider (10):
pkt-line: rename packet_write() to packet_write_fmt()
pkt-line: extract set_packet_header()
pkt-line: add packet_write_fmt_gently()
pkt-line: add packet_flush_gently()
pkt-line: add packet_write_gently()
pkt-line: add functions to read/write flush terminated packet streams
convert: quote filter names in error messages
convert: modernize tests
convert: make apply_filter() adhere to standard Git error handling
convert: add filter.<driver>.process option
Documentation/gitattributes.txt | 154 +++++++++++-
builtin/archive.c | 4 +-
builtin/receive-pack.c | 4 +-
builtin/remote-ext.c | 4 +-
builtin/upload-archive.c | 4 +-
connect.c | 2 +-
contrib/long-running-filter/example.pl | 111 +++++++++
convert.c | 373 +++++++++++++++++++++++++----
daemon.c | 2 +-
http-backend.c | 2 +-
pkt-line.c | 160 ++++++++++++-
pkt-line.h | 12 +-
shallow.c | 2 +-
t/t0021-conversion.sh | 423 ++++++++++++++++++++++++++++++---
t/t0021/rot13-filter.pl | 179 ++++++++++++++
unpack-trees.c | 1 +
upload-pack.c | 30 +--
17 files changed, 1356 insertions(+), 111 deletions(-)
create mode 100755 contrib/long-running-filter/example.pl
create mode 100755 t/t0021/rot13-filter.pl
--
2.10.0
^ permalink raw reply related
* [PATCH v7 01/10] pkt-line: rename packet_write() to packet_write_fmt()
From: larsxschneider @ 2016-09-08 18:21 UTC (permalink / raw)
To: git
Cc: peff, gitster, sbeller, Johannes.Schindelin, jnareb, mlbright,
tboegi, jacob.keller, Lars Schneider
In-Reply-To: <20160908182132.50788-1-larsxschneider@gmail.com>
From: Lars Schneider <larsxschneider@gmail.com>
packet_write() should be called packet_write_fmt() as the string
parameter can be formatted.
Suggested-by: Junio C Hamano <gitster@pobox.com>
Signed-off-by: Lars Schneider <larsxschneider@gmail.com>
---
builtin/archive.c | 4 ++--
builtin/receive-pack.c | 4 ++--
builtin/remote-ext.c | 4 ++--
builtin/upload-archive.c | 4 ++--
connect.c | 2 +-
daemon.c | 2 +-
http-backend.c | 2 +-
pkt-line.c | 2 +-
pkt-line.h | 2 +-
shallow.c | 2 +-
upload-pack.c | 30 +++++++++++++++---------------
11 files changed, 29 insertions(+), 29 deletions(-)
diff --git a/builtin/archive.c b/builtin/archive.c
index a1e3b94..49f4914 100644
--- a/builtin/archive.c
+++ b/builtin/archive.c
@@ -47,10 +47,10 @@ static int run_remote_archiver(int argc, const char **argv,
if (name_hint) {
const char *format = archive_format_from_filename(name_hint);
if (format)
- packet_write(fd[1], "argument --format=%s\n", format);
+ packet_write_fmt(fd[1], "argument --format=%s\n", format);
}
for (i = 1; i < argc; i++)
- packet_write(fd[1], "argument %s\n", argv[i]);
+ packet_write_fmt(fd[1], "argument %s\n", argv[i]);
packet_flush(fd[1]);
buf = packet_read_line(fd[0], NULL);
diff --git a/builtin/receive-pack.c b/builtin/receive-pack.c
index 011db00..1ce7682 100644
--- a/builtin/receive-pack.c
+++ b/builtin/receive-pack.c
@@ -218,7 +218,7 @@ static int receive_pack_config(const char *var, const char *value, void *cb)
static void show_ref(const char *path, const unsigned char *sha1)
{
if (sent_capabilities) {
- packet_write(1, "%s %s\n", sha1_to_hex(sha1), path);
+ packet_write_fmt(1, "%s %s\n", sha1_to_hex(sha1), path);
} else {
struct strbuf cap = STRBUF_INIT;
@@ -233,7 +233,7 @@ static void show_ref(const char *path, const unsigned char *sha1)
if (advertise_push_options)
strbuf_addstr(&cap, " push-options");
strbuf_addf(&cap, " agent=%s", git_user_agent_sanitized());
- packet_write(1, "%s %s%c%s\n",
+ packet_write_fmt(1, "%s %s%c%s\n",
sha1_to_hex(sha1), path, 0, cap.buf);
strbuf_release(&cap);
sent_capabilities = 1;
diff --git a/builtin/remote-ext.c b/builtin/remote-ext.c
index 88eb8f9..11b48bf 100644
--- a/builtin/remote-ext.c
+++ b/builtin/remote-ext.c
@@ -128,9 +128,9 @@ static void send_git_request(int stdin_fd, const char *serv, const char *repo,
const char *vhost)
{
if (!vhost)
- packet_write(stdin_fd, "%s %s%c", serv, repo, 0);
+ packet_write_fmt(stdin_fd, "%s %s%c", serv, repo, 0);
else
- packet_write(stdin_fd, "%s %s%chost=%s%c", serv, repo, 0,
+ packet_write_fmt(stdin_fd, "%s %s%chost=%s%c", serv, repo, 0,
vhost, 0);
}
diff --git a/builtin/upload-archive.c b/builtin/upload-archive.c
index 2caedf1..dc872f6 100644
--- a/builtin/upload-archive.c
+++ b/builtin/upload-archive.c
@@ -88,11 +88,11 @@ int cmd_upload_archive(int argc, const char **argv, const char *prefix)
writer.git_cmd = 1;
if (start_command(&writer)) {
int err = errno;
- packet_write(1, "NACK unable to spawn subprocess\n");
+ packet_write_fmt(1, "NACK unable to spawn subprocess\n");
die("upload-archive: %s", strerror(err));
}
- packet_write(1, "ACK\n");
+ packet_write_fmt(1, "ACK\n");
packet_flush(1);
while (1) {
diff --git a/connect.c b/connect.c
index 722dc3f..5330d9c 100644
--- a/connect.c
+++ b/connect.c
@@ -730,7 +730,7 @@ struct child_process *git_connect(int fd[2], const char *url,
* Note: Do not add any other headers here! Doing so
* will cause older git-daemon servers to crash.
*/
- packet_write(fd[1],
+ packet_write_fmt(fd[1],
"%s %s%chost=%s%c",
prog, path, 0,
target_host, 0);
diff --git a/daemon.c b/daemon.c
index 425aad0..afce1b9 100644
--- a/daemon.c
+++ b/daemon.c
@@ -281,7 +281,7 @@ static int daemon_error(const char *dir, const char *msg)
{
if (!informative_errors)
msg = "access denied or repository not exported";
- packet_write(1, "ERR %s: %s", msg, dir);
+ packet_write_fmt(1, "ERR %s: %s", msg, dir);
return -1;
}
diff --git a/http-backend.c b/http-backend.c
index adc8c8c..eef0a36 100644
--- a/http-backend.c
+++ b/http-backend.c
@@ -464,7 +464,7 @@ static void get_info_refs(struct strbuf *hdr, char *arg)
hdr_str(hdr, content_type, buf.buf);
end_headers(hdr);
- packet_write(1, "# service=git-%s\n", svc->name);
+ packet_write_fmt(1, "# service=git-%s\n", svc->name);
packet_flush(1);
argv[0] = svc->name;
diff --git a/pkt-line.c b/pkt-line.c
index 62fdb37..0a9b61c 100644
--- a/pkt-line.c
+++ b/pkt-line.c
@@ -118,7 +118,7 @@ static void format_packet(struct strbuf *out, const char *fmt, va_list args)
packet_trace(out->buf + orig_len + 4, n - 4, 1);
}
-void packet_write(int fd, const char *fmt, ...)
+void packet_write_fmt(int fd, const char *fmt, ...)
{
static struct strbuf buf = STRBUF_INIT;
va_list args;
diff --git a/pkt-line.h b/pkt-line.h
index 3cb9d91..1902fb3 100644
--- a/pkt-line.h
+++ b/pkt-line.h
@@ -20,7 +20,7 @@
* side can't, we stay with pure read/write interfaces.
*/
void packet_flush(int fd);
-void packet_write(int fd, const char *fmt, ...) __attribute__((format (printf, 2, 3)));
+void packet_write_fmt(int fd, const char *fmt, ...) __attribute__((format (printf, 2, 3)));
void packet_buf_flush(struct strbuf *buf);
void packet_buf_write(struct strbuf *buf, const char *fmt, ...) __attribute__((format (printf, 2, 3)));
diff --git a/shallow.c b/shallow.c
index 54e2db7..d666e24 100644
--- a/shallow.c
+++ b/shallow.c
@@ -260,7 +260,7 @@ static int advertise_shallow_grafts_cb(const struct commit_graft *graft, void *c
{
int fd = *(int *)cb;
if (graft->nr_parent == -1)
- packet_write(fd, "shallow %s\n", oid_to_hex(&graft->oid));
+ packet_write_fmt(fd, "shallow %s\n", oid_to_hex(&graft->oid));
return 0;
}
diff --git a/upload-pack.c b/upload-pack.c
index ca7f941..cd47de6 100644
--- a/upload-pack.c
+++ b/upload-pack.c
@@ -393,13 +393,13 @@ static int get_common_commits(void)
if (multi_ack == 2 && got_common
&& !got_other && ok_to_give_up()) {
sent_ready = 1;
- packet_write(1, "ACK %s ready\n", last_hex);
+ packet_write_fmt(1, "ACK %s ready\n", last_hex);
}
if (have_obj.nr == 0 || multi_ack)
- packet_write(1, "NAK\n");
+ packet_write_fmt(1, "NAK\n");
if (no_done && sent_ready) {
- packet_write(1, "ACK %s\n", last_hex);
+ packet_write_fmt(1, "ACK %s\n", last_hex);
return 0;
}
if (stateless_rpc)
@@ -416,20 +416,20 @@ static int get_common_commits(void)
const char *hex = sha1_to_hex(sha1);
if (multi_ack == 2) {
sent_ready = 1;
- packet_write(1, "ACK %s ready\n", hex);
+ packet_write_fmt(1, "ACK %s ready\n", hex);
} else
- packet_write(1, "ACK %s continue\n", hex);
+ packet_write_fmt(1, "ACK %s continue\n", hex);
}
break;
default:
got_common = 1;
memcpy(last_hex, sha1_to_hex(sha1), 41);
if (multi_ack == 2)
- packet_write(1, "ACK %s common\n", last_hex);
+ packet_write_fmt(1, "ACK %s common\n", last_hex);
else if (multi_ack)
- packet_write(1, "ACK %s continue\n", last_hex);
+ packet_write_fmt(1, "ACK %s continue\n", last_hex);
else if (have_obj.nr == 1)
- packet_write(1, "ACK %s\n", last_hex);
+ packet_write_fmt(1, "ACK %s\n", last_hex);
break;
}
continue;
@@ -437,10 +437,10 @@ static int get_common_commits(void)
if (!strcmp(line, "done")) {
if (have_obj.nr > 0) {
if (multi_ack)
- packet_write(1, "ACK %s\n", last_hex);
+ packet_write_fmt(1, "ACK %s\n", last_hex);
return 0;
}
- packet_write(1, "NAK\n");
+ packet_write_fmt(1, "NAK\n");
return -1;
}
die("git upload-pack: expected SHA1 list, got '%s'", line);
@@ -650,7 +650,7 @@ static void receive_needs(void)
while (result) {
struct object *object = &result->item->object;
if (!(object->flags & (CLIENT_SHALLOW|NOT_SHALLOW))) {
- packet_write(1, "shallow %s",
+ packet_write_fmt(1, "shallow %s",
oid_to_hex(&object->oid));
register_shallow(object->oid.hash);
shallow_nr++;
@@ -662,7 +662,7 @@ static void receive_needs(void)
struct object *object = shallows.objects[i].item;
if (object->flags & NOT_SHALLOW) {
struct commit_list *parents;
- packet_write(1, "unshallow %s",
+ packet_write_fmt(1, "unshallow %s",
oid_to_hex(&object->oid));
object->flags &= ~CLIENT_SHALLOW;
/* make sure the real parents are parsed */
@@ -741,7 +741,7 @@ static int send_ref(const char *refname, const struct object_id *oid,
struct strbuf symref_info = STRBUF_INIT;
format_symref_info(&symref_info, cb_data);
- packet_write(1, "%s %s%c%s%s%s%s%s agent=%s\n",
+ packet_write_fmt(1, "%s %s%c%s%s%s%s%s agent=%s\n",
oid_to_hex(oid), refname_nons,
0, capabilities,
(allow_unadvertised_object_request & ALLOW_TIP_SHA1) ?
@@ -753,11 +753,11 @@ static int send_ref(const char *refname, const struct object_id *oid,
git_user_agent_sanitized());
strbuf_release(&symref_info);
} else {
- packet_write(1, "%s %s\n", oid_to_hex(oid), refname_nons);
+ packet_write_fmt(1, "%s %s\n", oid_to_hex(oid), refname_nons);
}
capabilities = NULL;
if (!peel_ref(refname, peeled.hash))
- packet_write(1, "%s %s^{}\n", oid_to_hex(&peeled), refname_nons);
+ packet_write_fmt(1, "%s %s^{}\n", oid_to_hex(&peeled), refname_nons);
return 0;
}
--
2.10.0
^ permalink raw reply related
* [PATCH v7 02/10] pkt-line: extract set_packet_header()
From: larsxschneider @ 2016-09-08 18:21 UTC (permalink / raw)
To: git
Cc: peff, gitster, sbeller, Johannes.Schindelin, jnareb, mlbright,
tboegi, jacob.keller, Lars Schneider
In-Reply-To: <20160908182132.50788-1-larsxschneider@gmail.com>
From: Lars Schneider <larsxschneider@gmail.com>
set_packet_header() converts an integer to a 4 byte hex string. Make
this function locally available so that other pkt-line functions can
use it.
Signed-off-by: Lars Schneider <larsxschneider@gmail.com>
---
pkt-line.c | 19 +++++++++++++------
1 file changed, 13 insertions(+), 6 deletions(-)
diff --git a/pkt-line.c b/pkt-line.c
index 0a9b61c..e8adc0f 100644
--- a/pkt-line.c
+++ b/pkt-line.c
@@ -97,10 +97,20 @@ void packet_buf_flush(struct strbuf *buf)
strbuf_add(buf, "0000", 4);
}
-#define hex(a) (hexchar[(a) & 15])
-static void format_packet(struct strbuf *out, const char *fmt, va_list args)
+static void set_packet_header(char *buf, const int size)
{
static char hexchar[] = "0123456789abcdef";
+
+ #define hex(a) (hexchar[(a) & 15])
+ buf[0] = hex(size >> 12);
+ buf[1] = hex(size >> 8);
+ buf[2] = hex(size >> 4);
+ buf[3] = hex(size);
+ #undef hex
+}
+
+static void format_packet(struct strbuf *out, const char *fmt, va_list args)
+{
size_t orig_len, n;
orig_len = out->len;
@@ -111,10 +121,7 @@ static void format_packet(struct strbuf *out, const char *fmt, va_list args)
if (n > LARGE_PACKET_MAX)
die("protocol error: impossibly long line");
- out->buf[orig_len + 0] = hex(n >> 12);
- out->buf[orig_len + 1] = hex(n >> 8);
- out->buf[orig_len + 2] = hex(n >> 4);
- out->buf[orig_len + 3] = hex(n);
+ set_packet_header(&out->buf[orig_len], n);
packet_trace(out->buf + orig_len + 4, n - 4, 1);
}
--
2.10.0
^ permalink raw reply related
* [PATCH v7 03/10] pkt-line: add packet_write_fmt_gently()
From: larsxschneider @ 2016-09-08 18:21 UTC (permalink / raw)
To: git
Cc: peff, gitster, sbeller, Johannes.Schindelin, jnareb, mlbright,
tboegi, jacob.keller, Lars Schneider
In-Reply-To: <20160908182132.50788-1-larsxschneider@gmail.com>
From: Lars Schneider <larsxschneider@gmail.com>
packet_write_fmt() would die in case of a write error even though for
some callers an error would be acceptable. Add packet_write_fmt_gently()
which writes a formatted pkt-line and returns `0` for success and `-1`
for an error.
Signed-off-by: Lars Schneider <larsxschneider@gmail.com>
---
pkt-line.c | 43 +++++++++++++++++++++++++++++++++++++++----
pkt-line.h | 1 +
2 files changed, 40 insertions(+), 4 deletions(-)
diff --git a/pkt-line.c b/pkt-line.c
index e8adc0f..3824d05 100644
--- a/pkt-line.c
+++ b/pkt-line.c
@@ -125,16 +125,51 @@ static void format_packet(struct strbuf *out, const char *fmt, va_list args)
packet_trace(out->buf + orig_len + 4, n - 4, 1);
}
+static int packet_write_fmt_1(int fd, int gently,
+ const char *fmt, va_list args)
+{
+ struct strbuf buf = STRBUF_INIT;
+ size_t count;
+
+ format_packet(&buf, fmt, args);
+ count = write_in_full(fd, buf.buf, buf.len);
+ if (count == buf.len)
+ return 0;
+
+ if (!gently) {
+ if (errno == EPIPE) {
+ if (in_async())
+ async_exit(141);
+
+ signal(SIGPIPE, SIG_DFL);
+ raise(SIGPIPE);
+ /* Should never happen, but just in case... */
+ exit(141);
+ }
+ die_errno("packet write error");
+ }
+ error("packet write failed");
+ return -1;
+}
+
void packet_write_fmt(int fd, const char *fmt, ...)
{
- static struct strbuf buf = STRBUF_INIT;
va_list args;
- strbuf_reset(&buf);
va_start(args, fmt);
- format_packet(&buf, fmt, args);
+ packet_write_fmt_1(fd, 0, fmt, args);
+ va_end(args);
+}
+
+int packet_write_fmt_gently(int fd, const char *fmt, ...)
+{
+ int status;
+ va_list args;
+
+ va_start(args, fmt);
+ status = packet_write_fmt_1(fd, 1, fmt, args);
va_end(args);
- write_or_die(fd, buf.buf, buf.len);
+ return status;
}
void packet_buf_write(struct strbuf *buf, const char *fmt, ...)
diff --git a/pkt-line.h b/pkt-line.h
index 1902fb3..3caea77 100644
--- a/pkt-line.h
+++ b/pkt-line.h
@@ -23,6 +23,7 @@ void packet_flush(int fd);
void packet_write_fmt(int fd, const char *fmt, ...) __attribute__((format (printf, 2, 3)));
void packet_buf_flush(struct strbuf *buf);
void packet_buf_write(struct strbuf *buf, const char *fmt, ...) __attribute__((format (printf, 2, 3)));
+int packet_write_fmt_gently(int fd, const char *fmt, ...) __attribute__((format (printf, 2, 3)));
/*
* Read a packetized line into the buffer, which must be at least size bytes
--
2.10.0
^ permalink raw reply related
* [PATCH v7 04/10] pkt-line: add packet_flush_gently()
From: larsxschneider @ 2016-09-08 18:21 UTC (permalink / raw)
To: git
Cc: peff, gitster, sbeller, Johannes.Schindelin, jnareb, mlbright,
tboegi, jacob.keller, Lars Schneider
In-Reply-To: <20160908182132.50788-1-larsxschneider@gmail.com>
From: Lars Schneider <larsxschneider@gmail.com>
packet_flush() would die in case of a write error even though for some
callers an error would be acceptable. Add packet_flush_gently() which
writes a pkt-line flush packet and returns `0` for success and `-1` for
failure.
Signed-off-by: Lars Schneider <larsxschneider@gmail.com>
---
pkt-line.c | 9 +++++++++
pkt-line.h | 1 +
2 files changed, 10 insertions(+)
diff --git a/pkt-line.c b/pkt-line.c
index 3824d05..37345ca 100644
--- a/pkt-line.c
+++ b/pkt-line.c
@@ -91,6 +91,15 @@ void packet_flush(int fd)
write_or_die(fd, "0000", 4);
}
+int packet_flush_gently(int fd)
+{
+ packet_trace("0000", 4, 1);
+ if (write_in_full(fd, "0000", 4) == 4)
+ return 0;
+ error("flush packet write failed");
+ return -1;
+}
+
void packet_buf_flush(struct strbuf *buf)
{
packet_trace("0000", 4, 1);
diff --git a/pkt-line.h b/pkt-line.h
index 3caea77..3fa0899 100644
--- a/pkt-line.h
+++ b/pkt-line.h
@@ -23,6 +23,7 @@ void packet_flush(int fd);
void packet_write_fmt(int fd, const char *fmt, ...) __attribute__((format (printf, 2, 3)));
void packet_buf_flush(struct strbuf *buf);
void packet_buf_write(struct strbuf *buf, const char *fmt, ...) __attribute__((format (printf, 2, 3)));
+int packet_flush_gently(int fd);
int packet_write_fmt_gently(int fd, const char *fmt, ...) __attribute__((format (printf, 2, 3)));
/*
--
2.10.0
^ permalink raw reply related
* [PATCH v7 05/10] pkt-line: add packet_write_gently()
From: larsxschneider @ 2016-09-08 18:21 UTC (permalink / raw)
To: git
Cc: peff, gitster, sbeller, Johannes.Schindelin, jnareb, mlbright,
tboegi, jacob.keller, Lars Schneider
In-Reply-To: <20160908182132.50788-1-larsxschneider@gmail.com>
From: Lars Schneider <larsxschneider@gmail.com>
packet_write_fmt_gently() uses format_packet() which lets the caller
only send string data via "%s". That means it cannot be used for
arbitrary data that may contain NULs.
Add packet_write_gently() which writes arbitrary data and returns `0`
for success and `-1` for an error. This function is used by other
pkt-line functions in a subsequent patch.
Signed-off-by: Lars Schneider <larsxschneider@gmail.com>
---
pkt-line.c | 19 +++++++++++++++++++
1 file changed, 19 insertions(+)
diff --git a/pkt-line.c b/pkt-line.c
index 37345ca..1d3d725 100644
--- a/pkt-line.c
+++ b/pkt-line.c
@@ -181,6 +181,25 @@ int packet_write_fmt_gently(int fd, const char *fmt, ...)
return status;
}
+int packet_write_gently(const int fd_out, const char *buf, size_t size)
+{
+ static char packet_write_buffer[LARGE_PACKET_MAX];
+
+ if (size > sizeof(packet_write_buffer) - 4) {
+ error("packet write failed");
+ return -1;
+ }
+ packet_trace(buf, size, 1);
+ size += 4;
+ set_packet_header(packet_write_buffer, size);
+ memcpy(packet_write_buffer + 4, buf, size - 4);
+ if (write_in_full(fd_out, packet_write_buffer, size) == size)
+ return 0;
+
+ error("packet write failed");
+ return -1;
+}
+
void packet_buf_write(struct strbuf *buf, const char *fmt, ...)
{
va_list args;
--
2.10.0
^ permalink raw reply related
* [PATCH v7 06/10] pkt-line: add functions to read/write flush terminated packet streams
From: larsxschneider @ 2016-09-08 18:21 UTC (permalink / raw)
To: git
Cc: peff, gitster, sbeller, Johannes.Schindelin, jnareb, mlbright,
tboegi, jacob.keller, Lars Schneider
In-Reply-To: <20160908182132.50788-1-larsxschneider@gmail.com>
From: Lars Schneider <larsxschneider@gmail.com>
write_packetized_from_fd() and write_packetized_from_buf() write a
stream of packets. All content packets use the maximal packet size
except for the last one. After the last content packet a `flush` control
packet is written.
read_packetized_to_buf() reads arbitrary sized packets until it detects
a `flush` packet.
Signed-off-by: Lars Schneider <larsxschneider@gmail.com>
---
pkt-line.c | 68 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
pkt-line.h | 7 +++++++
2 files changed, 75 insertions(+)
diff --git a/pkt-line.c b/pkt-line.c
index 1d3d725..5001a07 100644
--- a/pkt-line.c
+++ b/pkt-line.c
@@ -209,6 +209,47 @@ void packet_buf_write(struct strbuf *buf, const char *fmt, ...)
va_end(args);
}
+int write_packetized_from_fd(int fd_in, int fd_out)
+{
+ static char buf[PKTLINE_DATA_MAXLEN];
+ int err = 0;
+ ssize_t bytes_to_write;
+
+ while (!err) {
+ bytes_to_write = xread(fd_in, buf, sizeof(buf));
+ if (bytes_to_write < 0)
+ return COPY_READ_ERROR;
+ if (bytes_to_write == 0)
+ break;
+ err = packet_write_gently(fd_out, buf, bytes_to_write);
+ }
+ if (!err)
+ err = packet_flush_gently(fd_out);
+ return err;
+}
+
+int write_packetized_from_buf(const char *src_in, size_t len, int fd_out)
+{
+ static char buf[PKTLINE_DATA_MAXLEN];
+ int err = 0;
+ size_t bytes_written = 0;
+ size_t bytes_to_write;
+
+ while (!err) {
+ if ((len - bytes_written) > sizeof(buf))
+ bytes_to_write = sizeof(buf);
+ else
+ bytes_to_write = len - bytes_written;
+ if (bytes_to_write == 0)
+ break;
+ err = packet_write_gently(fd_out, src_in + bytes_written, bytes_to_write);
+ bytes_written += bytes_to_write;
+ }
+ if (!err)
+ err = packet_flush_gently(fd_out);
+ return err;
+}
+
static int get_packet_data(int fd, char **src_buf, size_t *src_size,
void *dst, unsigned size, int options)
{
@@ -318,3 +359,30 @@ char *packet_read_line_buf(char **src, size_t *src_len, int *dst_len)
{
return packet_read_line_generic(-1, src, src_len, dst_len);
}
+
+ssize_t read_packetized_to_buf(int fd_in, struct strbuf *sb_out)
+{
+ int paket_len;
+ int options = PACKET_READ_GENTLE_ON_EOF;
+
+ size_t oldlen = sb_out->len;
+ size_t oldalloc = sb_out->alloc;
+
+ for (;;) {
+ strbuf_grow(sb_out, PKTLINE_DATA_MAXLEN+1);
+ paket_len = packet_read(fd_in, NULL, NULL,
+ sb_out->buf + sb_out->len, PKTLINE_DATA_MAXLEN+1, options);
+ if (paket_len <= 0)
+ break;
+ sb_out->len += paket_len;
+ }
+
+ if (paket_len < 0) {
+ if (oldalloc == 0)
+ strbuf_release(sb_out);
+ else
+ strbuf_setlen(sb_out, oldlen);
+ return paket_len;
+ }
+ return sb_out->len - oldlen;
+}
diff --git a/pkt-line.h b/pkt-line.h
index 3fa0899..6df8449 100644
--- a/pkt-line.h
+++ b/pkt-line.h
@@ -25,6 +25,8 @@ void packet_buf_flush(struct strbuf *buf);
void packet_buf_write(struct strbuf *buf, const char *fmt, ...) __attribute__((format (printf, 2, 3)));
int packet_flush_gently(int fd);
int packet_write_fmt_gently(int fd, const char *fmt, ...) __attribute__((format (printf, 2, 3)));
+int write_packetized_from_fd(int fd_in, int fd_out);
+int write_packetized_from_buf(const char *src_in, size_t len, int fd_out);
/*
* Read a packetized line into the buffer, which must be at least size bytes
@@ -77,6 +79,11 @@ char *packet_read_line(int fd, int *size);
*/
char *packet_read_line_buf(char **src_buf, size_t *src_len, int *size);
+/*
+ * Reads a stream of variable sized packets until a flush packet is detected.
+ */
+ssize_t read_packetized_to_buf(int fd_in, struct strbuf *sb_out);
+
#define DEFAULT_PACKET_MAX 1000
#define LARGE_PACKET_MAX 65520
extern char packet_buffer[LARGE_PACKET_MAX];
--
2.10.0
^ permalink raw reply related
* [PATCH v7 09/10] convert: make apply_filter() adhere to standard Git error handling
From: larsxschneider @ 2016-09-08 18:21 UTC (permalink / raw)
To: git
Cc: peff, gitster, sbeller, Johannes.Schindelin, jnareb, mlbright,
tboegi, jacob.keller, Lars Schneider
In-Reply-To: <20160908182132.50788-1-larsxschneider@gmail.com>
From: Lars Schneider <larsxschneider@gmail.com>
apply_filter() returns a boolean that tells the caller if it
"did convert or did not convert". The variable `ret` was used throughout
the function to track errors whereas `1` denoted success and `0`
failure. This is unusual for the Git source where `0` denotes success.
Rename the variable and flip its value to make the function easier
readable for Git developers.
Signed-off-by: Lars Schneider <larsxschneider@gmail.com>
---
convert.c | 12 ++++++------
1 file changed, 6 insertions(+), 6 deletions(-)
diff --git a/convert.c b/convert.c
index 986c239..a068df7 100644
--- a/convert.c
+++ b/convert.c
@@ -451,7 +451,7 @@ static int apply_filter(const char *path, const char *src, size_t len, int fd,
*
* (child --> cmd) --> us
*/
- int ret = 1;
+ int err = 0;
struct strbuf nbuf = STRBUF_INIT;
struct async async;
struct filter_params params;
@@ -478,22 +478,22 @@ static int apply_filter(const char *path, const char *src, size_t len, int fd,
if (strbuf_read(&nbuf, async.out, len) < 0) {
error("read from external filter '%s' failed", cmd);
- ret = 0;
+ err = -1;
}
if (close(async.out)) {
error("read from external filter '%s' failed", cmd);
- ret = 0;
+ err = -1;
}
if (finish_async(&async)) {
error("external filter '%s' failed", cmd);
- ret = 0;
+ err = -1;
}
- if (ret) {
+ if (!err) {
strbuf_swap(dst, &nbuf);
}
strbuf_release(&nbuf);
- return ret;
+ return !err;
}
static struct convert_driver {
--
2.10.0
^ permalink raw reply related
* [PATCH v7 08/10] convert: modernize tests
From: larsxschneider @ 2016-09-08 18:21 UTC (permalink / raw)
To: git
Cc: peff, gitster, sbeller, Johannes.Schindelin, jnareb, mlbright,
tboegi, jacob.keller, Lars Schneider
In-Reply-To: <20160908182132.50788-1-larsxschneider@gmail.com>
From: Lars Schneider <larsxschneider@gmail.com>
Use `test_config` to set the config, check that files are empty with
`test_must_be_empty`, compare files with `test_cmp`, and remove spaces
after ">" and "<".
Please note that the "rot13" filter configured in "setup" keeps using
`git config` instead of `test_config` because subsequent tests might
depend on it.
Signed-off-by: Lars Schneider <larsxschneider@gmail.com>
---
t/t0021-conversion.sh | 58 +++++++++++++++++++++++++--------------------------
1 file changed, 29 insertions(+), 29 deletions(-)
diff --git a/t/t0021-conversion.sh b/t/t0021-conversion.sh
index e799e59..dc50938 100755
--- a/t/t0021-conversion.sh
+++ b/t/t0021-conversion.sh
@@ -38,8 +38,8 @@ script='s/^\$Id: \([0-9a-f]*\) \$/\1/p'
test_expect_success check '
- cmp test.o test &&
- cmp test.o test.t &&
+ test_cmp test.o test &&
+ test_cmp test.o test.t &&
# ident should be stripped in the repository
git diff --raw --exit-code :test :test.i &&
@@ -47,10 +47,10 @@ test_expect_success check '
embedded=$(sed -ne "$script" test.i) &&
test "z$id" = "z$embedded" &&
- git cat-file blob :test.t > test.r &&
+ git cat-file blob :test.t >test.r &&
- ./rot13.sh < test.o > test.t &&
- cmp test.r test.t
+ ./rot13.sh <test.o >test.t &&
+ test_cmp test.r test.t
'
# If an expanded ident ever gets into the repository, we want to make sure that
@@ -130,7 +130,7 @@ test_expect_success 'filter shell-escaped filenames' '
# delete the files and check them out again, using a smudge filter
# that will count the args and echo the command-line back to us
- git config filter.argc.smudge "sh ./argc.sh %f" &&
+ test_config filter.argc.smudge "sh ./argc.sh %f" &&
rm "$normal" "$special" &&
git checkout -- "$normal" "$special" &&
@@ -141,7 +141,7 @@ test_expect_success 'filter shell-escaped filenames' '
test_cmp expect "$special" &&
# do the same thing, but with more args in the filter expression
- git config filter.argc.smudge "sh ./argc.sh %f --my-extra-arg" &&
+ test_config filter.argc.smudge "sh ./argc.sh %f --my-extra-arg" &&
rm "$normal" "$special" &&
git checkout -- "$normal" "$special" &&
@@ -154,9 +154,9 @@ test_expect_success 'filter shell-escaped filenames' '
'
test_expect_success 'required filter should filter data' '
- git config filter.required.smudge ./rot13.sh &&
- git config filter.required.clean ./rot13.sh &&
- git config filter.required.required true &&
+ test_config filter.required.smudge ./rot13.sh &&
+ test_config filter.required.clean ./rot13.sh &&
+ test_config filter.required.required true &&
echo "*.r filter=required" >.gitattributes &&
@@ -165,17 +165,17 @@ test_expect_success 'required filter should filter data' '
rm -f test.r &&
git checkout -- test.r &&
- cmp test.o test.r &&
+ test_cmp test.o test.r &&
./rot13.sh <test.o >expected &&
git cat-file blob :test.r >actual &&
- cmp expected actual
+ test_cmp expected actual
'
test_expect_success 'required filter smudge failure' '
- git config filter.failsmudge.smudge false &&
- git config filter.failsmudge.clean cat &&
- git config filter.failsmudge.required true &&
+ test_config filter.failsmudge.smudge false &&
+ test_config filter.failsmudge.clean cat &&
+ test_config filter.failsmudge.required true &&
echo "*.fs filter=failsmudge" >.gitattributes &&
@@ -186,9 +186,9 @@ test_expect_success 'required filter smudge failure' '
'
test_expect_success 'required filter clean failure' '
- git config filter.failclean.smudge cat &&
- git config filter.failclean.clean false &&
- git config filter.failclean.required true &&
+ test_config filter.failclean.smudge cat &&
+ test_config filter.failclean.clean false &&
+ test_config filter.failclean.required true &&
echo "*.fc filter=failclean" >.gitattributes &&
@@ -197,8 +197,8 @@ test_expect_success 'required filter clean failure' '
'
test_expect_success 'filtering large input to small output should use little memory' '
- git config filter.devnull.clean "cat >/dev/null" &&
- git config filter.devnull.required true &&
+ test_config filter.devnull.clean "cat >/dev/null" &&
+ test_config filter.devnull.required true &&
for i in $(test_seq 1 30); do printf "%1048576d" 1; done >30MB &&
echo "30MB filter=devnull" >.gitattributes &&
GIT_MMAP_LIMIT=1m GIT_ALLOC_LIMIT=1m git add 30MB
@@ -207,7 +207,7 @@ test_expect_success 'filtering large input to small output should use little mem
test_expect_success 'filter that does not read is fine' '
test-genrandom foo $((128 * 1024 + 1)) >big &&
echo "big filter=epipe" >.gitattributes &&
- git config filter.epipe.clean "echo xyzzy" &&
+ test_config filter.epipe.clean "echo xyzzy" &&
git add big &&
git cat-file blob :big >actual &&
echo xyzzy >expect &&
@@ -215,20 +215,20 @@ test_expect_success 'filter that does not read is fine' '
'
test_expect_success EXPENSIVE 'filter large file' '
- git config filter.largefile.smudge cat &&
- git config filter.largefile.clean cat &&
+ test_config filter.largefile.smudge cat &&
+ test_config filter.largefile.clean cat &&
for i in $(test_seq 1 2048); do printf "%1048576d" 1; done >2GB &&
echo "2GB filter=largefile" >.gitattributes &&
git add 2GB 2>err &&
- ! test -s err &&
+ test_must_be_empty err &&
rm -f 2GB &&
git checkout -- 2GB 2>err &&
- ! test -s err
+ test_must_be_empty err
'
test_expect_success "filter: clean empty file" '
- git config filter.in-repo-header.clean "echo cleaned && cat" &&
- git config filter.in-repo-header.smudge "sed 1d" &&
+ test_config filter.in-repo-header.clean "echo cleaned && cat" &&
+ test_config filter.in-repo-header.smudge "sed 1d" &&
echo "empty-in-worktree filter=in-repo-header" >>.gitattributes &&
>empty-in-worktree &&
@@ -240,8 +240,8 @@ test_expect_success "filter: clean empty file" '
'
test_expect_success "filter: smudge empty file" '
- git config filter.empty-in-repo.clean "cat >/dev/null" &&
- git config filter.empty-in-repo.smudge "echo smudged && cat" &&
+ test_config filter.empty-in-repo.clean "cat >/dev/null" &&
+ test_config filter.empty-in-repo.smudge "echo smudged && cat" &&
echo "empty-in-repo filter=empty-in-repo" >>.gitattributes &&
echo dead data walking >empty-in-repo &&
--
2.10.0
^ permalink raw reply related
* [PATCH v7 07/10] convert: quote filter names in error messages
From: larsxschneider @ 2016-09-08 18:21 UTC (permalink / raw)
To: git
Cc: peff, gitster, sbeller, Johannes.Schindelin, jnareb, mlbright,
tboegi, jacob.keller, Lars Schneider
In-Reply-To: <20160908182132.50788-1-larsxschneider@gmail.com>
From: Lars Schneider <larsxschneider@gmail.com>
Git filter driver commands with spaces (e.g. `filter.sh foo`) are hard to
read in error messages. Quote them to improve the readability.
Signed-off-by: Lars Schneider <larsxschneider@gmail.com>
---
convert.c | 12 ++++++------
1 file changed, 6 insertions(+), 6 deletions(-)
diff --git a/convert.c b/convert.c
index 077f5e6..986c239 100644
--- a/convert.c
+++ b/convert.c
@@ -412,7 +412,7 @@ static int filter_buffer_or_fd(int in, int out, void *data)
child_process.out = out;
if (start_command(&child_process))
- return error("cannot fork to run external filter %s", params->cmd);
+ return error("cannot fork to run external filter '%s'", params->cmd);
sigchain_push(SIGPIPE, SIG_IGN);
@@ -430,13 +430,13 @@ static int filter_buffer_or_fd(int in, int out, void *data)
if (close(child_process.in))
write_err = 1;
if (write_err)
- error("cannot feed the input to external filter %s", params->cmd);
+ error("cannot feed the input to external filter '%s'", params->cmd);
sigchain_pop(SIGPIPE);
status = finish_command(&child_process);
if (status)
- error("external filter %s failed %d", params->cmd, status);
+ error("external filter '%s' failed %d", params->cmd, status);
strbuf_release(&cmd);
return (write_err || status);
@@ -477,15 +477,15 @@ static int apply_filter(const char *path, const char *src, size_t len, int fd,
return 0; /* error was already reported */
if (strbuf_read(&nbuf, async.out, len) < 0) {
- error("read from external filter %s failed", cmd);
+ error("read from external filter '%s' failed", cmd);
ret = 0;
}
if (close(async.out)) {
- error("read from external filter %s failed", cmd);
+ error("read from external filter '%s' failed", cmd);
ret = 0;
}
if (finish_async(&async)) {
- error("external filter %s failed", cmd);
+ error("external filter '%s' failed", cmd);
ret = 0;
}
--
2.10.0
^ permalink raw reply related
* [PATCH v7 10/10] convert: add filter.<driver>.process option
From: larsxschneider @ 2016-09-08 18:21 UTC (permalink / raw)
To: git
Cc: peff, gitster, sbeller, Johannes.Schindelin, jnareb, mlbright,
tboegi, jacob.keller, Lars Schneider
In-Reply-To: <20160908182132.50788-1-larsxschneider@gmail.com>
From: Lars Schneider <larsxschneider@gmail.com>
Git's clean/smudge mechanism invokes an external filter process for
every single blob that is affected by a filter. If Git filters a lot of
blobs then the startup time of the external filter processes can become
a significant part of the overall Git execution time.
In a preliminary performance test this developer used a clean/smudge
filter written in golang to filter 12,000 files. This process took 364s
with the existing filter mechanism and 5s with the new mechanism. See
details here: https://github.com/github/git-lfs/pull/1382
This patch adds the `filter.<driver>.process` string option which, if
used, keeps the external filter process running and processes all blobs
with the packet format (pkt-line) based protocol over standard input and
standard output. The full protocol is explained in detail in
`Documentation/gitattributes.txt`.
A few key decisions:
* The long running filter process is referred to as filter protocol
version 2 because the existing single shot filter invocation is
considered version 1.
* Git sends a welcome message and expects a response right after the
external filter process has started. This ensures that Git will not
hang if a version 1 filter is incorrectly used with the
filter.<driver>.process option for version 2 filters. In addition,
Git can detect this kind of error and warn the user.
* The status of a filter operation (e.g. "success" or "error) is set
before the actual response and (if necessary!) re-set after the
response. The advantage of this two step status response is that if
the filter detects an error early, then the filter can communicate
this and Git does not even need to create structures to read the
response.
* All status responses are pkt-line lists terminated with a flush
packet. This allows us to send other status fields with the same
protocol in the future.
Helped-by: Martin-Louis Bright <mlbright@gmail.com>
Reviewed-by: Jakub Narebski <jnareb@gmail.com>
Signed-off-by: Lars Schneider <larsxschneider@gmail.com>
---
Documentation/gitattributes.txt | 154 +++++++++++++-
contrib/long-running-filter/example.pl | 111 ++++++++++
convert.c | 349 ++++++++++++++++++++++++++++---
pkt-line.h | 1 +
t/t0021-conversion.sh | 365 ++++++++++++++++++++++++++++++++-
t/t0021/rot13-filter.pl | 179 ++++++++++++++++
unpack-trees.c | 1 +
7 files changed, 1129 insertions(+), 31 deletions(-)
create mode 100755 contrib/long-running-filter/example.pl
create mode 100755 t/t0021/rot13-filter.pl
diff --git a/Documentation/gitattributes.txt b/Documentation/gitattributes.txt
index 7aff940..ac000ea 100644
--- a/Documentation/gitattributes.txt
+++ b/Documentation/gitattributes.txt
@@ -293,7 +293,13 @@ checkout, when the `smudge` command is specified, the command is
fed the blob object from its standard input, and its standard
output is used to update the worktree file. Similarly, the
`clean` command is used to convert the contents of worktree file
-upon checkin.
+upon checkin. By default these commands process only a single
+blob and terminate. If a long running `process` filter is used
+in place of `clean` and/or `smudge` filters, then Git can process
+all blobs with a single filter command invocation for the entire
+life of a single Git command, for example `git add --all`. See
+section below for the description of the protocol used to
+communicate with a `process` filter.
One use of the content filtering is to massage the content into a shape
that is more convenient for the platform, filesystem, and the user to use.
@@ -373,6 +379,152 @@ not exist, or may have different contents. So, smudge and clean commands
should not try to access the file on disk, but only act as filters on the
content provided to them on standard input.
+Long Running Filter Process
+^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+If the filter command (a string value) is defined via
+`filter.<driver>.process` then Git can process all blobs with a
+single filter invocation for the entire life of a single Git
+command. This is achieved by using the following packet format
+(pkt-line, see technical/protocol-common.txt) based protocol over
+standard input and standard output.
+
+Git starts the filter when it encounters the first file
+that needs to be cleaned or smudged. After the filter started
+Git sends a welcome message ("git-filter-client"), a list of
+supported protocol version numbers, and a flush packet. Git expects
+to read a welcome response message ("git-filter-server") and exactly
+one protocol version number from the previously sent list. All further
+communication will be based on the selected version. The remaining
+protocol description below documents "version=2". Please note that
+"version=42" in the example below does not exist and is only there
+to illustrate how the protocol would look like with more than one
+version.
+
+After the version negotiation Git sends a list of supported capabilities
+and a flush packet. Git expects to read a list of desired capabilities,
+which must be a subset of the supported capabilities list, and a flush
+packet as response:
+------------------------
+packet: git> git-filter-client
+packet: git> version=2
+packet: git> version=42
+packet: git> 0000
+packet: git< git-filter-server
+packet: git< version=2
+packet: git> clean=true
+packet: git> smudge=true
+packet: git> not-yet-invented=true
+packet: git> 0000
+packet: git< clean=true
+packet: git< smudge=true
+packet: git< 0000
+------------------------
+Supported filter capabilities in version 2 are "clean" and
+"smudge".
+
+Afterwards Git sends a list of "key=value" pairs terminated with
+a flush packet. The list will contain at least the filter command
+(based on the supported capabilities) and the pathname of the file
+to filter relative to the repository root. Right after these packets
+Git sends the content split in zero or more pkt-line packets and a
+flush packet to terminate content.
+------------------------
+packet: git> command=smudge\n
+packet: git> pathname=path/testfile.dat\n
+packet: git> 0000
+packet: git> CONTENT
+packet: git> 0000
+------------------------
+
+The filter is expected to respond with a list of "key=value" pairs
+terminated with a flush packet. If the filter does not experience
+problems then the list must contain a "success" status. Right after
+these packets the filter is expected to send the content in zero
+or more pkt-line packets and a flush packet at the end. Finally, a
+second list of "key=value" pairs terminated with a flush packet
+is expected. The filter can change the status in the second list.
+------------------------
+packet: git< status=success\n
+packet: git< 0000
+packet: git< SMUDGED_CONTENT
+packet: git< 0000
+packet: git< 0000 # empty list!
+------------------------
+
+If the result content is empty then the filter is expected to respond
+with a success status and an empty list.
+------------------------
+packet: git< status=success\n
+packet: git< 0000
+packet: git< 0000 # empty content!
+packet: git< 0000 # empty list!
+------------------------
+
+In case the filter cannot or does not want to process the content,
+it is expected to respond with an "error" status. Depending on the
+`filter.<driver>.required` flag Git will interpret that as error
+but it will not stop or restart the filter process.
+------------------------
+packet: git< status=error\n
+packet: git< 0000
+------------------------
+
+If the filter experiences an error during processing, then it can
+send the status "error" after the content was (partially or
+completely) sent. Depending on the `filter.<driver>.required` flag
+Git will interpret that as error but it will not stop or restart the
+filter process.
+------------------------
+packet: git< status=success\n
+packet: git< 0000
+packet: git< HALF_WRITTEN_ERRONEOUS_CONTENT
+packet: git< 0000
+packet: git< status=error\n
+packet: git< 0000
+------------------------
+
+If the filter dies during the communication or does not adhere to
+the protocol then Git will stop the filter process and restart it
+with the next file that needs to be processed. Depending on the
+`filter.<driver>.required` flag Git will interpret that as error.
+
+The error handling for all cases above mimic the behavior of
+the `filter.<driver>.clean` / `filter.<driver>.smudge` error
+handling.
+
+In case the filter cannot or does not want to process the content
+as well as any future content for the lifetime of the Git process,
+it is expected to respond with an "abort" status. Depending on
+the `filter.<driver>.required` flag Git will interpret that as error
+for the content as well as any future content for the lifetime of the
+Git process but it will not stop or restart the filter process.
+------------------------
+packet: git< status=abort\n
+packet: git< 0000
+------------------------
+
+After the filter has processed a blob it is expected to wait for
+the next "key=value" list containing a command. Git will close
+the command pipe on exit. The filter is expected to detect EOF
+and exit gracefully on its own.
+
+A long running filter demo implementation can be found in
+`contrib/long-running-filter/example.pl` located in the Git
+core repository. If you develop your own long running filter
+process then the `GIT_TRACE_PACKET` environment variables can be
+very helpful for debugging (see linkgit:git[1]).
+
+If a `filter.<driver>.process` command is configured then it
+always takes precedence over a configured `filter.<driver>.clean`
+or `filter.<driver>.smudge` command.
+
+Please note that you cannot use an existing `filter.<driver>.clean`
+or `filter.<driver>.smudge` command with `filter.<driver>.process`
+because the former two use a different inter process communication
+protocol than the latter one.
+
+
Interaction between checkin/checkout attributes
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
diff --git a/contrib/long-running-filter/example.pl b/contrib/long-running-filter/example.pl
new file mode 100755
index 0000000..279fbfb
--- /dev/null
+++ b/contrib/long-running-filter/example.pl
@@ -0,0 +1,111 @@
+#!/usr/bin/perl
+#
+# Example implementation for the Git filter protocol version 2
+# See Documentation/gitattributes.txt, section "Filter Protocol"
+#
+
+use strict;
+use warnings;
+
+my $MAX_PACKET_CONTENT_SIZE = 65516;
+
+sub packet_read {
+ my $buffer;
+ my $bytes_read = read STDIN, $buffer, 4;
+ if ( $bytes_read == 0 ) {
+
+ # EOF - Git stopped talking to us!
+ exit();
+ }
+ elsif ( $bytes_read != 4 ) {
+ die "invalid packet size '$bytes_read' field";
+ }
+ my $pkt_size = hex($buffer);
+ if ( $pkt_size == 0 ) {
+ return ( 1, "" );
+ }
+ elsif ( $pkt_size > 4 ) {
+ my $content_size = $pkt_size - 4;
+ $bytes_read = read STDIN, $buffer, $content_size;
+ if ( $bytes_read != $content_size ) {
+ die "invalid packet ($content_size expected; $bytes_read read)";
+ }
+ return ( 0, $buffer );
+ }
+ else {
+ die "invalid packet size";
+ }
+}
+
+sub packet_write {
+ my ($packet) = @_;
+ print STDOUT sprintf( "%04x", length($packet) + 4 );
+ print STDOUT $packet;
+ STDOUT->flush();
+}
+
+sub packet_flush {
+ print STDOUT sprintf( "%04x", 0 );
+ STDOUT->flush();
+}
+
+( packet_read() eq ( 0, "git-filter-client" ) ) || die "bad initialization";
+( packet_read() eq ( 0, "version=2" ) ) || die "bad version";
+( packet_read() eq ( 1, "" ) ) || die "bad version end";
+
+packet_write("git-filter-server\n");
+packet_write("version=2\n");
+
+( packet_read() eq ( 0, "clean=true" ) ) || die "bad capability";
+( packet_read() eq ( 0, "smudge=true" ) ) || die "bad capability";
+( packet_read() eq ( 1, "" ) ) || die "bad capability end";
+
+packet_write( "clean=true\n" );
+packet_write( "smudge=true\n" );
+packet_flush();
+
+while (1) {
+ my ($command) = packet_read() =~ /^command=([^=]+)\n$/;
+ my ($pathname) = packet_read() =~ /^pathname=([^=]+)\n$/;
+
+ packet_read();
+
+ my $input = "";
+ {
+ binmode(STDIN);
+ my $buffer;
+ my $done = 0;
+ while ( !$done ) {
+ ( $done, $buffer ) = packet_read();
+ $input .= $buffer;
+ }
+ }
+
+ my $output;
+ if ( $command eq "clean" ) {
+ ### Perform clean here ###
+ $output = $input;
+ }
+ elsif ( $command eq "smudge" ) {
+ ### Perform smudge here ###
+ $output = $input;
+ }
+ else {
+ die "bad command '$command'";
+ }
+
+ packet_write("status=success\n");
+ packet_flush();
+ while ( length($output) > 0 ) {
+ my $packet = substr( $output, 0, $MAX_PACKET_CONTENT_SIZE );
+ packet_write($packet);
+ if ( length($output) > $MAX_PACKET_CONTENT_SIZE ) {
+ $output = substr( $output, $MAX_PACKET_CONTENT_SIZE );
+ }
+ else {
+ $output = "";
+ }
+ }
+ packet_flush(); # flush content!
+ packet_flush(); # empty list!
+}
diff --git a/convert.c b/convert.c
index a068df7..0ed48ed 100644
--- a/convert.c
+++ b/convert.c
@@ -3,6 +3,7 @@
#include "run-command.h"
#include "quote.h"
#include "sigchain.h"
+#include "pkt-line.h"
/*
* convert.c - convert a file when checking it out and checking it in.
@@ -442,7 +443,7 @@ static int filter_buffer_or_fd(int in, int out, void *data)
return (write_err || status);
}
-static int apply_filter(const char *path, const char *src, size_t len, int fd,
+static int apply_single_file_filter(const char *path, const char *src, size_t len, int fd,
struct strbuf *dst, const char *cmd)
{
/*
@@ -456,12 +457,6 @@ static int apply_filter(const char *path, const char *src, size_t len, int fd,
struct async async;
struct filter_params params;
- if (!cmd || !*cmd)
- return 0;
-
- if (!dst)
- return 1;
-
memset(&async, 0, sizeof(async));
async.proc = filter_buffer_or_fd;
async.data = ¶ms;
@@ -496,14 +491,318 @@ static int apply_filter(const char *path, const char *src, size_t len, int fd,
return !err;
}
+#define CAP_CLEAN (1u<<0)
+#define CAP_SMUDGE (1u<<1)
+
+struct cmd2process {
+ struct hashmap_entry ent; /* must be the first member! */
+ int supported_capabilities;
+ const char *cmd;
+ struct child_process process;
+};
+
+static int cmd_process_map_initialized;
+static struct hashmap cmd_process_map;
+
+static int cmd2process_cmp(const struct cmd2process *e1,
+ const struct cmd2process *e2,
+ const void *unused)
+{
+ return strcmp(e1->cmd, e2->cmd);
+}
+
+static struct cmd2process *find_multi_file_filter_entry(struct hashmap *hashmap, const char *cmd)
+{
+ struct cmd2process key;
+ hashmap_entry_init(&key, strhash(cmd));
+ key.cmd = cmd;
+ return hashmap_get(hashmap, &key, NULL);
+}
+
+static void kill_multi_file_filter(struct hashmap *hashmap, struct cmd2process *entry)
+{
+ if (!entry)
+ return;
+ sigchain_push(SIGPIPE, SIG_IGN);
+ /*
+ * We kill the filter most likely because an error happened already.
+ * That's why we are not interested in any error code here.
+ */
+ close(entry->process.in);
+ close(entry->process.out);
+ sigchain_pop(SIGPIPE);
+ finish_command(&entry->process);
+ hashmap_remove(hashmap, entry, NULL);
+ free(entry);
+}
+
+static int packet_write_list(int fd, const char *line, ...)
+{
+ va_list args;
+ int err;
+ va_start(args, line);
+ for (;;)
+ {
+ if (!line)
+ break;
+ if (strlen(line) > PKTLINE_DATA_MAXLEN)
+ return -1;
+ err = packet_write_fmt_gently(fd, "%s", line);
+ if (err)
+ return err;
+ line = va_arg(args, const char*);
+ }
+ va_end(args);
+ return packet_flush_gently(fd);
+}
+
+static struct cmd2process *start_multi_file_filter(struct hashmap *hashmap, const char *cmd)
+{
+ int err;
+ struct cmd2process *entry;
+ struct child_process *process;
+ const char *argv[] = { cmd, NULL };
+ struct string_list cap_list = STRING_LIST_INIT_NODUP;
+ char *cap_buf;
+ const char *cap_name;
+
+ entry = xmalloc(sizeof(*entry));
+ hashmap_entry_init(entry, strhash(cmd));
+ entry->cmd = cmd;
+ entry->supported_capabilities = 0;
+ process = &entry->process;
+
+ child_process_init(process);
+ process->argv = argv;
+ process->use_shell = 1;
+ process->in = -1;
+ process->out = -1;
+
+ if (start_command(process)) {
+ error("cannot fork to run external filter '%s'", cmd);
+ kill_multi_file_filter(hashmap, entry);
+ return NULL;
+ }
+
+ sigchain_push(SIGPIPE, SIG_IGN);
+
+ err = packet_write_list(process->in, "git-filter-client", "version=2", NULL);
+ if (err)
+ goto done;
+
+ err = strcmp(packet_read_line(process->out, NULL), "git-filter-server");
+ if (err) {
+ error("external filter '%s' does not support long running filter protocol", cmd);
+ goto done;
+ }
+ err = strcmp(packet_read_line(process->out, NULL), "version=2");
+ if (err)
+ goto done;
+
+ err = packet_write_list(process->in, "clean=true", "smudge=true", NULL);
+
+ for (;;)
+ {
+ cap_buf = packet_read_line(process->out, NULL);
+ if (!cap_buf)
+ break;
+ string_list_split_in_place(&cap_list, cap_buf, '=', 1);
+
+ if (cap_list.nr != 2 || strcmp(cap_list.items[1].string, "true"))
+ continue;
+
+ cap_name = cap_list.items[0].string;
+ if (!strcmp(cap_name, "clean")) {
+ entry->supported_capabilities |= CAP_CLEAN;
+ } else if (!strcmp(cap_name, "smudge")) {
+ entry->supported_capabilities |= CAP_SMUDGE;
+ } else {
+ warning(
+ "external filter '%s' requested unsupported filter capability '%s'",
+ cmd, cap_name
+ );
+ }
+
+ string_list_clear(&cap_list, 0);
+ }
+
+done:
+ sigchain_pop(SIGPIPE);
+
+ if (err || errno == EPIPE) {
+ error("initialization for external filter '%s' failed", cmd);
+ kill_multi_file_filter(hashmap, entry);
+ return NULL;
+ }
+
+ hashmap_add(hashmap, entry);
+ return entry;
+}
+
+static void read_multi_file_filter_values(int fd, struct strbuf *status) {
+ struct strbuf **pair;
+ char *line;
+ for (;;) {
+ line = packet_read_line(fd, NULL);
+ if (!line)
+ break;
+ pair = strbuf_split_str(line, '=', 2);
+ if (pair[0] && pair[0]->len && pair[1]) {
+ if (!strcmp(pair[0]->buf, "status=")) {
+ strbuf_reset(status);
+ strbuf_addbuf(status, pair[1]);
+ }
+ }
+ }
+}
+
+static int apply_multi_file_filter(const char *path, const char *src, size_t len,
+ int fd, struct strbuf *dst, const char *cmd,
+ const int wanted_capability)
+{
+ int err;
+ struct cmd2process *entry;
+ struct child_process *process;
+ struct stat file_stat;
+ struct strbuf nbuf = STRBUF_INIT;
+ struct strbuf filter_status = STRBUF_INIT;
+ char *filter_type;
+
+ if (!cmd_process_map_initialized) {
+ cmd_process_map_initialized = 1;
+ hashmap_init(&cmd_process_map, (hashmap_cmp_fn) cmd2process_cmp, 0);
+ entry = NULL;
+ } else {
+ entry = find_multi_file_filter_entry(&cmd_process_map, cmd);
+ }
+
+ fflush(NULL);
+
+ if (!entry) {
+ entry = start_multi_file_filter(&cmd_process_map, cmd);
+ if (!entry)
+ return 0;
+ }
+ process = &entry->process;
+
+ if (!(wanted_capability & entry->supported_capabilities))
+ return 0;
+
+ if (CAP_CLEAN & wanted_capability)
+ filter_type = "clean";
+ else if (CAP_SMUDGE & wanted_capability)
+ filter_type = "smudge";
+ else
+ die("unexpected filter type");
+
+ if (fd >= 0 && !src) {
+ if (fstat(fd, &file_stat) == -1)
+ return 0;
+ len = xsize_t(file_stat.st_size);
+ }
+
+ sigchain_push(SIGPIPE, SIG_IGN);
+
+
+ err = (strlen(filter_type) > PKTLINE_DATA_MAXLEN);
+ if (err)
+ goto done;
+ err = packet_write_fmt_gently(process->in, "command=%s\n", filter_type);
+ if (err)
+ goto done;
+
+ err = (strlen(path) > PKTLINE_DATA_MAXLEN);
+ if (err)
+ goto done;
+ err = packet_write_fmt_gently(process->in, "pathname=%s\n", path);
+ if (err)
+ goto done;
+
+ err = packet_flush_gently(process->in);
+ if (err)
+ goto done;
+
+ if (fd >= 0)
+ err = write_packetized_from_fd(fd, process->in);
+ else
+ err = write_packetized_from_buf(src, len, process->in);
+ if (err)
+ goto done;
+
+ read_multi_file_filter_values(process->out, &filter_status);
+ err = strcmp(filter_status.buf, "success");
+ if (err)
+ goto done;
+
+ err = read_packetized_to_buf(process->out, &nbuf) < 0;
+ if (err)
+ goto done;
+
+ read_multi_file_filter_values(process->out, &filter_status);
+ err = strcmp(filter_status.buf, "success");
+
+done:
+ sigchain_pop(SIGPIPE);
+
+ if (err || errno == EPIPE) {
+ if (!strcmp(filter_status.buf, "error")) {
+ /* The filter signaled a problem with the file. */
+ } else if (!strcmp(filter_status.buf, "abort")) {
+ /*
+ * The filter signaled a permanent problem. Don't try to filter
+ * files with the same command for the lifetime of the current
+ * Git process.
+ */
+ entry->supported_capabilities &= ~wanted_capability;
+ } else {
+ /*
+ * Something went wrong with the protocol filter.
+ * Force shutdown and restart if another blob requires filtering!
+ */
+ error("external filter '%s' failed", cmd);
+ kill_multi_file_filter(&cmd_process_map, entry);
+ }
+ } else {
+ strbuf_swap(dst, &nbuf);
+ }
+ strbuf_release(&nbuf);
+ return !err;
+}
+
static struct convert_driver {
const char *name;
struct convert_driver *next;
const char *smudge;
const char *clean;
+ const char *process;
int required;
} *user_convert, **user_convert_tail;
+static int apply_filter(const char *path, const char *src, size_t len,
+ int fd, struct strbuf *dst, struct convert_driver *drv,
+ const int wanted_capability)
+{
+ const char* cmd = NULL;
+
+ if (!drv)
+ return 0;
+
+ if (!dst)
+ return 1;
+
+ if (!drv->process && (CAP_CLEAN & wanted_capability) && drv->clean)
+ cmd = drv->clean;
+ else if (!drv->process && (CAP_SMUDGE & wanted_capability) && drv->smudge)
+ cmd = drv->smudge;
+
+ if (cmd && *cmd)
+ return apply_single_file_filter(path, src, len, fd, dst, cmd);
+ else if (drv->process && *drv->process)
+ return apply_multi_file_filter(path, src, len, fd, dst, drv->process, wanted_capability);
+
+ return 0;
+}
+
static int read_convert_config(const char *var, const char *value, void *cb)
{
const char *key, *name;
@@ -541,6 +840,9 @@ static int read_convert_config(const char *var, const char *value, void *cb)
if (!strcmp("clean", key))
return git_config_string(&drv->clean, var, value);
+ if (!strcmp("process", key))
+ return git_config_string(&drv->process, var, value);
+
if (!strcmp("required", key)) {
drv->required = git_config_bool(var, value);
return 0;
@@ -842,7 +1144,7 @@ int would_convert_to_git_filter_fd(const char *path)
if (!ca.drv->required)
return 0;
- return apply_filter(path, NULL, 0, -1, NULL, ca.drv->clean);
+ return apply_filter(path, NULL, 0, -1, NULL, ca.drv, CAP_CLEAN);
}
const char *get_convert_attr_ascii(const char *path)
@@ -875,18 +1177,12 @@ int convert_to_git(const char *path, const char *src, size_t len,
struct strbuf *dst, enum safe_crlf checksafe)
{
int ret = 0;
- const char *filter = NULL;
- int required = 0;
struct conv_attrs ca;
convert_attrs(&ca, path);
- if (ca.drv) {
- filter = ca.drv->clean;
- required = ca.drv->required;
- }
- ret |= apply_filter(path, src, len, -1, dst, filter);
- if (!ret && required)
+ ret |= apply_filter(path, src, len, -1, dst, ca.drv, CAP_CLEAN);
+ if (!ret && ca.drv && ca.drv->required)
die("%s: clean filter '%s' failed", path, ca.drv->name);
if (ret && dst) {
@@ -908,9 +1204,9 @@ void convert_to_git_filter_fd(const char *path, int fd, struct strbuf *dst,
convert_attrs(&ca, path);
assert(ca.drv);
- assert(ca.drv->clean);
+ assert(ca.drv->clean || ca.drv->process);
- if (!apply_filter(path, NULL, 0, fd, dst, ca.drv->clean))
+ if (!apply_filter(path, NULL, 0, fd, dst, ca.drv, CAP_CLEAN))
die("%s: clean filter '%s' failed", path, ca.drv->name);
crlf_to_git(path, dst->buf, dst->len, dst, ca.crlf_action, checksafe);
@@ -922,15 +1218,9 @@ static int convert_to_working_tree_internal(const char *path, const char *src,
int normalizing)
{
int ret = 0, ret_filter = 0;
- const char *filter = NULL;
- int required = 0;
struct conv_attrs ca;
convert_attrs(&ca, path);
- if (ca.drv) {
- filter = ca.drv->smudge;
- required = ca.drv->required;
- }
ret |= ident_to_worktree(path, src, len, dst, ca.ident);
if (ret) {
@@ -939,9 +1229,10 @@ static int convert_to_working_tree_internal(const char *path, const char *src,
}
/*
* CRLF conversion can be skipped if normalizing, unless there
- * is a smudge filter. The filter might expect CRLFs.
+ * is a smudge or process filter (even if the process filter doesn't
+ * support smudge). The filters might expect CRLFs.
*/
- if (filter || !normalizing) {
+ if ((ca.drv && (ca.drv->smudge || ca.drv->process)) || !normalizing) {
ret |= crlf_to_worktree(path, src, len, dst, ca.crlf_action);
if (ret) {
src = dst->buf;
@@ -949,8 +1240,8 @@ static int convert_to_working_tree_internal(const char *path, const char *src,
}
}
- ret_filter = apply_filter(path, src, len, -1, dst, filter);
- if (!ret_filter && required)
+ ret_filter = apply_filter(path, src, len, -1, dst, ca.drv, CAP_SMUDGE);
+ if (!ret_filter && ca.drv && ca.drv->required)
die("%s: smudge filter %s failed", path, ca.drv->name);
return ret | ret_filter;
@@ -1402,7 +1693,7 @@ struct stream_filter *get_stream_filter(const char *path, const unsigned char *s
struct stream_filter *filter = NULL;
convert_attrs(&ca, path);
- if (ca.drv && (ca.drv->smudge || ca.drv->clean))
+ if (ca.drv && (ca.drv->process || ca.drv->smudge || ca.drv->clean))
return NULL;
if (ca.crlf_action == CRLF_AUTO || ca.crlf_action == CRLF_AUTO_CRLF)
diff --git a/pkt-line.h b/pkt-line.h
index 6df8449..3d873f3 100644
--- a/pkt-line.h
+++ b/pkt-line.h
@@ -86,6 +86,7 @@ ssize_t read_packetized_to_buf(int fd_in, struct strbuf *sb_out);
#define DEFAULT_PACKET_MAX 1000
#define LARGE_PACKET_MAX 65520
+#define PKTLINE_DATA_MAXLEN (LARGE_PACKET_MAX - 4)
extern char packet_buffer[LARGE_PACKET_MAX];
#endif
diff --git a/t/t0021-conversion.sh b/t/t0021-conversion.sh
index dc50938..1c98ac3 100755
--- a/t/t0021-conversion.sh
+++ b/t/t0021-conversion.sh
@@ -31,7 +31,10 @@ test_expect_success setup '
cat test >test.i &&
git add test test.t test.i &&
rm -f test test.t test.i &&
- git checkout -- test test.t test.i
+ git checkout -- test test.t test.i &&
+
+ echo "content-test2" >test2.o &&
+ echo "content-test3-subdir" >test3-subdir.o
'
script='s/^\$Id: \([0-9a-f]*\) \$/\1/p'
@@ -279,4 +282,364 @@ test_expect_success 'diff does not reuse worktree files that need cleaning' '
test_line_count = 0 count
'
+check_filter () {
+ rm -f rot13-filter.log actual.log &&
+ "$@" 2> git_stderr.log &&
+ test_must_be_empty git_stderr.log &&
+ cat >expected.log &&
+ sort rot13-filter.log | uniq -c | sed "s/^[ ]*//" >actual.log &&
+ test_cmp expected.log actual.log
+}
+
+check_filter_count_clean () {
+ rm -f rot13-filter.log actual.log &&
+ "$@" 2> git_stderr.log &&
+ test_must_be_empty git_stderr.log &&
+ cat >expected.log &&
+ sort rot13-filter.log | uniq -c | sed "s/^[ ]*//" |
+ sed "s/^\([0-9]\) IN: clean/x IN: clean/" >actual.log &&
+ test_cmp expected.log actual.log
+}
+
+check_filter_ignore_clean () {
+ rm -f rot13-filter.log actual.log &&
+ "$@" &&
+ cat >expected.log &&
+ grep -v "IN: clean" rot13-filter.log >actual.log &&
+ test_cmp expected.log actual.log
+}
+
+check_filter_no_call () {
+ rm -f rot13-filter.log &&
+ "$@" 2> git_stderr.log &&
+ test_must_be_empty git_stderr.log &&
+ test_must_be_empty rot13-filter.log
+}
+
+check_rot13 () {
+ test_cmp $1 $2 &&
+ ./../rot13.sh <$1 >expected &&
+ git cat-file blob :$2 >actual &&
+ test_cmp expected actual
+}
+
+test_expect_success PERL 'required process filter should filter data' '
+ test_config_global filter.protocol.process "$TEST_DIRECTORY/t0021/rot13-filter.pl clean smudge" &&
+ test_config_global filter.protocol.required true &&
+ rm -rf repo &&
+ mkdir repo &&
+ (
+ cd repo &&
+ git init &&
+
+ echo "*.r filter=protocol" >.gitattributes &&
+ git add . &&
+ git commit . -m "test commit" &&
+ git branch empty &&
+
+ cp ../test.o test.r &&
+ cp ../test2.o test2.r &&
+ mkdir testsubdir &&
+ cp ../test3-subdir.o testsubdir/test3-subdir.r &&
+ >test4-empty.r &&
+
+ check_filter \
+ git add . \
+ <<-\EOF &&
+ 1 IN: clean test.r 57 [OK] -- OUT: 57 . [OK]
+ 1 IN: clean test2.r 14 [OK] -- OUT: 14 . [OK]
+ 1 IN: clean test4-empty.r 0 [OK] -- OUT: 0 [OK]
+ 1 IN: clean testsubdir/test3-subdir.r 21 [OK] -- OUT: 21 . [OK]
+ 1 START
+ 1 STOP
+ 1 wrote filter header
+ EOF
+
+ check_filter_count_clean \
+ git commit . -m "test commit" \
+ <<-\EOF &&
+ x IN: clean test.r 57 [OK] -- OUT: 57 . [OK]
+ x IN: clean test2.r 14 [OK] -- OUT: 14 . [OK]
+ x IN: clean test4-empty.r 0 [OK] -- OUT: 0 [OK]
+ x IN: clean testsubdir/test3-subdir.r 21 [OK] -- OUT: 21 . [OK]
+ 1 START
+ 1 STOP
+ 1 wrote filter header
+ EOF
+
+ rm -f test?.r testsubdir/test3-subdir.r &&
+
+ check_filter_ignore_clean \
+ git checkout . \
+ <<-\EOF &&
+ START
+ wrote filter header
+ IN: smudge test2.r 14 [OK] -- OUT: 14 . [OK]
+ IN: smudge testsubdir/test3-subdir.r 21 [OK] -- OUT: 21 . [OK]
+ STOP
+ EOF
+
+ check_filter_ignore_clean \
+ git checkout empty \
+ <<-\EOF &&
+ START
+ wrote filter header
+ STOP
+ EOF
+
+ check_filter_ignore_clean \
+ git checkout master \
+ <<-\EOF &&
+ START
+ wrote filter header
+ IN: smudge test.r 57 [OK] -- OUT: 57 . [OK]
+ IN: smudge test2.r 14 [OK] -- OUT: 14 . [OK]
+ IN: smudge test4-empty.r 0 [OK] -- OUT: 0 [OK]
+ IN: smudge testsubdir/test3-subdir.r 21 [OK] -- OUT: 21 . [OK]
+ STOP
+ EOF
+
+ check_rot13 ../test.o test.r &&
+ check_rot13 ../test2.o test2.r &&
+ check_rot13 ../test3-subdir.o testsubdir/test3-subdir.r
+ )
+'
+
+test_expect_success PERL 'required process filter should clean only and take precedence' '
+ test_config_global filter.protocol.clean ./../rot13.sh &&
+ test_config_global filter.protocol.process "$TEST_DIRECTORY/t0021/rot13-filter.pl clean" &&
+ test_config_global filter.protocol.required true &&
+ rm -rf repo &&
+ mkdir repo &&
+ (
+ cd repo &&
+ git init &&
+
+ echo "*.r filter=protocol" >.gitattributes &&
+ git add . &&
+ git commit . -m "test commit" &&
+ git branch empty &&
+
+ cp ../test.o test.r &&
+
+ check_filter \
+ git add . \
+ <<-\EOF &&
+ 1 IN: clean test.r 57 [OK] -- OUT: 57 . [OK]
+ 1 START
+ 1 STOP
+ 1 wrote filter header
+ EOF
+
+ check_filter_count_clean \
+ git commit . -m "test commit" \
+ <<-\EOF
+ x IN: clean test.r 57 [OK] -- OUT: 57 . [OK]
+ 1 START
+ 1 STOP
+ 1 wrote filter header
+ EOF
+ )
+'
+
+generate_test_data () {
+ LEN=$1
+ NAME=$2
+ test-genrandom end $LEN |
+ perl -pe "s/./chr((ord($&) % 26) + 97)/sge" >../$NAME.file &&
+ cp ../$NAME.file . &&
+ ./../rot13.sh <../$NAME.file >../$NAME.file.rot13
+}
+
+test_expect_success PERL 'required process filter should process multiple packets' '
+ test_config_global filter.protocol.process "$TEST_DIRECTORY/t0021/rot13-filter.pl clean smudge" &&
+ test_config_global filter.protocol.required true &&
+
+ rm -rf repo &&
+ mkdir repo &&
+ (
+ cd repo &&
+ git init &&
+
+ # Generate data that requires 3 packets
+ PKTLINE_DATA_MAXLEN=65516 &&
+
+ generate_test_data $(($PKTLINE_DATA_MAXLEN )) 1pkt_1__ &&
+ generate_test_data $(($PKTLINE_DATA_MAXLEN + 1)) 2pkt_1+1 &&
+ generate_test_data $(($PKTLINE_DATA_MAXLEN * 2 - 1)) 2pkt_2-1 &&
+ generate_test_data $(($PKTLINE_DATA_MAXLEN * 2 )) 2pkt_2__ &&
+ generate_test_data $(($PKTLINE_DATA_MAXLEN * 2 + 1)) 3pkt_2+1 &&
+
+ echo "*.file filter=protocol" >.gitattributes &&
+ check_filter \
+ git add *.file .gitattributes \
+ <<-\EOF &&
+ 1 IN: clean 1pkt_1__.file 65516 [OK] -- OUT: 65516 . [OK]
+ 1 IN: clean 2pkt_1+1.file 65517 [OK] -- OUT: 65517 .. [OK]
+ 1 IN: clean 2pkt_2-1.file 131031 [OK] -- OUT: 131031 .. [OK]
+ 1 IN: clean 2pkt_2__.file 131032 [OK] -- OUT: 131032 .. [OK]
+ 1 IN: clean 3pkt_2+1.file 131033 [OK] -- OUT: 131033 ... [OK]
+ 1 START
+ 1 STOP
+ 1 wrote filter header
+ EOF
+ git commit . -m "test commit" &&
+
+ rm -f *.file &&
+ git checkout -- *.file &&
+
+ for f in *.file
+ do
+ git cat-file blob :$f >actual &&
+ test_cmp ../$f.rot13 actual
+ done
+ )
+'
+
+test_expect_success PERL 'required process filter should with clean error should fail' '
+ test_config_global filter.protocol.process "$TEST_DIRECTORY/t0021/rot13-filter.pl clean smudge" &&
+ test_config_global filter.protocol.required true &&
+ rm -rf repo &&
+ mkdir repo &&
+ (
+ cd repo &&
+ git init &&
+
+ echo "*.r filter=protocol" >.gitattributes &&
+
+ cp ../test.o test.r &&
+ echo "this is going to fail" >clean-write-fail.r &&
+ echo "content-test3-subdir" >test3.r &&
+
+ # Note: There are three clean paths in convert.c we just test one here.
+ test_must_fail git add .
+ )
+'
+
+test_expect_success PERL 'process filter should restart after unexpected write failure' '
+ test_config_global filter.protocol.process "$TEST_DIRECTORY/t0021/rot13-filter.pl clean smudge" &&
+ rm -rf repo &&
+ mkdir repo &&
+ (
+ cd repo &&
+ git init &&
+
+ echo "*.r filter=protocol" >.gitattributes &&
+
+ cp ../test.o test.r &&
+ cp ../test2.o test2.r &&
+ echo "this is going to fail" >smudge-write-fail.o &&
+ cat smudge-write-fail.o >smudge-write-fail.r &&
+ git add . &&
+ git commit . -m "test commit" &&
+ rm -f *.r &&
+
+ check_filter_ignore_clean \
+ git checkout . \
+ <<-\EOF &&
+ START
+ wrote filter header
+ IN: smudge smudge-write-fail.r 22 [OK] -- OUT: 22 [WRITE FAIL]
+ START
+ wrote filter header
+ IN: smudge test.r 57 [OK] -- OUT: 57 . [OK]
+ IN: smudge test2.r 14 [OK] -- OUT: 14 . [OK]
+ STOP
+ EOF
+
+ check_rot13 ../test.o test.r &&
+ check_rot13 ../test2.o test2.r &&
+
+ ! test_cmp smudge-write-fail.o smudge-write-fail.r && # Smudge failed!
+ ./../rot13.sh <smudge-write-fail.o >expected &&
+ git cat-file blob :smudge-write-fail.r >actual &&
+ test_cmp expected actual # Clean worked!
+ )
+'
+
+test_expect_success PERL 'process filter should not restart in case of an error' '
+ test_config_global filter.protocol.process "$TEST_DIRECTORY/t0021/rot13-filter.pl clean smudge" &&
+ rm -rf repo &&
+ mkdir repo &&
+ (
+ cd repo &&
+ git init &&
+
+ echo "*.r filter=protocol" >.gitattributes &&
+
+ cp ../test.o test.r &&
+ cp ../test2.o test2.r &&
+ echo "this will cause an error" >error.o &&
+ cp error.o error.r &&
+ git add . &&
+ git commit . -m "test commit" &&
+ rm -f *.r &&
+
+ check_filter_ignore_clean \
+ git checkout . \
+ <<-\EOF &&
+ START
+ wrote filter header
+ IN: smudge error.r 25 [OK] -- OUT: 0 [ERROR]
+ IN: smudge test.r 57 [OK] -- OUT: 57 . [OK]
+ IN: smudge test2.r 14 [OK] -- OUT: 14 . [OK]
+ STOP
+ EOF
+
+ check_rot13 ../test.o test.r &&
+ check_rot13 ../test2.o test2.r &&
+ test_cmp error.o error.r
+ )
+'
+
+test_expect_success PERL 'process filter should be able to signal an error for all future files' '
+ test_config_global filter.protocol.process "$TEST_DIRECTORY/t0021/rot13-filter.pl clean smudge" &&
+ rm -rf repo &&
+ mkdir repo &&
+ (
+ cd repo &&
+ git init &&
+
+ echo "*.r filter=protocol" >.gitattributes &&
+
+ cp ../test.o test.r &&
+ cp ../test2.o test2.r &&
+ echo "error this blob and all future blobs" >abort.o &&
+ cp abort.o abort.r &&
+ git add . &&
+ git commit . -m "test commit" &&
+ rm -f *.r &&
+
+ check_filter_ignore_clean \
+ git checkout . \
+ <<-\EOF &&
+ START
+ wrote filter header
+ IN: smudge abort.r 37 [OK] -- OUT: 0 [ABORT]
+ STOP
+ EOF
+
+ test_cmp ../test.o test.r &&
+ test_cmp ../test2.o test2.r &&
+ test_cmp abort.o abort.r
+ )
+'
+
+test_expect_success PERL 'invalid process filter must fail (and not hang!)' '
+ test_config_global filter.protocol.process cat &&
+ test_config_global filter.protocol.required true &&
+ rm -rf repo &&
+ mkdir repo &&
+ (
+ cd repo &&
+ git init &&
+
+ echo "*.r filter=protocol" >.gitattributes &&
+
+ cp ../test.o test.r &&
+ test_must_fail git add . 2> git_stderr.log &&
+ grep "not support long running filter protocol" git_stderr.log
+ )
+'
+
test_done
diff --git a/t/t0021/rot13-filter.pl b/t/t0021/rot13-filter.pl
new file mode 100755
index 0000000..8e27877
--- /dev/null
+++ b/t/t0021/rot13-filter.pl
@@ -0,0 +1,179 @@
+#!/usr/bin/perl
+#
+# Example implementation for the Git filter protocol version 2
+# See Documentation/gitattributes.txt, section "Filter Protocol"
+#
+# The script takes the list of supported protocol capabilities as
+# arguments ("clean", "smudge", etc).
+#
+# This implementation supports special test cases:
+# (1) If data with the pathname "clean-write-fail.r" is processed with
+# a "clean" operation then the write operation will die.
+# (2) If data with the pathname "smudge-write-fail.r" is processed with
+# a "smudge" operation then the write operation will die.
+# (3) If data with the pathname "error.r" is processed with any
+# operation then the filter signals that it cannot or does not want
+# to process the file.
+# (4) If data with the pathname "abort.r" is processed with any
+# operation then the filter signals that it cannot or does not want
+# to process the file and any file after that is processed with the
+# same command.
+#
+
+use strict;
+use warnings;
+
+my $MAX_PACKET_CONTENT_SIZE = 65516;
+my @capabilities = @ARGV;
+
+open my $debug, ">>", "rot13-filter.log";
+
+sub rot13 {
+ my ($str) = @_;
+ $str =~ y/A-Za-z/N-ZA-Mn-za-m/;
+ return $str;
+}
+
+sub packet_read {
+ my $buffer;
+ my $bytes_read = read STDIN, $buffer, 4;
+ if ( $bytes_read == 0 ) {
+
+ # EOF - Git stopped talking to us!
+ print $debug "STOP\n";
+ exit();
+ }
+ elsif ( $bytes_read != 4 ) {
+ die "invalid packet size '$bytes_read' field";
+ }
+ my $pkt_size = hex($buffer);
+ if ( $pkt_size == 0 ) {
+ return ( 1, "" );
+ }
+ elsif ( $pkt_size > 4 ) {
+ my $content_size = $pkt_size - 4;
+ $bytes_read = read STDIN, $buffer, $content_size;
+ if ( $bytes_read != $content_size ) {
+ die "invalid packet ($content_size expected; $bytes_read read)";
+ }
+ return ( 0, $buffer );
+ }
+ else {
+ die "invalid packet size";
+ }
+}
+
+sub packet_write {
+ my ($packet) = @_;
+ print STDOUT sprintf( "%04x", length($packet) + 4 );
+ print STDOUT $packet;
+ STDOUT->flush();
+}
+
+sub packet_flush {
+ print STDOUT sprintf( "%04x", 0 );
+ STDOUT->flush();
+}
+
+print $debug "START\n";
+$debug->flush();
+
+( packet_read() eq ( 0, "git-filter-client" ) ) || die "bad initialization";
+( packet_read() eq ( 0, "version=2" ) ) || die "bad version";
+( packet_read() eq ( 1, "" ) ) || die "bad version end";
+
+packet_write("git-filter-server\n");
+packet_write("version=2\n");
+
+( packet_read() eq ( 0, "clean=true" ) ) || die "bad capability";
+( packet_read() eq ( 0, "smudge=true" ) ) || die "bad capability";
+( packet_read() eq ( 1, "" ) ) || die "bad capability end";
+
+foreach (@capabilities) {
+ packet_write( $_ . "=true\n" );
+}
+packet_flush();
+print $debug "wrote filter header\n";
+$debug->flush();
+
+while (1) {
+ my ($command) = packet_read() =~ /^command=([^=]+)\n$/;
+ print $debug "IN: $command";
+ $debug->flush();
+
+ my ($pathname) = packet_read() =~ /^pathname=([^=]+)\n$/;
+ print $debug " $pathname";
+ $debug->flush();
+
+ # Flush
+ packet_read();
+
+ my $input = "";
+ {
+ binmode(STDIN);
+ my $buffer;
+ my $done = 0;
+ while ( !$done ) {
+ ( $done, $buffer ) = packet_read();
+ $input .= $buffer;
+ }
+ print $debug " " . length($input) . " [OK] -- ";
+ $debug->flush();
+ }
+
+ my $output;
+ if ( $pathname eq "error.r" or $pathname eq "abort.r" ) {
+ $output = "";
+ }
+ elsif ( $command eq "clean" and grep( /^clean$/, @capabilities ) ) {
+ $output = rot13($input);
+ }
+ elsif ( $command eq "smudge" and grep( /^smudge$/, @capabilities ) ) {
+ $output = rot13($input);
+ }
+ else {
+ die "bad command '$command'";
+ }
+
+ print $debug "OUT: " . length($output) . " ";
+ $debug->flush();
+
+ if ( $pathname eq "error.r" ) {
+ print $debug "[ERROR]\n";
+ $debug->flush();
+ packet_write("status=error\n");
+ packet_flush();
+ }
+ elsif ( $pathname eq "abort.r" ) {
+ print $debug "[ABORT]\n";
+ $debug->flush();
+ packet_write("status=abort\n");
+ packet_flush();
+ }
+ else {
+ packet_write("status=success\n");
+ packet_flush();
+
+ if ( $pathname eq "${command}-write-fail.r" ) {
+ print $debug "[WRITE FAIL]\n";
+ $debug->flush();
+ die "${command} write error";
+ }
+
+ while ( length($output) > 0 ) {
+ my $packet = substr( $output, 0, $MAX_PACKET_CONTENT_SIZE );
+ packet_write($packet);
+ print $debug ".";
+ if ( length($output) > $MAX_PACKET_CONTENT_SIZE ) {
+ $output = substr( $output, $MAX_PACKET_CONTENT_SIZE );
+ }
+ else {
+ $output = "";
+ }
+ }
+ packet_flush();
+ print $debug " [OK]\n";
+ $debug->flush();
+ packet_flush();
+ }
+}
diff --git a/unpack-trees.c b/unpack-trees.c
index 11c37fb..f6798f8 100644
--- a/unpack-trees.c
+++ b/unpack-trees.c
@@ -10,6 +10,7 @@
#include "attr.h"
#include "split-index.h"
#include "dir.h"
+#include "convert.h"
/*
* Error messages expected by scripts out of plumbing commands such as
--
2.10.0
^ permalink raw reply related
* Re: [PATCH 2/3] diff_populate_filespec: NUL-terminate buffers
From: Johannes Schindelin @ 2016-09-08 18:22 UTC (permalink / raw)
To: Junio C Hamano; +Cc: Jeff King, git
In-Reply-To: <xmqqa8fijpmg.fsf@gitster.mtv.corp.google.com>
Hi Peff & Junio,
On Thu, 8 Sep 2016, Junio C Hamano wrote:
> Jeff King <peff@peff.net> writes:
>
> > Between the two options for regexec_buf(), I think you have convinced me
> > that REG_STARTEND is better than just using compat/regex everywhere. I
> > do think the fallback for platforms like musl should be "use
> > compat/regex" and not doing an expensive copy (which in most cases is
> > not even necessary).
>
> I agree with you that it would be the best approach to build
> regexec_buf() that unconditionally uses REG_STARTEND and tell people
> without REG_STARTEND to use compat/regex instead of their platform
> regex library.
>
> The description in Makefile may want to be rephrased to clarify.
>
> -# Define NO_REGEX if you have no or inferior regex support in your C library.
> +# Define NO_REGEX if your C library lacks regex support with REG_STARTEND
> +# feature.
Okay, I will make that happen. But first I need to sleep.
Ciao,
Dscho
^ permalink raw reply
* Re: [PATCH] Move format-patch base commit and prerequisites before email signature
From: Junio C Hamano @ 2016-09-08 18:34 UTC (permalink / raw)
To: Josh Triplett; +Cc: git
In-Reply-To: <20160908011200.qzvbdt4wjwiji4h5@x>
Josh Triplett <josh@joshtriplett.org> writes:
> Any text below the "-- " for the email signature gets treated as part of
> the signature, and many mail clients will trim it from the quoted text
> for a reply. Move it above the signature, so people can reply to it
> more easily.
>
> Add tests for the exact format of the email signature, and add tests to
> ensure the email signature appears last.
>
> (Patch by Junio Hamano; tests by Josh Triplett.)
> Signed-off-by: Josh Triplett <josh@joshtriplett.org>
> ---
>
> Does the above seem reasonable, for a patch that incorporates the
> proposed patch from Message-Id
> xmqqh99rpud4.fsf@gitster.mtv.corp.google.com and adds tests?
Other than that I'd probably retitle it, your problem description
looks perfect. I am still not sure if the code does a reasonable
thing in MIME case, though.
Thanks for tying the loose ends anyway.
> diff --git a/t/t4014-format-patch.sh b/t/t4014-format-patch.sh
> index b0579dd..a4af275 100755
> --- a/t/t4014-format-patch.sh
> +++ b/t/t4014-format-patch.sh
> @@ -754,9 +754,22 @@ test_expect_success 'format-patch --ignore-if-in-upstream HEAD' '
> git format-patch --ignore-if-in-upstream HEAD
> '
>
> +git_version="$(git --version | sed "s/.* //")"
> +
> +signature() {
> + printf "%s\n%s\n\n" "-- " "${1:-$git_version}"
> +}
Hmph. I would actually have expected that you would force a fixed
and an easily noticeable string via format.signature for the purpose
of the test, but I guess this test covers a lot more than what the
purpose of the main part of the patch does (i.e. enforces that the
default signature must be made from the version string of Git). It
is not a bad thing to test, but it probably does not belong to this
change. If you _were_ to split the patch in two, that is where I
probably would split, i.e. "we didn't test what the default signature
looks like, or we didn't make sure --signature option overrides the
default signature, so let's test it" as the preliminary preparation,
followed by "having base info after sig is inconvenient, let's move
it and make sure base info stays before sig with additional test" as
the second (and primary) patch.
But a single patch is fine.
Thanks.
^ permalink raw reply
* Re: "fatal error in commit_refs" from pushing to github
From: Jeff King @ 2016-09-08 18:40 UTC (permalink / raw)
To: Duy Nguyen; +Cc: Git Mailing List
In-Reply-To: <CACsJy8DW1-EsfgGkbnD3LgiygAu2JV9vs_8e+oQbenXERUA6hw@mail.gmail.com>
On Thu, Sep 08, 2016 at 06:03:33PM +0700, Duy Nguyen wrote:
> > So this is really an internal failure at the ref-update stage. There
> > _should_ be a reasonable error message, but I think "fatal error in
> > commit_refs" is the generic last-ditch fallback. I'll pass this along to
> > people in charge of that code, as we should be generating a more useful
> > error message.
>
> Hmm.. I'm interested in this because the "fix" is from client side. I
> did "git gc" and "git fetch" and the problem was gone.
It may also have been a transient error inside GitHub that resolved
itself between your two pushes.
-Peff
^ permalink raw reply
* Re: [PATCH 2/3] diff_populate_filespec: NUL-terminate buffers
From: Jeff King @ 2016-09-08 18:48 UTC (permalink / raw)
To: Junio C Hamano; +Cc: Johannes Schindelin, git
In-Reply-To: <xmqqa8fijpmg.fsf@gitster.mtv.corp.google.com>
On Thu, Sep 08, 2016 at 09:57:43AM -0700, Junio C Hamano wrote:
> Jeff King <peff@peff.net> writes:
>
> > Between the two options for regexec_buf(), I think you have convinced me
> > that REG_STARTEND is better than just using compat/regex everywhere. I
> > do think the fallback for platforms like musl should be "use
> > compat/regex" and not doing an expensive copy (which in most cases is
> > not even necessary).
>
> I agree with you that it would be the best approach to build
> regexec_buf() that unconditionally uses REG_STARTEND and tell people
> without REG_STARTEND to use compat/regex instead of their platform
> regex library.
>
> The description in Makefile may want to be rephrased to clarify.
>
> -# Define NO_REGEX if you have no or inferior regex support in your C library.
> +# Define NO_REGEX if your C library lacks regex support with REG_STARTEND
> +# feature.
>
> The word "inferior" is not giving any useful information there.
Yeah, very much agreed (and the "#error" when we lack REG_STARTEND I
mentioned earlier may be a good hint).
-Peff
^ permalink raw reply
* Re: [PATCH] Move format-patch base commit and prerequisites before email signature
From: Josh Triplett @ 2016-09-08 18:54 UTC (permalink / raw)
To: Junio C Hamano; +Cc: git
In-Reply-To: <xmqqshtags0o.fsf@gitster.mtv.corp.google.com>
On Thu, Sep 08, 2016 at 11:34:15AM -0700, Junio C Hamano wrote:
> Josh Triplett <josh@joshtriplett.org> writes:
>
> > Any text below the "-- " for the email signature gets treated as part of
> > the signature, and many mail clients will trim it from the quoted text
> > for a reply. Move it above the signature, so people can reply to it
> > more easily.
> >
> > Add tests for the exact format of the email signature, and add tests to
> > ensure the email signature appears last.
> >
> > (Patch by Junio Hamano; tests by Josh Triplett.)
> > Signed-off-by: Josh Triplett <josh@joshtriplett.org>
> > ---
> >
> > Does the above seem reasonable, for a patch that incorporates the
> > proposed patch from Message-Id
> > xmqqh99rpud4.fsf@gitster.mtv.corp.google.com and adds tests?
>
> Other than that I'd probably retitle it,
Ah, true, I should have titled it "format-patch: move base commit ...".
> your problem description
> looks perfect. I am still not sure if the code does a reasonable
> thing in MIME case, though.
It *looks* correct to me.
> Thanks for tying the loose ends anyway.
>
> > diff --git a/t/t4014-format-patch.sh b/t/t4014-format-patch.sh
> > index b0579dd..a4af275 100755
> > --- a/t/t4014-format-patch.sh
> > +++ b/t/t4014-format-patch.sh
> > @@ -754,9 +754,22 @@ test_expect_success 'format-patch --ignore-if-in-upstream HEAD' '
> > git format-patch --ignore-if-in-upstream HEAD
> > '
> >
> > +git_version="$(git --version | sed "s/.* //")"
> > +
> > +signature() {
> > + printf "%s\n%s\n\n" "-- " "${1:-$git_version}"
> > +}
>
> Hmph. I would actually have expected that you would force a fixed
> and an easily noticeable string via format.signature for the purpose
> of the test,
One of the git tests already did that. I just modified that test to
test the exact signature format and that it appears at the end, rather
than just grepping to check that the signature string appears somewhere.
Then when doing so, I realized that I should check the default case too
(at which point that test change probably should have gone in a separate
patch).
> but I guess this test covers a lot more than what the
> purpose of the main part of the patch does (i.e. enforces that the
> default signature must be made from the version string of Git). It
> is not a bad thing to test, but it probably does not belong to this
> change. If you _were_ to split the patch in two, that is where I
> probably would split, i.e. "we didn't test what the default signature
> looks like, or we didn't make sure --signature option overrides the
> default signature, so let's test it" as the preliminary preparation,
> followed by "having base info after sig is inconvenient, let's move
> it and make sure base info stays before sig with additional test" as
> the second (and primary) patch.
>
> But a single patch is fine.
>
> Thanks.
If any other change ends up being necessary, I'll split the patch in v2.
- Josh Triplett
^ permalink raw reply
* Re: [PATCH v2 3/3] Use the newly-introduced regexec_buf() function
From: Ramsay Jones @ 2016-09-08 19:06 UTC (permalink / raw)
To: Jeff King, Johannes Schindelin; +Cc: git, Junio C Hamano
In-Reply-To: <20160908083509.ygh4flpunyyp4hua@sigill.intra.peff.net>
On 08/09/16 09:35, Jeff King wrote:
> On Thu, Sep 08, 2016 at 04:14:46AM -0400, Jeff King wrote:
>
>> On Thu, Sep 08, 2016 at 04:10:24AM -0400, Jeff King wrote:
>>
>>> On Thu, Sep 08, 2016 at 09:54:51AM +0200, Johannes Schindelin wrote:
>>>
>>>>> diff.c | 3 ++-
>>>>> diffcore-pickaxe.c | 18 ++++++++----------
>>>>> xdiff-interface.c | 13 ++++---------
>>>>> 3 files changed, 14 insertions(+), 20 deletions(-)
>>>>
>>>> I just realized that this should switch the test_expect_failure from 1/3
>>>> to a test_expect_success.
>>>
>>> Yep. I wonder if we also would want to test that we correctly find
>>> regexes inside binary files.
>>>
>>> E.g., given a mixed binary/text file like:
>>>
>>> printf 'binary\0text' >file &&
>>> git add file &&
>>> git commit -m file
>>>
>>> then "git log -Stext" will find that file, but "--pickaxe-regex" will
>>> not (using stock git). Ditto for "-Gtext".
>>>
>>> Your patch should fix that.
>>
>> Of course if I had actually _looked carefully_ at your patch, I would
>> have seen that your test doesn't just check that we don't segfault, but
>> actually confirms that we find the entry.
>>
>> Sorry for the noise.
>
> Actually, I take it back again. Your test case doesn't have an embedded
> NUL in it (so we check that git finds it, but aside from the lack of
> segfault, stock git would already find it).
This reminds me ... despite the native cygwin regex library no longer
having the 'regex bug' (ie t0070.5 now passes), I still have NO_REGEX
set on cygwin. This is because, when building with the native library,
we have an "unexpected pass" for t7008.12, which looks like:
test_expect_failure 'git grep .fi a' '
git grep .fi a
'
[where the file a is set up earlier by: echo 'binaryQfile' | q_to_nul >a]
commit f96e5673 ("grep: use REG_STARTEND for all matching if available",
22-05-2010) introduced this test and expects ".. NUL characters themselves
are not matched in any way". With the native library on cygwin they are
matched, with the compat/regex they are not. Indeed, if you use the system
'grep' command (rather than 'git grep'), then it will also not match ... :-D
Slightly off topic, but ...
ATB,
Ramsay Jones
^ permalink raw reply
* Re: [PATCH] Move format-patch base commit and prerequisites before email signature
From: Junio C Hamano @ 2016-09-08 19:11 UTC (permalink / raw)
To: Josh Triplett; +Cc: git
In-Reply-To: <20160908185408.5qtfnztjbastlrtw@x>
Josh Triplett <josh@joshtriplett.org> writes:
> If any other change ends up being necessary, I'll split the patch in v2.
Thanks. I do not see anything else offhand myself, but other people
watching the topic from the sideline may spot something we missed.
^ permalink raw reply
* Re: [PATCH 1/3] init: correct re-initialization from a linked worktree
From: Junio C Hamano @ 2016-09-08 19:37 UTC (permalink / raw)
To: Nguyễn Thái Ngọc Duy; +Cc: git, git, max.nordlund
In-Reply-To: <20160908134719.27955-2-pclouds@gmail.com>
Nguyễn Thái Ngọc Duy <pclouds@gmail.com> writes:
> When 'git init' is called from a linked worktree, '.git' dir as the main
> '.git' (i.e. $GIT_COMMON_DIR) and populate the whole repository skeleton
> in there. It does not harm anything (*) but it is still wrong.
-ECANNOTPARSE. Did you mean "... worktree, we treat '.git' dir as
if it is the main '.git' ..." or something entirely different?
> diff --git a/builtin/init-db.c b/builtin/init-db.c
> index 3a45f0b..6d9552e 100644
> --- a/builtin/init-db.c
> +++ b/builtin/init-db.c
> @@ -138,7 +138,7 @@ static void copy_templates(const char *template_dir)
> goto close_free_return;
> }
>
> - strbuf_addstr(&path, get_git_dir());
> + strbuf_addstr(&path, get_git_common_dir());
> strbuf_complete(&path, '/');
> copy_templates_1(&path, &template_path, dir);
> close_free_return:
> diff --git a/t/t0001-init.sh b/t/t0001-init.sh
> index a6fdd5e..d64e5e3 100755
> --- a/t/t0001-init.sh
> +++ b/t/t0001-init.sh
> @@ -384,4 +384,19 @@ test_expect_success MINGW 'bare git dir not hidden' '
> ! is_hidden newdir
> '
>
> +test_expect_success 're-init from a linked worktree' '
> + git init main-worktree &&
> + (
> + cd main-worktree &&
> + test_commit first &&
> + git worktree add ../linked-worktree &&
> + mv .git/info/exclude expected-exclude &&
> + find .git/worktrees -print | sort >expected &&
> + git -C ../linked-worktree init &&
> + test_cmp expected-exclude .git/info/exclude &&
> + find .git/worktrees -print | sort >actual &&
> + test_cmp expected actual
> + )
> +'
> +
> test_done
^ permalink raw reply
* Re: [PATCH 2/3] t0001: work around the bug that reads config file before repo setup
From: Junio C Hamano @ 2016-09-08 19:44 UTC (permalink / raw)
To: Nguyễn Thái Ngọc Duy; +Cc: git, git, max.nordlund
In-Reply-To: <20160908134719.27955-3-pclouds@gmail.com>
Nguyễn Thái Ngọc Duy <pclouds@gmail.com> writes:
> git-init somehow reads '.git/config' at current directory and sets
> log_all_ref_updates based on this file. Because log_all_ref_updates is
> not unspecified (-1) any more. It will not be written to the new repo's
> config file (see create_default_files() function).
I vaguely recall this was discussed recently somewhere. Is this
related to <20160902091130.jxckdeomw35eewky@sigill.intra.peff.net>?
> This will affect our tests in the next patch as we will compare the
> config file and expect that core.logallrefupdates is already set to true
> by "git init main-worktree".
Ugh. The "mv" workaround is fine to demonstrate that you have fixed
one of two problems, but as you said, it would be nice to have another
that expects failure until both of them gets fixed.
> Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
> ---
> t/t0001-init.sh | 2 ++
> 1 file changed, 2 insertions(+)
>
> diff --git a/t/t0001-init.sh b/t/t0001-init.sh
> index d64e5e3..393c940 100755
> --- a/t/t0001-init.sh
> +++ b/t/t0001-init.sh
> @@ -385,7 +385,9 @@ test_expect_success MINGW 'bare git dir not hidden' '
> '
>
> test_expect_success 're-init from a linked worktree' '
> + mv .git/config work-around-init-reading-wrong-file &&
> git init main-worktree &&
> + mv work-around-init-reading-wrong-file .git/config &&
> (
> cd main-worktree &&
> test_commit first &&
^ permalink raw reply
* Re: [PATCH v2 3/3] Use the newly-introduced regexec_buf() function
From: Jeff King @ 2016-09-08 19:53 UTC (permalink / raw)
To: Ramsay Jones; +Cc: Johannes Schindelin, git, Junio C Hamano
In-Reply-To: <ac0f3441-d28f-fc5c-0273-f614b2cd4e07@ramsayjones.plus.com>
On Thu, Sep 08, 2016 at 08:06:35PM +0100, Ramsay Jones wrote:
> > Actually, I take it back again. Your test case doesn't have an embedded
> > NUL in it (so we check that git finds it, but aside from the lack of
> > segfault, stock git would already find it).
>
> This reminds me ... despite the native cygwin regex library no longer
> having the 'regex bug' (ie t0070.5 now passes), I still have NO_REGEX
> set on cygwin. This is because, when building with the native library,
> we have an "unexpected pass" for t7008.12, which looks like:
>
> test_expect_failure 'git grep .fi a' '
> git grep .fi a
> '
> [where the file a is set up earlier by: echo 'binaryQfile' | q_to_nul >a]
>
> commit f96e5673 ("grep: use REG_STARTEND for all matching if available",
> 22-05-2010) introduced this test and expects ".. NUL characters themselves
> are not matched in any way". With the native library on cygwin they are
> matched, with the compat/regex they are not. Indeed, if you use the system
> 'grep' command (rather than 'git grep'), then it will also not match ... :-D
>
> Slightly off topic, but ...
Hmm. So it sounds like the "regmatch" in grep.c could go away in favor
of Johannes's regexec_buf(), and cygwin ought to be using NO_REGEX.
-Peff
^ permalink raw reply
* Re: [PATCH 3/3] init: do not set core.worktree more often than necessary
From: Junio C Hamano @ 2016-09-08 19:54 UTC (permalink / raw)
To: Nguyễn Thái Ngọc Duy; +Cc: git, git, max.nordlund
In-Reply-To: <20160908134719.27955-4-pclouds@gmail.com>
Nguyễn Thái Ngọc Duy <pclouds@gmail.com> writes:
> +/*
> + * Return the first ".git" that we have encountered.
> + * FIXME this function for not entirely correct because
> + * setup_git_directory() and enter_repo() do not update first_git_dir
> + * when they follow .git files. The function in its current state is
> + * only suitable for "git init".
> + */
Would it be possible to move this to "init-db.c" then?
The very first thing cmd_init_db() does to what is in the
environment.c is to call set_git_dir() via set_git_dir_init() to
tell it where the ".git" thing is, no? Can't that code remember the
location itself, instead of adding code that is known not to be
usable by other callers? That would help avoiding the future
confusion.
> +const char *get_first_git_dir(void)
> +{
> + return first_git_dir ? first_git_dir : git_dir;
> +}
> +
> const char *get_git_common_dir(void)
> {
> return git_common_dir;
> diff --git a/t/t0001-init.sh b/t/t0001-init.sh
> index 393c940..d59669a 100755
> --- a/t/t0001-init.sh
> +++ b/t/t0001-init.sh
> @@ -393,9 +393,11 @@ test_expect_success 're-init from a linked worktree' '
> test_commit first &&
> git worktree add ../linked-worktree &&
> mv .git/info/exclude expected-exclude &&
> + cp .git/config expected-config &&
> find .git/worktrees -print | sort >expected &&
> git -C ../linked-worktree init &&
> test_cmp expected-exclude .git/info/exclude &&
> + test_cmp expected-config .git/config &&
> find .git/worktrees -print | sort >actual &&
> test_cmp expected actual
> )
^ permalink raw reply
* Re: [PATCH 2/3] t0001: work around the bug that reads config file before repo setup
From: Jeff King @ 2016-09-08 20:02 UTC (permalink / raw)
To: Nguyễn Thái Ngọc Duy; +Cc: git, git, max.nordlund
In-Reply-To: <20160908134719.27955-3-pclouds@gmail.com>
On Thu, Sep 08, 2016 at 08:47:18PM +0700, Nguyễn Thái Ngọc Duy wrote:
> git-init somehow reads '.git/config' at current directory and sets
> log_all_ref_updates based on this file. Because log_all_ref_updates is
> not unspecified (-1) any more. It will not be written to the new repo's
> config file (see create_default_files() function).
>
> This will affect our tests in the next patch as we will compare the
> config file and expect that core.logallrefupdates is already set to true
> by "git init main-worktree".
This is a bug for more than worktrees, and is something I'm working on
fixing (what I'd like to do is kill off the lazy fallback to ".git/" as
the repo name, which is almost always the wrong thing to do).
I'm not opposed to your workaround, but just FYI.
-Peff
^ permalink raw reply
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox