* Re: Something is broken in repack
From: Morten Welinder @ 2007-12-10 20:16 UTC (permalink / raw)
To: Jon Smirl; +Cc: Nicolas Pitre, Git Mailing List
In-Reply-To: <9e4733910712101205q218152a2td14a8931e63d2610@mail.gmail.com>
> Here's another observation, the gcc objects are larger. Kernel has
> 650K objects in 190MB, gcc has 870K objects in 330MB. Average gcc
> object is 30% larger. How should the average kernel developer
> interpret this?
Could this be explained by the ChangeLog file? It's large; it has tons of
revisions; it is a prime candidate for delta compression.
Morten
^ permalink raw reply
* Re: In future, to replace autotools by cmake like KDE4 did?
From: Jan Hudec @ 2007-12-10 20:23 UTC (permalink / raw)
To: J.C. Pizarro
Cc: Jakub Narebski, Andreas Ericsson, gcc, git, David Miller,
Daniel Berlin, Ismail Donmez, Marcel Holtmann
In-Reply-To: <998d0e4a0712070642u6ae75232t9cb5bfd0920b2439@mail.gmail.com>
[-- Attachment #1: Type: text/plain, Size: 2116 bytes --]
On Fri, Dec 07, 2007 at 15:42:31 +0100, J.C. Pizarro wrote:
> A powerful tool can do better things that old generators-based tools
> (as autotools).
>
> To imagine, there are many scripts in subdirectories or subprojects:
No, there are not. There is just one. Multiple configuration scripts rarely
make sense.
> * Before: (many copy and paste of code as below paragraph)
> A_VARIABLE_OS = `uname -a | grep .... ` # <- slow
> case "$A_VARIABLE_OS" in
> *linux*) ... ;;
> *bsd*) ... ;;
> *aix*) ... ;;
> *) ...;;
> esac
> m4 foo.sh.m4 > bar.sh # <- very slow
Done once at release time.
> ./bar.sh
>
> * Later: (with the powerful tool that had cached many predefined variables in
> a ramdisk's file or in a daemon's memory)
A daemon not runnin' here. No ramdisk here either. Freshly downloaded tarball
to an ancient Un*x with some quirky barely POSIX-compliant shell.
> # call once at 1st time to internal uname of powerful tool for all ocurrences of
> # below predefined variable from many scripts:
> case "$FOO_VARIABLE_OS" in
Someone had to create that variable. And there is just one way to: uname -a | ....
> *linux*) ... ;;
> *bsd*) ... ;;
> *aix*) ... ;;
> *) ...;;
> esac
And how exactly does this differ from the code you had above? For task that
runs once per installation (and for most users never, because their
distribution's build server runs it for them), it's simplicity of code that
matters.
> # i don't need to generate more scripts to inspect still more it.
And how exactly did you find, from the uname, whether I have libcrypto
installed? And whether I have it in /usr/lib, /opt/openssl/lib or
/usr/@foobar.com/sw/system/lib? How did you find libcurl, tcl, zlib...?
Besides, I inspected the configure.ac script that comes with git and it does
not actually contain any code like you show above. Git's configure script is
NOT looking at the platform name AT ALL. The makefile does, but that
obviously does not need anything generated by M4.
--
Jan 'Bulb' Hudec <bulb@ucw.cz>
[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 189 bytes --]
^ permalink raw reply
* backups with git and inotify
From: Luciano Rocha @ 2007-12-10 20:29 UTC (permalink / raw)
To: git
[-- Attachment #1.1: Type: text/plain, Size: 1152 bytes --]
Hello,
The following is a work in progress. There are some problems in how I'm
using git and recording the history:
1. I use an opened fd for each monitored directory (and subdirectories),
(inotify_add_watch_at would be nice).
I fchdir(fd) when a change happens to register and commit it.
2. git-rm dir/file also removes <dir> if file was the only entry of
<dir>. So, when committing the removal, git complains that it can't
find cwd. So I record the parent directory, do the git command, check
if getcwd() works, and if not do the commit in the parent directory.
3. git-rm (empty) directory fails
4. Changes aren't atomic, but I can live with that and I doubt I would
be able to make it atomic without implementing a filesystem (FUSE or
not).
I can work around most of the problems, and rewrite to use recorded path
names instead of directories fd, but before I do that, and while I'm
at the beginning, I'd like to probe for opinions and suggestions.
So, please, suggest.
Regards,
Luciano Rocha
--
Luciano Rocha <luciano@eurotux.com>
Eurotux Informática, S.A. <http://www.eurotux.com/>
[-- Attachment #1.2: ino.c --]
[-- Type: text/plain, Size: 9197 bytes --]
/*
monitor with inotify, record with git
Copyright (C) 2007, Luciano Rocha <luciano@nsk.pt>
Released under the GPL v2 or later. See LICENSE.GPL.
This program is free software; you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation; either version 2 of the License, or
(at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program; if not, write to the Free Software
Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
*/
#define _ATFILE_SOURCE 1
#define _GNU_SOURCE 1
#include <sys/inotify.h>
#include <sys/types.h>
#include <sys/stat.h>
#include <sys/wait.h>
#include <errno.h>
#include <fcntl.h>
#include <unistd.h>
#include <dirent.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
struct ilist {
int fd;
uint32_t wd;
struct ilist *prev, *next;
};
typedef struct ilist *ilist;
/* inotify_add_watch macro, with desired mask */
#define INOTIFY_ADD(ifd, dir) (inotify_add_watch((ifd), (dir), \
IN_CLOSE_WRITE | IN_CREATE \
| IN_DELETE | IN_DELETE_SELF \
| IN_MOVED_FROM | IN_MOVED_TO))
/* add directory to inotify watch list
* inotify_add_watchat would be nice, but it doesn't exist, so
* read symlink in /proc/self/fd/<dirfd> instead
*/
int add_inotify(int fd, int ifd)
{
char p[32];
char *dir;
int l;
int wd;
ssize_t ll;
snprintf(p, sizeof p, "/proc/self/fd/%d", fd);
dir = NULL;
l = 0;
do {
l += 256;
if (dir)
free(dir);
dir = malloc(l);
ll = readlink(p, dir, l);
if (ll < 0) {
perror(p);
free(dir);
return -1;
}
} while (ll >= l);
dir[strlen(dir) - 1] = '\0';
wd = INOTIFY_ADD(ifd, dir);
if (wd < 0)
perror(dir);
free(dir);
return wd;
}
/* add watch for directory and each sub-directory, unless
* there's a .git inside
*/
void add_watch(int root, const char *name, ilist *head, int ifd,
const char **except, const char **skip)
{
ilist new;
DIR *dir;
struct dirent *d;
int dirfd, DIRfd;
int wd;
int i;
new = malloc(sizeof *new);
if (!new) {
perror(name);
return;
}
dirfd = openat(root, name, O_RDONLY | O_DIRECTORY | O_NOATIME
| O_NOFOLLOW);
if (dirfd < 0) {
perror(name);
free(new);
return;
}
if (except) {
for (i = 0; except[i] && faccessat(dirfd, except[i],
R_OK | X_OK,
AT_EACCESS | AT_SYMLINK_NOFOLLOW); i++);
if (except[i]) {
printf("skipping %s (%s exists)\n", name, except[i]);
free(new);
close(dirfd);
return;
}
}
wd = add_inotify(dirfd, ifd);
if (wd < 0) {
free(new);
close(dirfd);
return;
}
DIRfd = dup(dirfd);
dir = fdopendir(DIRfd);
if (!dir) {
perror(name);
free(new);
close(dirfd);
close(DIRfd);
inotify_rm_watch(ifd, wd);
return;
}
while ((d = readdir(dir))) {
if (!S_ISDIR(d->d_type << 12))
continue;
if (d->d_name[0] == '.' && (d->d_name[1] == '\0'
|| (d->d_name[1] == '.'
&& d->d_name[2] == '\0')))
continue;
if (skip) {
for (i = 0; skip[i] && strcmp(skip[i], d->d_name); i++);
if (skip[i])
continue;
}
add_watch(dirfd, d->d_name, head, ifd, except, skip);
}
closedir(dir);
/* add to list */
new->fd = dirfd;
new->wd = wd;
new->next = *head;
if (*head) (*head)->prev = new;
new->prev = NULL;
*head = new;
}
static const char *default_except[] = {
".git",
NULL,
};
/* add watch to a directory and its sub-directories, complain and do
* nothing if no .git exists
*/
void git_watch(const char *name, ilist *head, int ifd, const char **skip)
{
ilist new;
DIR *dir;
struct dirent *d;
int dirfd, DIRfd;
int wd;
int i;
new = malloc(sizeof *new);
if (!new) {
perror(name);
return;
}
dirfd = open(name, O_RDONLY | O_DIRECTORY | O_NOATIME | O_NOFOLLOW);
if (dirfd < 0) {
perror(name);
free(new);
return;
}
if (faccessat(dirfd, ".git", R_OK | X_OK,
AT_EACCESS | AT_SYMLINK_NOFOLLOW)) {
fprintf(stderr, "couldn't access .git subdir of %s: %s\n",
name, strerror(errno));
free(new);
close(dirfd);
return;
}
wd = INOTIFY_ADD(ifd, name);
if (wd < 0) {
perror(name);
free(new);
close(dirfd);
return;
}
DIRfd = dup(dirfd);
dir = fdopendir(DIRfd);
if (!dir) {
perror(name);
free(new);
close(dirfd);
close(DIRfd);
inotify_rm_watch(ifd, wd);
return;
}
while ((d = readdir(dir))) {
if (!S_ISDIR(d->d_type << 12))
continue;
if (d->d_name[0] == '.' && (d->d_name[1] == '\0'
|| (d->d_name[1] == '.'
&& d->d_name[2] == '\0')))
continue;
if (!strcmp(".git", d->d_name))
continue;
if (skip) {
for (i = 0; skip[i] && strcmp(skip[i], d->d_name); i++);
if (skip[i])
continue;
}
add_watch(dirfd, d->d_name, head, ifd, default_except, skip);
}
closedir(dir);
/* add to list */
new->fd = dirfd;
new->wd = wd;
new->next = *head;
if (*head) (*head)->prev = new;
new->prev = NULL;
*head = new;
}
/* run a shell command, abort on error
*/
void run_command(char *argv[])
{
pid_t pid;
int status;
pid = fork();
if (pid < 0) {
perror("fork(2)");
exit(1);
}
if (pid == 0) {
execvp(argv[0], argv);
perror(argv[0]);
exit(1);
}
if (waitpid(pid, &status, 0) < 0) {
perror("couldn't wait for child");
exit(1);
}
if (WIFEXITED(status) && WEXITSTATUS(status) == 0)
return;
fprintf(stderr, "sub-command %s returned invalid exit code: %d\n",
argv[0], status);
exit(1);
}
/* run git command, and follow it with git-commit
*/
void run_git(int fd, char *cmd, char *what)
{
int parent;
int cl = strlen(cmd);
int wl = strlen(what);
char commit[cl + wl + 5];
char *argv[] = {
"git",
cmd,
what,
NULL,
};
if (fchdir(fd))
return;
/* save parent:
* git-rm of last file in subdir removes the directory, so the
* following git-commit fails
*/
parent = open("..", O_RDONLY | O_DIRECTORY | O_NOATIME | O_NOFOLLOW);
/* run git sub-command */
run_command(argv);
if (getcwd(commit, cl + wl) == NULL && errno != ERANGE && parent >= 0) {
printf("errno: %d, %s\n", errno, strerror(errno));
fchdir(parent);
}
/* create commit message */
commit[0] = '-';
commit[1] = 'm';
memcpy(commit + 2, cmd, cl);
commit[cl + 2] = ':';
commit[cl + 3] = ' ';
memcpy(commit + cl + 4, what, wl + 1);
/* commit change(s) */
argv[1] = "commit";
argv[2] = commit;
run_command(argv);
}
/* get inotify events, run git as appropriate
*/
void check_event(void *buffer, int len, ilist *head, int ifd)
{
while (len >= sizeof(struct inotify_event)) {
struct inotify_event *p = buffer;
ilist l;
/* advance buffer position */
len -= sizeof(struct inotify_event) + p->len;
buffer += sizeof(struct inotify_event) + p->len;
for (l = *head; l && l->wd != p->wd; l = l->next);
if (!l) {
/* not found in list? */
continue;
}
if (p->mask & (IN_IGNORED | IN_UNMOUNT | IN_DELETE_SELF)) {
/* remove it */
inotify_rm_watch(ifd, p->wd);
close(l->fd);
if (l->prev)
l->prev->next = l->next;
else
*head = l->next;
if (l->next)
l->next->prev = l->prev;
}
/* the following events require a file name specification,
* changes to the directory itself aren't of our interest
*/
if (p->len == 0)
continue;
if (p->mask & IN_CREATE) {
/* add new watch if directory, otherwise ignore,
* IN_CLOSE_WRITE should follow
*/
struct stat st;
if (!fstatat(l->fd, p->name, &st, AT_SYMLINK_NOFOLLOW)
&& S_ISDIR(st.st_mode))
add_watch(l->fd, p->name, head, ifd,
default_except, NULL);
}
if (p->mask & (IN_CLOSE_WRITE | IN_MOVED_TO)) {
/* add/commit */
run_git(l->fd, "add", p->name);
}
if (p->mask & (IN_DELETE | IN_MOVED_FROM)) {
/* rm/commit */
run_git(l->fd, "rm", p->name);
}
}
}
int main(int argc, char *argv[])
{
int fd;
int i;
ilist head;
void *buffer;
fd = inotify_init();
if (fd < 0) {
perror("init inotify");
return 1;
}
buffer = malloc(1<<20);
if (!buffer) {
perror("buffer allocation");
return 1;
}
head = NULL;
while (*++argv) {
git_watch(*argv, &head, fd, NULL);
}
if (!head) {
printf("nothing to do\n");
return 0;
}
/* loop until there's an error or all watched elements are
* removed or made inaccessible
*/
while (head) {
i = read(fd, buffer, 1<<20);
if (i == 0 || (i < 0 && errno != EINTR && errno != EAGAIN))
break;
if (i < 0)
continue;
check_event(buffer, i, &head, fd);
}
if (i < 0) {
perror("reading event");
return 1;
}
return 0;
}
[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]
^ permalink raw reply
* Re: On running git via proxy
From: Jan Hudec @ 2007-12-10 20:42 UTC (permalink / raw)
To: Assim Deodia; +Cc: git
In-Reply-To: <fcf013560712072113s6b41b437t3176412356e258a6@mail.gmail.com>
On Sat, Dec 08, 2007 at 10:43:40 +0530, Assim Deodia wrote:
> I am a newbie to git. I am running git on my virtual machine which has
> the internet access via proxy through the host machine.
What exactly do you mean by internet here? Aren't you confusing it with the
web?
> I am unable to configure git to run via proxy. system proxy seems not
> to be working for git..
Depends on what kind of proxy it is. IMAP proxy won't work for git, mainly
because git is not using IMAP.
> Can you please guide me for the same?
Depends on what you want to do.
--
Jan 'Bulb' Hudec <bulb@ucw.cz>
^ permalink raw reply
* Re: backups with git and inotify
From: David Tweed @ 2007-12-10 21:18 UTC (permalink / raw)
To: Luciano Rocha; +Cc: git
In-Reply-To: <20071210202911.GA14738@bit.office.eurotux.com>
Hi, looks interesting project.
I've been doing something similar-ish (but periodically rather than
upon changes). Here are some rough off-the-cuff observations (probably
telling you things you already know).
Firstly, are you doing backups (be able to restore to n previous
states upon catastrophe) or archives (being able to lookup arbitrary
points in history to compare with current stuff, eg, for regressions)?
(Archiving is useful if you aren't in a disciplined enough project to
do rewritten proper commits but still want to be able to look around
and try to figure out what's caused regressions, etc.)
The scripts I use are at
http://www.personal.rdg.ac.uk/~sis05dst/chronoversion.tgz
but they're designed around archiving rather than backups.
On Dec 10, 2007 8:29 PM, Luciano Rocha <luciano@eurotux.com> wrote:
> The following is a work in progress. There are some problems in how I'm
> using git and recording the history:
>
> 1. I use an opened fd for each monitored directory (and subdirectories),
> (inotify_add_watch_at would be nice).
> I fchdir(fd) when a change happens to register and commit it.
I thought about trying to have a daemon using inotify to record the
git-add's/git-rm's but keeping the cron driven actual commits, and
looked at python support module. I didn't because firstly I wasn't
sure how far inotify scaled (the fact the Linux VFS maintainer insists
on calling it "idiotify" doesn't inspire confidence). If it was me,
I'd pull the git-commit outside your loop that does the git-add/git-rm
(see later comment about emacs, etc). Obviously if your buffer isn't
completely emptied you'll get a misleading granularity of commits, but
then I guess that'll happen anyway. I think inotify drops events if an
internal queue fills: personally I'd try to check for that and
initiate manually scanning if I detected that happening.
> 2. git-rm dir/file also removes <dir> if file was the only entry of
> <dir>. So, when committing the removal, git complains that it can't
> find cwd. So I record the parent directory, do the git command, check
> if getcwd() works, and if not do the commit in the parent directory.
>
> 3. git-rm (empty) directory fails
>
> 4. Changes aren't atomic, but I can live with that and I doubt I would
> be able to make it atomic without implementing a filesystem (FUSE or
> not).
With things like emacs that do update writes by writing a new file
with a temporary name and then copying it over the top of the old
file, you'll get presumably 3 commits. Is that acceptable?
> I can work around most of the problems, and rewrite to use recorded path
> names instead of directories fd, but before I do that, and while I'm
> at the beginning, I'd like to probe for opinions and suggestions.
The only other thing that occurs to me is whether you need any greater
support for stopping the automatic monitoring than just stopping the
daemon. Eg, what happens if you decide you need to recover a previous
version of a file. Git checks it out, presumably updates the index
itself and then inotify fires off a git-add that will want to write to
the index. Basically, I'm trying to think if there's any situation
where you can have a delete event that git causes, followed by a
creating some new content where delay in your program processing the
delete will cause the new content to be `lost'? (I know, I sould read
the code.) In chronoversion, the first thing it does is check for a
"suppress" file which stops it doing anything automatically and I put
one in there whenever I'm doing anything more than looking at the data
(eg, switch branch, checkout old version, etc). But I might be being
hyper-cautious.
--
cheers, dave tweed__________________________
david.tweed@gmail.com
Rm 124, School of Systems Engineering, University of Reading.
"we had no idea that when we added templates we were adding a Turing-
complete compile-time language." -- C++ standardisation committee
^ permalink raw reply
* [PATCH 4/5] Correctly initialize buffer in start_put() in http-push.c
From: Mike Hommey @ 2007-12-10 21:36 UTC (permalink / raw)
To: git; +Cc: Junio C Hamano
In-Reply-To: <1197322571-25023-3-git-send-email-mh@glandium.org>
Brown paper bag fix to avoid random misbehaviour.
Signed-off-by: Mike Hommey <mh@glandium.org>
---
This is a brown paper bag fix for my strbuf patch for the http code.
http-push.c | 1 +
1 files changed, 1 insertions(+), 0 deletions(-)
diff --git a/http-push.c b/http-push.c
index 2ef764c..ad8167e 100644
--- a/http-push.c
+++ b/http-push.c
@@ -495,6 +495,7 @@ static void start_put(struct transfer_request *request)
deflateInit(&stream, zlib_compression_level);
size = deflateBound(&stream, len + hdrlen);
strbuf_init(&request->buffer.buf, size);
+ request->buffer.posn = 0;
/* Compress it */
stream.next_out = (unsigned char *)request->buffer.buf.buf;
--
1.5.3.7.1159.gdd4a4
^ permalink raw reply related
* [PATCH 5/5] Fix various memory leaks in http-push.c and http-walker.c
From: Mike Hommey @ 2007-12-10 21:36 UTC (permalink / raw)
To: git; +Cc: Junio C Hamano
In-Reply-To: <1197322571-25023-4-git-send-email-mh@glandium.org>
Signed-off-by: Mike Hommey <mh@glandium.org>
---
This one, too, sits on top of my strbuf patch for the http code. Note that
I only went for the obvious ones I saw in the code I touched. There is more
in these files.
http-push.c | 32 +++++++++++++++++++++-----------
http-walker.c | 40 +++++++++++++++++++++++++---------------
2 files changed, 46 insertions(+), 26 deletions(-)
diff --git a/http-push.c b/http-push.c
index ad8167e..610ed9c 100644
--- a/http-push.c
+++ b/http-push.c
@@ -924,6 +924,7 @@ static int fetch_index(unsigned char *sha1)
hex);
}
} else {
+ free(url);
return error("Unable to start request");
}
@@ -1114,6 +1115,7 @@ int fetch_ref(char *ref, unsigned char *sha1)
char *base = remote->url;
struct active_request_slot *slot;
struct slot_results results;
+ int ret;
url = quote_ref_url(base, ref);
slot = get_active_slot();
@@ -1124,17 +1126,23 @@ int fetch_ref(char *ref, unsigned char *sha1)
curl_easy_setopt(slot->curl, CURLOPT_URL, url);
if (start_active_slot(slot)) {
run_active_slot(slot);
- if (results.curl_result != CURLE_OK)
- return error("Couldn't get %s for %s\n%s",
- url, ref, curl_errorstr);
+ if (results.curl_result == CURLE_OK) {
+ strbuf_rtrim(&buffer);
+ if (buffer.len == 40)
+ ret = get_sha1_hex(buffer.buf, sha1);
+ else
+ ret = 1;
+ } else {
+ ret = error("Couldn't get %s for %s\n%s",
+ url, ref, curl_errorstr);
+ }
} else {
- return error("Unable to start request");
+ ret = error("Unable to start request");
}
- strbuf_rtrim(&buffer);
- if (buffer.len != 40)
- return 1;
- return get_sha1_hex(buffer.buf, sha1);
+ strbuf_release(&buffer);
+ free(url);
+ return ret;
}
static void one_remote_object(const char *hex)
@@ -2033,6 +2041,7 @@ static int remote_exists(const char *path)
char *url = xmalloc(strlen(remote->url) + strlen(path) + 1);
struct active_request_slot *slot;
struct slot_results results;
+ int ret = -1;
sprintf(url, "%s%s", remote->url, path);
@@ -2044,16 +2053,17 @@ static int remote_exists(const char *path)
if (start_active_slot(slot)) {
run_active_slot(slot);
if (results.http_code == 404)
- return 0;
+ ret = 0;
else if (results.curl_result == CURLE_OK)
- return 1;
+ ret = 1;
else
fprintf(stderr, "HEAD HTTP error %ld\n", results.http_code);
} else {
fprintf(stderr, "Unable to start HEAD request\n");
}
- return -1;
+ free(url);
+ return ret;
}
static void fetch_symref(const char *path, char **symref, unsigned char *sha1)
diff --git a/http-walker.c b/http-walker.c
index 8dbf9cc..4e878b3 100644
--- a/http-walker.c
+++ b/http-walker.c
@@ -644,6 +644,7 @@ static int fetch_indices(struct walker *walker, struct alt_base *repo)
struct strbuf buffer = STRBUF_INIT;
char *data;
int i = 0;
+ int ret = 0;
struct active_request_slot *slot;
struct slot_results results;
@@ -666,19 +667,19 @@ static int fetch_indices(struct walker *walker, struct alt_base *repo)
if (start_active_slot(slot)) {
run_active_slot(slot);
if (results.curl_result != CURLE_OK) {
- strbuf_release(&buffer);
if (missing_target(&results)) {
repo->got_indices = 1;
- return 0;
+ goto cleanup;
} else {
repo->got_indices = 0;
- return error("%s", curl_errorstr);
+ ret = error("%s", curl_errorstr);
+ goto cleanup;
}
}
} else {
repo->got_indices = 0;
- strbuf_release(&buffer);
- return error("Unable to start request");
+ ret = error("Unable to start request");
+ goto cleanup;
}
data = buffer.buf;
@@ -701,9 +702,11 @@ static int fetch_indices(struct walker *walker, struct alt_base *repo)
i++;
}
- strbuf_release(&buffer);
repo->got_indices = 1;
- return 0;
+cleanup:
+ strbuf_release(&buffer);
+ free(url);
+ return ret;
}
static int fetch_pack(struct walker *walker, struct alt_base *repo, unsigned char *sha1)
@@ -939,6 +942,7 @@ static int fetch_ref(struct walker *walker, char *ref, unsigned char *sha1)
const char *base = data->alt->base;
struct active_request_slot *slot;
struct slot_results results;
+ int ret;
url = quote_ref_url(base, ref);
slot = get_active_slot();
@@ -949,17 +953,23 @@ static int fetch_ref(struct walker *walker, char *ref, unsigned char *sha1)
curl_easy_setopt(slot->curl, CURLOPT_URL, url);
if (start_active_slot(slot)) {
run_active_slot(slot);
- if (results.curl_result != CURLE_OK)
- return error("Couldn't get %s for %s\n%s",
- url, ref, curl_errorstr);
+ if (results.curl_result == CURLE_OK) {
+ strbuf_rtrim(&buffer);
+ if (buffer.len == 40)
+ ret = get_sha1_hex(buffer.buf, sha1);
+ else
+ ret = 1;
+ } else {
+ ret = error("Couldn't get %s for %s\n%s",
+ url, ref, curl_errorstr);
+ }
} else {
- return error("Unable to start request");
+ ret = error("Unable to start request");
}
- strbuf_rtrim(&buffer);
- if (buffer.len != 40)
- return 1;
- return get_sha1_hex(buffer.buf, sha1);
+ strbuf_release(&buffer);
+ free(url);
+ return ret;
}
static void cleanup(struct walker *walker)
--
1.5.3.7.1159.gdd4a4
^ permalink raw reply related
* [PATCH 1/5] Remove the default_headers variable from http-push.c
From: Mike Hommey @ 2007-12-10 21:36 UTC (permalink / raw)
To: git; +Cc: Junio C Hamano
It appears that despite being initialized, it was never used.
Signed-off-by: Mike Hommey <mh@glandium.org>
---
http-push.c | 7 -------
1 files changed, 0 insertions(+), 7 deletions(-)
diff --git a/http-push.c b/http-push.c
index a69d0e3..2ef764c 100644
--- a/http-push.c
+++ b/http-push.c
@@ -75,7 +75,6 @@ static int aborted;
static signed char remote_dir_exists[256];
static struct curl_slist *no_pragma_header;
-static struct curl_slist *default_headers;
static int push_verbosely;
static int push_all = MATCH_REFS_NONE;
@@ -2286,11 +2285,6 @@ int main(int argc, char **argv)
http_init();
no_pragma_header = curl_slist_append(no_pragma_header, "Pragma:");
- default_headers = curl_slist_append(default_headers, "Range:");
- default_headers = curl_slist_append(default_headers, "Destination:");
- default_headers = curl_slist_append(default_headers, "If:");
- default_headers = curl_slist_append(default_headers,
- "Pragma: no-cache");
/* Verify DAV compliance/lock support */
if (!locking_available()) {
@@ -2470,7 +2464,6 @@ int main(int argc, char **argv)
free(remote);
curl_slist_free_all(no_pragma_header);
- curl_slist_free_all(default_headers);
http_cleanup();
--
1.5.3.7.1159.gdd4a4
^ permalink raw reply related
* [PATCH 2/5] Remove a CURLOPT_HTTPHEADER (un)setting
From: Mike Hommey @ 2007-12-10 21:36 UTC (permalink / raw)
To: git; +Cc: Junio C Hamano
In-Reply-To: <1197322571-25023-1-git-send-email-mh@glandium.org>
Setting CURLOPT_HTTPHEADER doesn't add HTTP headers, but replaces whatever
set of headers was configured before, so setting to NULL doesn't have any
magic meaning, and is pretty much useless when setting to another list
right after.
Signed-off-by: Mike Hommey <mh@glandium.org>
---
http.c | 1 -
1 files changed, 0 insertions(+), 1 deletions(-)
diff --git a/http.c b/http.c
index 2d0b46d..784b93e 100644
--- a/http.c
+++ b/http.c
@@ -364,7 +364,6 @@ struct active_request_slot *get_active_slot(void)
slot->finished = NULL;
slot->callback_data = NULL;
slot->callback_func = NULL;
- curl_easy_setopt(slot->curl, CURLOPT_HTTPHEADER, NULL);
curl_easy_setopt(slot->curl, CURLOPT_HTTPHEADER, pragma_header);
curl_easy_setopt(slot->curl, CURLOPT_ERRORBUFFER, curl_errorstr);
curl_easy_setopt(slot->curl, CURLOPT_CUSTOMREQUEST, NULL);
--
1.5.3.7.1159.gdd4a4
^ permalink raw reply related
* [PATCH 3/5] Avoid redundant declaration of missing_target()
From: Mike Hommey @ 2007-12-10 21:36 UTC (permalink / raw)
To: git; +Cc: Junio C Hamano
In-Reply-To: <1197322571-25023-2-git-send-email-mh@glandium.org>
Signed-off-by: Mike Hommey <mh@glandium.org>
---
I though it's also small enough to grant inlining.
http-walker.c | 13 -------------
http.h | 13 +++++++++++++
transport.c | 13 -------------
3 files changed, 13 insertions(+), 26 deletions(-)
diff --git a/http-walker.c b/http-walker.c
index d9a5f1e..8dbf9cc 100644
--- a/http-walker.c
+++ b/http-walker.c
@@ -90,19 +90,6 @@ static size_t fwrite_sha1_file(void *ptr, size_t eltsize, size_t nmemb,
return size;
}
-static int missing__target(int code, int result)
-{
- return /* file:// URL -- do we ever use one??? */
- (result == CURLE_FILE_COULDNT_READ_FILE) ||
- /* http:// and https:// URL */
- (code == 404 && result == CURLE_HTTP_RETURNED_ERROR) ||
- /* ftp:// URL */
- (code == 550 && result == CURLE_FTP_COULDNT_RETR_FILE)
- ;
-}
-
-#define missing_target(a) missing__target((a)->http_code, (a)->curl_result)
-
static void fetch_alternates(struct walker *walker, const char *base);
static void process_object_response(void *callback_data);
diff --git a/http.h b/http.h
index a0fb4cf..87d638b 100644
--- a/http.h
+++ b/http.h
@@ -83,4 +83,17 @@ extern int active_requests;
extern char curl_errorstr[CURL_ERROR_SIZE];
+static inline int missing__target(int code, int result)
+{
+ return /* file:// URL -- do we ever use one??? */
+ (result == CURLE_FILE_COULDNT_READ_FILE) ||
+ /* http:// and https:// URL */
+ (code == 404 && result == CURLE_HTTP_RETURNED_ERROR) ||
+ /* ftp:// URL */
+ (code == 550 && result == CURLE_FTP_COULDNT_RETR_FILE)
+ ;
+}
+
+#define missing_target(a) missing__target((a)->http_code, (a)->curl_result)
+
#endif /* HTTP_H */
diff --git a/transport.c b/transport.c
index 22234e8..4e151a9 100644
--- a/transport.c
+++ b/transport.c
@@ -426,19 +426,6 @@ static int curl_transport_push(struct transport *transport, int refspec_nr, cons
return !!err;
}
-static int missing__target(int code, int result)
-{
- return /* file:// URL -- do we ever use one??? */
- (result == CURLE_FILE_COULDNT_READ_FILE) ||
- /* http:// and https:// URL */
- (code == 404 && result == CURLE_HTTP_RETURNED_ERROR) ||
- /* ftp:// URL */
- (code == 550 && result == CURLE_FTP_COULDNT_RETR_FILE)
- ;
-}
-
-#define missing_target(a) missing__target((a)->http_code, (a)->curl_result)
-
static struct ref *get_refs_via_curl(struct transport *transport)
{
struct strbuf buffer = STRBUF_INIT;
--
1.5.3.7.1159.gdd4a4
^ permalink raw reply related
* Re: backups with git and inotify
From: Luciano Rocha @ 2007-12-10 21:47 UTC (permalink / raw)
To: David Tweed; +Cc: git
In-Reply-To: <e1dab3980712101318v264fcce5pebbb829d8cefb1ac@mail.gmail.com>
[-- Attachment #1: Type: text/plain, Size: 5048 bytes --]
On Mon, Dec 10, 2007 at 09:18:18PM +0000, David Tweed wrote:
> Hi, looks interesting project.
>
> I've been doing something similar-ish (but periodically rather than
> upon changes). Here are some rough off-the-cuff observations (probably
> telling you things you already know).
Thanks.
>
> Firstly, are you doing backups (be able to restore to n previous
> states upon catastrophe) or archives (being able to lookup arbitrary
> points in history to compare with current stuff, eg, for regressions)?
> (Archiving is useful if you aren't in a disciplined enough project to
> do rewritten proper commits but still want to be able to look around
> and try to figure out what's caused regressions, etc.)
Archives, only. If the project requires a coherent state, then I don't
think that automatic commits are the way to go.
>
> The scripts I use are at
>
> http://www.personal.rdg.ac.uk/~sis05dst/chronoversion.tgz
>
> but they're designed around archiving rather than backups.
Thanks, I'll take a look, and maybe borrow some ideas. :)
>
> On Dec 10, 2007 8:29 PM, Luciano Rocha <luciano@eurotux.com> wrote:
> > The following is a work in progress. There are some problems in how I'm
> > using git and recording the history:
> >
> > 1. I use an opened fd for each monitored directory (and subdirectories),
> > (inotify_add_watch_at would be nice).
> > I fchdir(fd) when a change happens to register and commit it.
>
> I thought about trying to have a daemon using inotify to record the
> git-add's/git-rm's but keeping the cron driven actual commits, and
> looked at python support module. I didn't because firstly I wasn't
> sure how far inotify scaled (the fact the Linux VFS maintainer insists
> on calling it "idiotify" doesn't inspire confidence). If it was me,
> I'd pull the git-commit outside your loop that does the git-add/git-rm
> (see later comment about emacs, etc).
I'd like to have the changes committed as soon as an application closes
its files. As I monitor a small subset of possible inotify events, I
think I shouldn't have much problems with scale. I'll have to test it
with my Maildir, though, to have a definitive answer.
> Obviously if your buffer isn't
> completely emptied you'll get a misleading granularity of commits, but
> then I guess that'll happen anyway. I think inotify drops events if an
> internal queue fills: personally I'd try to check for that and
> initiate manually scanning if I detected that happening.
Hm, that could be a problem. Maybe a periodic git-status followed by
git-add/rm, etc.. Hourly, perhaps.
>
> > 2. git-rm dir/file also removes <dir> if file was the only entry of
> > <dir>. So, when committing the removal, git complains that it can't
> > find cwd. So I record the parent directory, do the git command, check
> > if getcwd() works, and if not do the commit in the parent directory.
> >
> > 3. git-rm (empty) directory fails
> >
> > 4. Changes aren't atomic, but I can live with that and I doubt I would
> > be able to make it atomic without implementing a filesystem (FUSE or
> > not).
>
> With things like emacs that do update writes by writing a new file
> with a temporary name and then copying it over the top of the old
> file, you'll get presumably 3 commits. Is that acceptable?
No. Vim also has that behaviour. I plan on accepting ignore patterns,
and maybe also parse .gitignore, and add those temporary files (*~,
.*.sw[po]), etc.) implicitly.
>
> > I can work around most of the problems, and rewrite to use recorded path
> > names instead of directories fd, but before I do that, and while I'm
> > at the beginning, I'd like to probe for opinions and suggestions.
>
> The only other thing that occurs to me is whether you need any greater
> support for stopping the automatic monitoring than just stopping the
> daemon. Eg, what happens if you decide you need to recover a previous
> version of a file. Git checks it out, presumably updates the index
> itself and then inotify fires off a git-add that will want to write to
> the index. Basically, I'm trying to think if there's any situation
> where you can have a delete event that git causes, followed by a
> creating some new content where delay in your program processing the
> delete will cause the new content to be `lost'? (I know, I sould read
> the code.) In chronoversion, the first thing it does is check for a
> "suppress" file which stops it doing anything automatically and I put
> one in there whenever I'm doing anything more than looking at the data
> (eg, switch branch, checkout old version, etc). But I might be being
> hyper-cautious.
I'll have to think about that. A stop/pause button is a good idea, and
checking if the tree is at HEAD. I don't think a commit of changes to a
file checked-out to a previous version will lose any information,
but I'll check.
--
Luciano Rocha <luciano@eurotux.com>
Eurotux Informática, S.A. <http://www.eurotux.com/>
[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]
^ permalink raw reply
* Re: backups with git and inotify
From: Björn Steinbrink @ 2007-12-10 21:57 UTC (permalink / raw)
To: Luciano Rocha; +Cc: git
In-Reply-To: <20071210202911.GA14738@bit.office.eurotux.com>
On 2007.12.10 20:29:11 +0000, Luciano Rocha wrote:
>
> Hello,
>
> The following is a work in progress. There are some problems in how I'm
> using git and recording the history:
>
> 1. I use an opened fd for each monitored directory (and subdirectories),
> (inotify_add_watch_at would be nice).
> I fchdir(fd) when a change happens to register and commit it.
>
> 2. git-rm dir/file also removes <dir> if file was the only entry of
> <dir>. So, when committing the removal, git complains that it can't
> find cwd. So I record the parent directory, do the git command, check
> if getcwd() works, and if not do the commit in the parent directory.
>
> 3. git-rm (empty) directory fails
>
> 4. Changes aren't atomic, but I can live with that and I doubt I would
> be able to make it atomic without implementing a filesystem (FUSE or
> not).
>
> I can work around most of the problems, and rewrite to use recorded path
> names instead of directories fd, but before I do that, and while I'm
> at the beginning, I'd like to probe for opinions and suggestions.
>
> So, please, suggest.
I posted an extremely simple bash script here:
http://lkml.org/lkml/2007/12/7/279
It just employs inotifywait to do all watching and just needs to
translate the events to the different git command. Did just glance over
your code, but it seems to do basically the same thing, just that it's a
lot shorter. The overhead of being a shell script is probably neglible,
as the amount of git calls are likely dominating anyway.
Feel free to ignore my comments on why I think that that is crap anyway
and do whatever you want with the script.
HTH
Björn
^ permalink raw reply
* [PATCH] diff: Make numstat machine friendly also for renames (and copies)
From: Jakub Narebski @ 2007-12-10 22:32 UTC (permalink / raw)
To: git
"git diff --numstat" used the same format as "git diff --stat" for
renamed (and copied) files, except that filenames were not shortened
when they didn't fit in the column width. This format is suitable for
human consumption, but it cannot be unambiguously parsed.
Instead of that always use final file name ("to" name) for numstat.
It is possible to find name before rename when name after is known.
This required to use pprint_rename (pretty print rename) during output
(in the show_stats function) and not during parsing (in diffstat_add
function).
Adding from_name field to struct diffstat_t makes is_renamed bitfield
redundant; nevertheless for the sake of clarity, readability and
making this patch minimal (and because it would not reduce memory
footprint) it was not removed, and its used not replaced by checking
from_name field.
Signed-off-by: Jakub Narebski <jnareb@gmail.com>
---
This would be useful for gitweb, later.
I hope I have made it in time before feature freeze...
diff.c | 15 ++++++++++++---
1 files changed, 12 insertions(+), 3 deletions(-)
diff --git a/diff.c b/diff.c
index f780e3e..38b9367 100644
--- a/diff.c
+++ b/diff.c
@@ -735,6 +735,7 @@ struct diffstat_t {
int alloc;
struct diffstat_file {
char *name;
+ char *from_name;
unsigned is_unmerged:1;
unsigned is_binary:1;
unsigned is_renamed:1;
@@ -755,11 +756,14 @@ static struct diffstat_file *diffstat_add(struct diffstat_t *diffstat,
}
diffstat->files[diffstat->nr++] = x;
if (name_b) {
- x->name = pprint_rename(name_a, name_b);
+ x->from_name = xstrdup(name_a);
+ x->name = xstrdup(name_b);
x->is_renamed = 1;
}
- else
+ else {
+ x->from_name = NULL;
x->name = xstrdup(name_a);
+ }
return x;
}
@@ -837,7 +841,7 @@ static void show_stats(struct diffstat_t* data, struct diff_options *options)
struct diffstat_file *file = data->files[i];
int change = file->added + file->deleted;
- if (!file->is_renamed) { /* renames are already quoted by pprint_rename */
+ if (!file->is_renamed) { /* renames will be quoted by pprint_rename */
struct strbuf buf;
strbuf_init(&buf, 0);
if (quote_c_style(file->name, &buf, NULL, 0)) {
@@ -846,6 +850,11 @@ static void show_stats(struct diffstat_t* data, struct diff_options *options)
} else {
strbuf_release(&buf);
}
+ } else {
+ char *qname = pprint_rename(file->from_name, file->name);
+ free(file->name);
+ free(file->from_name);
+ file->name = qname;
}
len = strlen(file->name);
--
1.5.3.7
^ permalink raw reply related
* v1.5.4 plans
From: Junio C Hamano @ 2007-12-10 22:37 UTC (permalink / raw)
To: git
In-Reply-To: <7vve78qhtf.fsf@gitster.siamese.dyndns.org>
People might have noticed that I've been ignoring most of the new
topics/enhancements for the past few days. Here is what I want to see
happen until we declare v1.5.4.
First, stabilize 'master' enough and tag v1.5.4-rc0 soon.
* Among what's already in 'next', Christian's "git help -w" enhancement
is the only candidate to be in v1.5.4. Johannes's "git remote" could
also be, but I've seen it fail tests when run in my k.org private
repository and haven't had chance to find time to diagnose it, so I'd
rather leave it after v1.5.4.
* Eric's sanely-compact mapping from SVN rev-ids to git commits saw a
positive feedback. I haven't carefully read that patch but it seemed
sane and I'd like to have it in v1.5.4.
* Please, everybody, no more new features until v1.5.4 final ships, and
please spend a bit more time on finding and fixing regressions than
you would spend time cooking your favorite new features. I do not
have infinite amount of time to comment on new feature patches while
concentrating on fixes at the same time.
There are outstanding issues that need to be resolved:
* I'd like to see the pack-object's memory performance issue resolved
before the release; two very capable people are looking into it and I
am fairly optimistic.
* We need to do something about "gc --aggressive". The documentation
removal patch from Linus, if it ever materializes, would be better
than nothing, but I have this nagging suspicion that the explosion is
merely a bad interation between -A and -f option to the repack, which
are not meant to be used together.
* We have a handful deprecation notices in the draft release notes, but
if I recall correctly, Nico wanted to add a few more. We need to
come up with a wording that is easy to understand for the end users
to decide which ancient versions will be affected.
"git help -w" will want to have the HTML pages installed, which means
we would need to add a new package to hold it in git.spec.in. I am
willing to work on the initial draft, but help in testing is very
much appreciated --- I do not work on RPM systems myself. The same
goes for "git help -i" which will want the INFO pages installed.
* I've seen t9119-git-svn-info.sh fail in my k.org private repository
and have been skipping the test, but this needs to be diagnosed and
fixed [*1*]. It could be just that the code is fine and the test is
not rejecting SVN that is too-old. I dunno.
* There have been quite a few HTTP paches from Mike Hommey. I'd like
to limit the changes only to fixes and trivially-correct clean-ups,
which means these will need to be looked at:
[PATCH 1/4] Cleanup variables in http.[ch]
[PATCH 2/4] Use strbuf in http code
[PATCH 1/5] Remove the default_headers variable from http-push.c
[PATCH 2/5] Remove a CURLOPT_HTTPHEADER (un)setting
[PATCH 3/5] Avoid redundant declaration of missing_target()
[PATCH 4/5] Correctly initialize buffer in start_put() in http-push.c
[PATCH 5/5] Fix various memory leaks in http-push.c and http-walker.c
Help in reviewing these from people who were involved in the http
part of the current codebase is very much appreciated.
If we can freeze by the end of the year, we may be able to release mid
January 2008.
[Footnote]
----------------------------------------------------------------
*1* t9119 first fails at the 6th test. Perhaps the test needs to check
svn version first and stop testing this feature. This test does not
fail on my personal box that has svn 1.4.2.
* expecting success:
(cd svnwc; svn info file) > expected.info-file &&
(cd gitwc; git-svn info file) > actual.info-file &&
git-diff expected.info-file actual.info-file
diff --git a/expected.info-file b/actual.info-file
index b1d57f4..997c927 100644
--- a/expected.info-file
+++ b/actual.info-file
@@ -10,6 +10,5 @@ Last Changed Author: junio
Last Changed Rev: 1
Last Changed Date: 2007-12-10 22:18:12 +0000 (Mon, 10 Dec 2007)
Text Last Updated: 2007-12-10 22:18:13 +0000 (Mon, 10 Dec 2007)
-Properties Last Updated: 2007-12-10 22:18:13 +0000 (Mon, 10 Dec 2007)
Checksum: 5bbf5a52328e7439ae6e719dfe712200
* FAIL 6: info file
(cd svnwc; svn info file) > expected.info-file &&
(cd gitwc; git-svn info file) > actual.info-file &&
git-diff expected.info-file actual.info-file
: hera t/master; svn --version
svn, version 1.3.2 (r19776)
compiled Jun 1 2006, 10:05:51
Copyright (C) 2000-2006 CollabNet.
Subversion is open source software, see http://subversion.tigris.org/
This product includes software developed by CollabNet (http://www.Collab.Net/).
The following repository access (RA) modules are available:
* ra_dav : Module for accessing a repository via WebDAV (DeltaV) protocol.
- handles 'http' scheme
- handles 'https' scheme
* ra_svn : Module for accessing a repository using the svn network protocol.
- handles 'svn' scheme
* ra_local : Module for accessing a repository on local disk.
- handles 'file' scheme
----------------------------------------------------------------
^ permalink raw reply related
* Using git with Eclipse
From: Wink Saville @ 2007-12-10 22:42 UTC (permalink / raw)
To: git
Hello,
I'm trying to use git on an Eclipse workspace and the .metadata
directory is chock full of files and was wondering what, if anything,
should be ignored. At the moment .history looks like a candidate for
ignoring there are probably others.
Wink Saville
^ permalink raw reply
* [PATCH (amend)] diff: Make numstat machine friendly also for renames (and copies)
From: Jakub Narebski @ 2007-12-10 22:55 UTC (permalink / raw)
To: git
In-Reply-To: <200712102332.53114.jnareb@gmail.com>
"git diff --numstat" used the same format as "git diff --stat" for
renamed (and copied) files, except that filenames were not shortened
when they didn't fit in the column width. This format is suitable for
human consumption, but it cannot be unambiguously parsed.
Instead of that always use final file name ("to" name) for numstat.
It is possible to find name before rename when name after is known.
This required to use pprint_rename (pretty print rename) during output
(in the show_stats function) and not during parsing (in diffstat_add
function).
Adding from_name field to struct diffstat_t makes is_renamed bitfield
redundant; nevertheless for the sake of clarity, readability and
making this patch minimal (and because it would not reduce memory
footprint) it was not removed, and its used not replaced by checking
from_name field.
Signed-off-by: Jakub Narebski <jnareb@gmail.com>
---
Sorry for mistake: I have tested this commit, corrected it... and forgot
to update patch to send.
Previous version of this patch (from 7 May 2007) used instead of current
only "to_name" format similar to git-diff-tree raw format for renames:
added deleted TAB path for "src" TAB path for "dst" LF
The problem was when -z option was used: how to separate end of record
from end of from_name and start of to_name. For git-diff we have status
to distinguish those; no such thing for numstat output. Previous version
of patch used (or was to use actually, because of error in the code)
added deleted TAB path for "src" NUL NUL path for "dst" NUL
when -z option was used.
This is left now for future --numstat-extended option...
diff.c | 22 +++++++++++++---------
1 files changed, 13 insertions(+), 9 deletions(-)
diff --git a/diff.c b/diff.c
index f780e3e..8039ac7 100644
--- a/diff.c
+++ b/diff.c
@@ -735,6 +735,7 @@ struct diffstat_t {
int alloc;
struct diffstat_file {
char *name;
+ char *from_name;
unsigned is_unmerged:1;
unsigned is_binary:1;
unsigned is_renamed:1;
@@ -755,11 +756,14 @@ static struct diffstat_file *diffstat_add(struct diffstat_t *diffstat,
}
diffstat->files[diffstat->nr++] = x;
if (name_b) {
- x->name = pprint_rename(name_a, name_b);
+ x->from_name = xstrdup(name_a);
+ x->name = xstrdup(name_b);
x->is_renamed = 1;
}
- else
+ else {
+ x->from_name = NULL;
x->name = xstrdup(name_a);
+ }
return x;
}
@@ -837,7 +841,7 @@ static void show_stats(struct diffstat_t* data, struct diff_options *options)
struct diffstat_file *file = data->files[i];
int change = file->added + file->deleted;
- if (!file->is_renamed) { /* renames are already quoted by pprint_rename */
+ if (!file->is_renamed) { /* renames will be quoted by pprint_rename */
struct strbuf buf;
strbuf_init(&buf, 0);
if (quote_c_style(file->name, &buf, NULL, 0)) {
@@ -846,6 +850,11 @@ static void show_stats(struct diffstat_t* data, struct diff_options *options)
} else {
strbuf_release(&buf);
}
+ } else {
+ char *qname = pprint_rename(file->from_name, file->name);
+ free(file->name);
+ free(file->from_name);
+ file->name = qname;
}
len = strlen(file->name);
@@ -982,12 +991,7 @@ static void show_numstat(struct diffstat_t* data, struct diff_options *options)
printf("-\t-\t");
else
printf("%d\t%d\t", file->added, file->deleted);
- if (!file->is_renamed) {
- write_name_quoted(file->name, stdout, options->line_termination);
- } else {
- fputs(file->name, stdout);
- putchar(options->line_termination);
- }
+ write_name_quoted(file->name, stdout, options->line_termination);
}
}
--
1.5.3.7
^ permalink raw reply related
* Re: [PATCH] diff: Make numstat machine friendly also for renames (and copies)
From: Junio C Hamano @ 2007-12-10 23:00 UTC (permalink / raw)
To: Jakub Narebski; +Cc: git
In-Reply-To: <200712102332.53114.jnareb@gmail.com>
Jakub Narebski <jnareb@gmail.com> writes:
> "git diff --numstat" used the same format as "git diff --stat" for
> renamed (and copied) files, except that filenames were not shortened
> when they didn't fit in the column width. This format is suitable for
> human consumption, but it cannot be unambiguously parsed.
Agreed about the (un)parsability, and --numstat is all about parsability
so I would not object. A fix is really needed there.
I do not have time to look at the patch right now, but if the changed
output is in line with what --name-status would show, that would be
great. I'd call that "the format that should have been from day one".
I.e. no '=>' rename marker, but show two names c-quoted (unless -z is
used) and separated with inter_name_termination). IIRC, that is how
rename/copy is shown with --name-status.
^ permalink raw reply
* [PATCH 6/5] Move fetch_ref from http-push.c and http-walker.c to http.c
From: Mike Hommey @ 2007-12-10 23:08 UTC (permalink / raw)
To: git; +Cc: Junio C Hamano
In-Reply-To: <1197322571-25023-5-git-send-email-mh@glandium.org>
Make the necessary changes to be ok with their difference, and rename the
function http_fetch_ref.
Signed-off-by: Mike Hommey <mh@glandium.org>
---
http-push.c | 88 ++------------------------------------------------------
http-walker.c | 80 +---------------------------------------------------
http.c | 82 +++++++++++++++++++++++++++++++++++++++++++++++++++++
http.h | 2 +
4 files changed, 89 insertions(+), 163 deletions(-)
diff --git a/http-push.c b/http-push.c
index 610ed9c..a4a9d1c 100644
--- a/http-push.c
+++ b/http-push.c
@@ -1063,88 +1063,6 @@ static int fetch_indices(void)
return 0;
}
-static inline int needs_quote(int ch)
-{
- if (((ch >= 'A') && (ch <= 'Z'))
- || ((ch >= 'a') && (ch <= 'z'))
- || ((ch >= '0') && (ch <= '9'))
- || (ch == '/')
- || (ch == '-')
- || (ch == '.'))
- return 0;
- return 1;
-}
-
-static inline int hex(int v)
-{
- if (v < 10) return '0' + v;
- else return 'A' + v - 10;
-}
-
-static char *quote_ref_url(const char *base, const char *ref)
-{
- const char *cp;
- char *dp, *qref;
- int len, baselen, ch;
-
- baselen = strlen(base);
- len = baselen + 1;
- for (cp = ref; (ch = *cp) != 0; cp++, len++)
- if (needs_quote(ch))
- len += 2; /* extra two hex plus replacement % */
- qref = xmalloc(len);
- memcpy(qref, base, baselen);
- for (cp = ref, dp = qref + baselen; (ch = *cp) != 0; cp++) {
- if (needs_quote(ch)) {
- *dp++ = '%';
- *dp++ = hex((ch >> 4) & 0xF);
- *dp++ = hex(ch & 0xF);
- }
- else
- *dp++ = ch;
- }
- *dp = 0;
-
- return qref;
-}
-
-int fetch_ref(char *ref, unsigned char *sha1)
-{
- char *url;
- struct strbuf buffer = STRBUF_INIT;
- char *base = remote->url;
- struct active_request_slot *slot;
- struct slot_results results;
- int ret;
-
- url = quote_ref_url(base, ref);
- slot = get_active_slot();
- slot->results = &results;
- curl_easy_setopt(slot->curl, CURLOPT_FILE, &buffer);
- curl_easy_setopt(slot->curl, CURLOPT_WRITEFUNCTION, fwrite_buffer);
- curl_easy_setopt(slot->curl, CURLOPT_HTTPHEADER, NULL);
- curl_easy_setopt(slot->curl, CURLOPT_URL, url);
- if (start_active_slot(slot)) {
- run_active_slot(slot);
- if (results.curl_result == CURLE_OK) {
- strbuf_rtrim(&buffer);
- if (buffer.len == 40)
- ret = get_sha1_hex(buffer.buf, sha1);
- else
- ret = 1;
- } else {
- ret = error("Couldn't get %s for %s\n%s",
- url, ref, curl_errorstr);
- }
- } else {
- ret = error("Unable to start request");
- }
-
- strbuf_release(&buffer);
- free(url);
- return ret;
-}
-
static void one_remote_object(const char *hex)
{
unsigned char sha1[20];
@@ -1827,7 +1745,8 @@ static void one_remote_ref(char *refname)
struct object *obj;
int len = strlen(refname) + 1;
- if (fetch_ref(refname, remote_sha1) != 0) {
+ if (http_fetch_ref(remote->url, refname + 5 /* "refs/" */,
+ remote_sha1) != 0) {
fprintf(stderr,
"Unable to fetch ref %s from %s\n",
refname, remote->url);
@@ -1959,7 +1878,8 @@ static void add_remote_info_ref(struct remote_ls_ctx *ls)
int len;
char *ref_info;
- if (fetch_ref(ls->dentry_name, remote_sha1) != 0) {
+ if (http_fetch_ref(remote->url, ls->dentry_name + 5 /* "refs/" */,
+ remote_sha1) != 0) {
fprintf(stderr,
"Unable to fetch ref %s from %s\n",
ls->dentry_name, remote->url);
diff --git a/http-walker.c b/http-walker.c
index 4e878b3..2c37868 100644
--- a/http-walker.c
+++ b/http-walker.c
@@ -888,88 +888,10 @@ static int fetch(struct walker *walker, unsigned char *sha1)
data->alt->base);
}
-static inline int needs_quote(int ch)
-{
- if (((ch >= 'A') && (ch <= 'Z'))
- || ((ch >= 'a') && (ch <= 'z'))
- || ((ch >= '0') && (ch <= '9'))
- || (ch == '/')
- || (ch == '-')
- || (ch == '.'))
- return 0;
- return 1;
-}
-
-static inline int hex(int v)
-{
- if (v < 10) return '0' + v;
- else return 'A' + v - 10;
-}
-
-static char *quote_ref_url(const char *base, const char *ref)
-{
- const char *cp;
- char *dp, *qref;
- int len, baselen, ch;
-
- baselen = strlen(base);
- len = baselen + 7; /* "/refs/" + NUL */
- for (cp = ref; (ch = *cp) != 0; cp++, len++)
- if (needs_quote(ch))
- len += 2; /* extra two hex plus replacement % */
- qref = xmalloc(len);
- memcpy(qref, base, baselen);
- memcpy(qref + baselen, "/refs/", 6);
- for (cp = ref, dp = qref + baselen + 6; (ch = *cp) != 0; cp++) {
- if (needs_quote(ch)) {
- *dp++ = '%';
- *dp++ = hex((ch >> 4) & 0xF);
- *dp++ = hex(ch & 0xF);
- }
- else
- *dp++ = ch;
- }
- *dp = 0;
-
- return qref;
-}
-
static int fetch_ref(struct walker *walker, char *ref, unsigned char *sha1)
{
- char *url;
- struct strbuf buffer = STRBUF_INIT;
struct walker_data *data = walker->data;
- const char *base = data->alt->base;
- struct active_request_slot *slot;
- struct slot_results results;
- int ret;
-
- url = quote_ref_url(base, ref);
- slot = get_active_slot();
- slot->results = &results;
- curl_easy_setopt(slot->curl, CURLOPT_FILE, &buffer);
- curl_easy_setopt(slot->curl, CURLOPT_WRITEFUNCTION, fwrite_buffer);
- curl_easy_setopt(slot->curl, CURLOPT_HTTPHEADER, NULL);
- curl_easy_setopt(slot->curl, CURLOPT_URL, url);
- if (start_active_slot(slot)) {
- run_active_slot(slot);
- if (results.curl_result == CURLE_OK) {
- strbuf_rtrim(&buffer);
- if (buffer.len == 40)
- ret = get_sha1_hex(buffer.buf, sha1);
- else
- ret = 1;
- } else {
- ret = error("Couldn't get %s for %s\n%s",
- url, ref, curl_errorstr);
- }
- } else {
- ret = error("Unable to start request");
- }
-
- strbuf_release(&buffer);
- free(url);
- return ret;
+ return http_fetch_ref(data->alt->base, ref, sha1);
}
static void cleanup(struct walker *walker)
diff --git a/http.c b/http.c
index 784b93e..c6de964 100644
--- a/http.c
+++ b/http.c
@@ -552,3 +552,85 @@ void finish_all_active_slots(void)
slot = slot->next;
}
}
+
+static inline int needs_quote(int ch)
+{
+ if (((ch >= 'A') && (ch <= 'Z'))
+ || ((ch >= 'a') && (ch <= 'z'))
+ || ((ch >= '0') && (ch <= '9'))
+ || (ch == '/')
+ || (ch == '-')
+ || (ch == '.'))
+ return 0;
+ return 1;
+}
+
+static inline int hex(int v)
+{
+ if (v < 10) return '0' + v;
+ else return 'A' + v - 10;
+}
+
+static char *quote_ref_url(const char *base, const char *ref)
+{
+ const char *cp;
+ char *dp, *qref;
+ int len, baselen, ch;
+
+ baselen = strlen(base);
+ len = baselen + 7; /* "/refs/" + NUL */
+ for (cp = ref; (ch = *cp) != 0; cp++, len++)
+ if (needs_quote(ch))
+ len += 2; /* extra two hex plus replacement % */
+ qref = xmalloc(len);
+ memcpy(qref, base, baselen);
+ memcpy(qref + baselen, "/refs/", 6);
+ for (cp = ref, dp = qref + baselen + 6; (ch = *cp) != 0; cp++) {
+ if (needs_quote(ch)) {
+ *dp++ = '%';
+ *dp++ = hex((ch >> 4) & 0xF);
+ *dp++ = hex(ch & 0xF);
+ }
+ else
+ *dp++ = ch;
+ }
+ *dp = 0;
+
+ return qref;
+}
+
+int http_fetch_ref(const char *base, const char *ref, unsigned char *sha1)
+{
+ char *url;
+ struct strbuf buffer = STRBUF_INIT;
+ struct active_request_slot *slot;
+ struct slot_results results;
+ int ret;
+
+ url = quote_ref_url(base, ref);
+ slot = get_active_slot();
+ slot->results = &results;
+ curl_easy_setopt(slot->curl, CURLOPT_FILE, &buffer);
+ curl_easy_setopt(slot->curl, CURLOPT_WRITEFUNCTION, fwrite_buffer);
+ curl_easy_setopt(slot->curl, CURLOPT_HTTPHEADER, NULL);
+ curl_easy_setopt(slot->curl, CURLOPT_URL, url);
+ if (start_active_slot(slot)) {
+ run_active_slot(slot);
+ if (results.curl_result == CURLE_OK) {
+ strbuf_rtrim(&buffer);
+ if (buffer.len == 40)
+ ret = get_sha1_hex(buffer.buf, sha1);
+ else
+ ret = 1;
+ } else {
+ ret = error("Couldn't get %s for %s\n%s",
+ url, ref, curl_errorstr);
+ }
+ } else {
+ ret = error("Unable to start request");
+ }
+
+ strbuf_release(&buffer);
+ free(url);
+ return ret;
+}
diff --git a/http.h b/http.h
index 87d638b..b709222 100644
--- a/http.h
+++ b/http.h
@@ -96,4 +96,6 @@ static inline int missing__target(int code, int result)
#define missing_target(a) missing__target((a)->http_code, (a)->curl_result)
+extern int http_fetch_ref(const char *base, const char *ref, unsigned char *sha1);
+
#endif /* HTTP_H */
--
1.5.3.7.1160.g1e7a-dirty
^ permalink raw reply related
* Re: [PATCH] diff: Make numstat machine friendly also for renames (and copies)
From: Jakub Narebski @ 2007-12-10 23:14 UTC (permalink / raw)
To: Junio C Hamano; +Cc: git
In-Reply-To: <7vir36jgty.fsf@gitster.siamese.dyndns.org>
Junio C Hamano wrote:
> Jakub Narebski <jnareb@gmail.com> writes:
>
> > "git diff --numstat" used the same format as "git diff --stat" for
> > renamed (and copied) files, except that filenames were not shortened
> > when they didn't fit in the column width. This format is suitable for
> > human consumption, but it cannot be unambiguously parsed.
>
> Agreed about the (un)parsability, and --numstat is all about parsability
> so I would not object. A fix is really needed there.
>
> I do not have time to look at the patch right now, but if the changed
> output is in line with what --name-status would show, that would be
> great. I'd call that "the format that should have been from day one".
>
> I.e. no '=>' rename marker, but show two names c-quoted (unless -z is
> used) and separated with inter_name_termination). IIRC, that is how
> rename/copy is shown with --name-status.
Unfortunately this is not possible, at least if we want to retain
the assertion that -z output looks like normal output, only without
quoting.
diff --name-status has _status_ field which can be used to distinguish
if the NUL (for -z output) is the end of source filename, or the end
of record.
The patch send changes --numstat to use only _destination_ name.
What you want I'd left for futore --numstat-extended (basically --numstat,
but with status field.
--
Jakub Narebski
Poland
^ permalink raw reply
* Re: v1.5.4 plans
From: Jeff King @ 2007-12-10 23:49 UTC (permalink / raw)
To: Junio C Hamano; +Cc: git
In-Reply-To: <7vmysijhwq.fsf_-_@gitster.siamese.dyndns.org>
On Mon, Dec 10, 2007 at 02:37:09PM -0800, Junio C Hamano wrote:
> * Please, everybody, no more new features until v1.5.4 final ships, and
> please spend a bit more time on finding and fixing regressions than
> you would spend time cooking your favorite new features. I do not
> have infinite amount of time to comment on new feature patches while
> concentrating on fixes at the same time.
>
> There are outstanding issues that need to be resolved:
A few regressions that you did not mention, but I think should be
addressed before 1.5.4:
- extra newline in builtin-commit output. You found a case that
needs it, but fixing it is non-trivial, and I wanted to get your
input before preparing a patch. See
http://mid.gmane.org/20071203075357.GB3614@sigill.intra.peff.net
- git-clean's handling of directory wildcards. I didn't get a response
to
http://mid.gmane.org/20071206043247.GC5499@coredump.intra.peff.net
I suspect there are still some bugs lurking in there, but it's hard
to say because I don't know what the behavior _should_ be (there are
some test cases in that email).
And perhaps not a regression, but I think we should bring git-svn's
handling of color.* in line with the changes to the rest of the code
before 1.5.4. I posted a "last resort" patch, but I think with your
changes to "git config --colorbool" it might be possible to use that.
I'll try to work up a new patch.
-Peff
^ permalink raw reply
* Re: [PATCH (amend)] diff: Make numstat machine friendly also for renames (and copies)
From: Junio C Hamano @ 2007-12-11 1:06 UTC (permalink / raw)
To: Jakub Narebski; +Cc: git
In-Reply-To: <200712102355.39084.jnareb@gmail.com>
Jakub Narebski <jnareb@gmail.com> writes:
> Previous version of this patch (from 7 May 2007) used instead of current
> only "to_name" format similar to git-diff-tree raw format for renames:
>
> added deleted TAB path for "src" TAB path for "dst" LF
>
> The problem was when -z option was used: how to separate end of record
> from end of from_name and start of to_name. For git-diff we have status
> to distinguish those; no such thing for numstat output. Previous version
> of patch used (or was to use actually, because of error in the code)
>
> added deleted TAB path for "src" NUL NUL path for "dst" NUL
>
> when -z option was used.
I think the cleanest at this point is to have --numstat-enhanced that
shows
<added> <deleted> <status> <path1>
<added> <deleted> <status> <path1> <path2>
Anything else would be a regression.
^ permalink raw reply
* Re: git help -t <topic>: list the help of the commands in a given topic
From: Junio C Hamano @ 2007-12-11 1:07 UTC (permalink / raw)
To: Santi Béjar; +Cc: Git Mailing List
In-Reply-To: <1197299021-28463-1-git-send-email-sbejar@gmail.com>
Santi Béjar <sbejar@gmail.com> writes:
> With 'git help -t' lists the available topics.
>
> Show a hint to get a longer list when listing the common commands.
I like the idea of making the categorized command list in git(7)
available, and agree with you that renaming common_cmds[] to cmd_list[]
and place everything in there would be the way to go.
However, I doubt about your presentation. Who are the intended
audience and what is the expected way this is used?
I highly suspect that it would be much easier to use if you add a mode
to "git help" that runs the pager over the categoized command list part
of git(7) manual page, without taking "show me list of topics" nor "show
commands only in this topic" parameters. It is highly unlikely that a
user knows which category an obscure command whose name he wants to
recall is in, or can guess which category it would be classified in
after seeing the "category list". It would be much more likely that he
finds it easier to scan (perhaps with "/<string>") the command list with
one line description in the pager.
^ permalink raw reply
* Re: [PATCH (amend)] diff: Make numstat machine friendly also for renames (and copies)
From: Jakub Narebski @ 2007-12-11 1:26 UTC (permalink / raw)
To: Junio C Hamano; +Cc: git
In-Reply-To: <7vejdujazu.fsf@gitster.siamese.dyndns.org>
Junio C Hamano wrote:
> Jakub Narebski <jnareb@gmail.com> writes:
>
> > Previous version of this patch (from 7 May 2007) used instead of current
> > only "to_name" format similar to git-diff-tree raw format for renames:
> >
> > added deleted TAB path for "src" TAB path for "dst" LF
> >
> > The problem was when -z option was used: how to separate end of record
> > from end of from_name and start of to_name. For git-diff we have status
> > to distinguish those; no such thing for numstat output. Previous version
> > of patch used (or was to use actually, because of error in the code)
> >
> > added deleted TAB path for "src" NUL NUL path for "dst" NUL
> >
> > when -z option was used.
>
> I think the cleanest at this point is to have --numstat-enhanced that
> shows
>
> <added> <deleted> <status> <path1>
> <added> <deleted> <status> <path1> <path2>
>
> Anything else would be a regression.
That is the plan[*1*]. Nevertheless always using destination filename for
"ordinary" numstat is a step in good direction. I don't think that would
break _any_ scripts (as previous version was not good to be parsed by
a machine); I think it is even more probable that old version _broke_
scripts if -M / -C was provided.
[*1*] When I (or somebody else) find time for that.
--
Jakub Narebski
Poland
^ permalink raw reply
* Re: v1.5.4 plans
From: Junio C Hamano @ 2007-12-11 1:27 UTC (permalink / raw)
To: Jeff King; +Cc: git
In-Reply-To: <20071210234941.GE22254@coredump.intra.peff.net>
Jeff King <peff@peff.net> writes:
> A few regressions that you did not mention, but I think should be
> addressed before 1.5.4:
>
> - extra newline in builtin-commit output. You found a case that
> needs it, but fixing it is non-trivial, and I wanted to get your
> input before preparing a patch. See
>
> http://mid.gmane.org/20071203075357.GB3614@sigill.intra.peff.net
I am actually becoming somewhat fond of the newline that makes the end
of a session that led to a commit stand out ;-). IOW, I was wondering if
we can have another for a merge commit case.
But I suspect that it amounts to the change in the same area and of
similar complexity.
> - git-clean's handling of directory wildcards. I didn't get a response
> to
>
> http://mid.gmane.org/20071206043247.GC5499@coredump.intra.peff.net
>
> I suspect there are still some bugs lurking in there, but it's hard
> to say because I don't know what the behavior _should_ be (there are
> some test cases in that email).
The last time I looked at the "directory" side of builtin-clean.c, I had
to quickly reach for my barf bag. I never use "git clean" without "-n"
and I never ever use "git clean" with "-d"; I do not have any idea what
behaviour when given "-d" would be useful. AFAIU, the scripted version
did not have clear semantics either.
Another thing that irritates me is it talks about not removing a
directory when run "git clean -n" (without -d). I did not ask it to
remove directories, so I did not expect it to talk about it not doing
what I did not ask it to.
> And perhaps not a regression, but I think we should bring git-svn's
> handling of color.* in line with the changes to the rest of the code
> before 1.5.4. I posted a "last resort" patch, but I think with your
> changes to "git config --colorbool" it might be possible to use that.
> I'll try to work up a new patch.
Thanks for a reminder. Anything else?
^ permalink raw reply
* Re: Something is broken in repack
From: Jon Smirl @ 2007-12-11 2:25 UTC (permalink / raw)
To: Git Mailing List, Nicolas Pitre
In-Reply-To: <9e4733910712071505y6834f040k37261d65a2d445c4@mail.gmail.com>
New run using same configuration. With the addition of the more
efficient load balancing patches and delta cache accounting.
Seconds are wall clock time. They are lower since the patch made
threading better at using all four cores. I am stuck at 380-390% CPU
utilization for the git process.
complete seconds RAM
10% 60 900M (includes counting)
20% 15 900M
30% 15 900M
40% 50 1.2G
50% 80 1.3G
60% 70 1.7G
70% 140 1.8G
80% 180 2.0G
90% 280 2.2G
95% 530 2.8G - 1,420 total to here, previous was 1,983
100% 1390 2.85G
During the writing phase RAM fell to 1.6G
What is being freed in the writing phase??
I have no explanation for the change in RAM usage. Two guesses come to
mind. Memory fragmentation. Or the change in the way the work was
split up altered RAM usage.
Total CPU time was 195 minutes in 70 minutes clock time. About 70%
efficient. During the compress phase all four cores were active until
the last 90 seconds. Writing the objects took over 23 minutes CPU
bound on one core.
New pack file is: 270,594,853
Old one was: 344,543,752
It still has 828,660 objects
On 12/7/07, Jon Smirl <jonsmirl@gmail.com> wrote:
> Using this config:
> [pack]
> threads = 4
> deltacachesize = 256M
> deltacachelimit = 0
>
> And the 330MB gcc pack for input
> git repack -a -d -f --depth=250 --window=250
>
> complete seconds RAM
> 10% 47 1GB
> 20% 29 1Gb
> 30% 24 1Gb
> 40% 18 1GB
> 50% 110 1.2GB
> 60% 85 1.4GB
> 70% 195 1.5GB
> 80% 186 2.5GB
> 90% 489 3.8GB
> 95% 800 4.8GB
> I killed it because it started swapping
>
> The mmaps are only about 400MB in this case.
> At the end the git process had 4.4GB of physical RAM allocated.
>
> Starting from a highly compressed pack greatly aggravates the problem.
> Starting with a 2GB pack of the same data my process size only grew to
> 3GB with 2GB of mmaps.
>
> --
> Jon Smirl
> jonsmirl@gmail.com
>
--
Jon Smirl
jonsmirl@gmail.com
^ permalink raw reply
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox