* Re: git-reflog 70 minutes at 100% cpu and counting
From: Sverre Rabbelier @ 2009-12-14 20:41 UTC (permalink / raw)
To: Eric Paris; +Cc: git
In-Reply-To: <1260822484.9379.53.camel@localhost>
Heya,
On Mon, Dec 14, 2009 at 21:28, Eric Paris <eparis@redhat.com> wrote:
> What can I collect, do, whatever?
If this really is a case that we end up wanting to optimize somehow,
it would probably be very helpful to make a copy of the repository
state _before_ the gc is done.
Also, 1.5.5 is really really old in git terms, consider compiling your
own. Something post 1.6.4 might be a good idea :).
--
Cheers,
Sverre Rabbelier
^ permalink raw reply
* Re: am fails to apply patches for files with CRLF lineendings
From: Björn Steinbrink @ 2009-12-14 20:55 UTC (permalink / raw)
To: Jason King; +Cc: Junio C Hamano, git, Brandon Casey
In-Reply-To: <776A5AB0-E6BC-4230-800E-BE1B7A6B09BF@silentcow.com>
On 2009.12.14 12:38:30 -0800, Jason King wrote:
> On Dec 14, 2009, at 12:27 PM, Junio C Hamano wrote:
>
> >Björn Steinbrink <B.Steinbrink@gmx.de> writes:
> >
> >>Commit c2ca1d7 "Allow mailsplit ... to handle mails with CRLF
> >>line-endings"
> >>seems to be responsible.
> >
> >Yes, that commit is not only responsible but was deliberate. For
> >a better
> >backstory, see:
> >
> > http://thread.gmane.org/gmane.comp.version-control.git/124718/focus=124721
> >
> >You'd notice that I was one of the people who didn't want to have this
> >change, so you don't need to convince _me_ that this was not a
> >change to
> >keep everybody happy, but you'd need to try a better job than I
> >did back
> >then to convince people who thought that "am" should directly work on
> >"Thunderbird saved mails" that what they want was a bad idea X-<.
>
> I dunno Junio, the back story doesn't really seem like a convincing
> argument for totally breaking am's handling of real CRLFs. Right
> now, it seems to be a very bad thing that git can create a patch
> that it can't apply. The default am should always be able to apply
> whatever format-patch has generated.
>
> If it's desirable to have am translate CRLFs to LFs, then why not
> provide this as an option to am so as not to break merging of real
> CRLF patches? Eg.:
>
> git am --convert-crlf
Hm, currently it checks everyline. I didn't really think this through,
but wouldn't it makes sense to have a per-mail flag, that checks just
the first line, and if that has CRLF, then enable the dropping (unless
--keep-cr is given), otherwise, keep things verbatim. That should (I
think) make things work with non-messed up patches regardless of the
files being patch, as well as with messed up patches as long as they
don't try to patch files with CRLF line-endings.
I can't seem to come up with a clean patch though (btw, is the comment
for split_one that says that buf may contain a partial line still
true?), but maybe I'll find some time to try again later this week
(don't count on it though, I'm pretty stressed...)
Björn
^ permalink raw reply
* Re: git-reflog 70 minutes at 100% cpu and counting
From: Jeff King @ 2009-12-14 21:11 UTC (permalink / raw)
To: Eric Paris; +Cc: git
In-Reply-To: <1260822484.9379.53.camel@localhost>
On Mon, Dec 14, 2009 at 03:28:04PM -0500, Eric Paris wrote:
> So I have no idea what is interesting or relevant what I can collect,
> what you want to know or anything like that, so this is a bit of a dump
> of info and I'm sorry to whoever tries to pick anything useful out of
It sounds like you might have found an infinite loop, as reflog should
never really need a lot of CPU. Is it possible to tar the whole
repository and make it available publicly for us to look at?
-Peff
^ permalink raw reply
* Re: am fails to apply patches for files with CRLF lineendings
From: Jason King @ 2009-12-14 21:16 UTC (permalink / raw)
To: Björn Steinbrink; +Cc: Junio C Hamano, git, Brandon Casey
In-Reply-To: <20091214205517.GA1317@atjola.homenet>
On Dec 14, 2009, at 12:55 PM, Björn Steinbrink wrote:
> On 2009.12.14 12:38:30 -0800, Jason King wrote:
>> On Dec 14, 2009, at 12:27 PM, Junio C Hamano wrote:
>>
>>> Björn Steinbrink <B.Steinbrink@gmx.de> writes:
>>>
>>>> Commit c2ca1d7 "Allow mailsplit ... to handle mails with CRLF
>>>> line-endings"
>>>> seems to be responsible.
>>>
>>> Yes, that commit is not only responsible but was deliberate. For
>>> a better
>>> backstory, see:
>>>
>>> http://thread.gmane.org/gmane.comp.version-control.git/124718/focus=124721
>>>
>>> You'd notice that I was one of the people who didn't want to have
>>> this
>>> change, so you don't need to convince _me_ that this was not a
>>> change to
>>> keep everybody happy, but you'd need to try a better job than I
>>> did back
>>> then to convince people who thought that "am" should directly work
>>> on
>>> "Thunderbird saved mails" that what they want was a bad idea X-<.
>>
>> I dunno Junio, the back story doesn't really seem like a convincing
>> argument for totally breaking am's handling of real CRLFs. Right
>> now, it seems to be a very bad thing that git can create a patch
>> that it can't apply. The default am should always be able to apply
>> whatever format-patch has generated.
>>
>> If it's desirable to have am translate CRLFs to LFs, then why not
>> provide this as an option to am so as not to break merging of real
>> CRLF patches? Eg.:
>>
>> git am --convert-crlf
>
> Hm, currently it checks everyline. I didn't really think this through,
> but wouldn't it makes sense to have a per-mail flag, that checks just
> the first line, and if that has CRLF, then enable the dropping (unless
> --keep-cr is given), otherwise, keep things verbatim. That should (I
> think) make things work with non-messed up patches regardless of the
> files being patch, as well as with messed up patches as long as they
> don't try to patch files with CRLF line-endings.
Sure, it won't affect LFers at all if you test the header lines of the
patch file - we'll be immune from that conversion then. Works for me :)
Thanks again,
Jason
^ permalink raw reply
* Re: git-reflog 70 minutes at 100% cpu and counting
From: Eric Paris @ 2009-12-14 21:20 UTC (permalink / raw)
To: Jeff King; +Cc: git
In-Reply-To: <20091214211142.GC9364@coredump.intra.peff.net>
On Mon, 2009-12-14 at 16:11 -0500, Jeff King wrote:
> On Mon, Dec 14, 2009 at 03:28:04PM -0500, Eric Paris wrote:
>
> > So I have no idea what is interesting or relevant what I can collect,
> > what you want to know or anything like that, so this is a bit of a dump
> > of info and I'm sorry to whoever tries to pick anything useful out of
>
> It sounds like you might have found an infinite loop, as reflog should
> never really need a lot of CPU. Is it possible to tar the whole
> repository and make it available publicly for us to look at?
Updated to git-1.6.5.3-1 from Fedora rawhide and still git reflog ran
for >5 minutes at 100% cpu (I killed it, it didn't finish)
I'm pushing a copy of the whole repo (all 1.9G after bzip compression)
to
http://people.redhat.com/~eparis/git-tar/
But it's going to take a couple hours.
-Eric
^ permalink raw reply
* Re: [PATCH] help.autocorrect: do not run a command if the command given is junk
From: Johannes Schindelin @ 2009-12-14 21:09 UTC (permalink / raw)
To: Junio C Hamano; +Cc: Johannes Sixt, Git Mailing List, Alex Riesen
In-Reply-To: <7v7hspjp3q.fsf@alter.siamese.dyndns.org>
Hi,
On Mon, 14 Dec 2009, Junio C Hamano wrote:
> I am curious about and would prefer to see the story behind '6' someday.
The '6' as a cut-off to the levenshtein distances we list when
autocorrecting was derived in a totally scientific manner:
1) first I implemented Levenshtein-Damerau with a configurable weight of
neighbor flips ("switches"), substitutions, additions and deletions,
2) next I patched the code to sort the availablecommands by their distance
to the mispelt command,
3) as this lists way too much, I implemented a cut-off that was
configurable by an environment variable (without any safety checks, as
I did not plan to release that code anyway),
4) now comes the totally, unbelievably cunningly scientific part: I did a
self-experiment! I deliberately mispelt commands in a totally random
manner!
5) then I changed the code to actually output the distances so I could
determine a cut-off that makes sense with my type of tyops,
6) after about 15 tries of deliberate mistakes (mostly doing what I
usually do, something like "git pull" and "git log" or something like
that, but watching TV, chatting on the phone _and_ cleaning the dishes
at the same time), I found that 5 was too low and 7 too large.
The number '6' happily coincided with the number of steps I needed to come
up with the number. You see? The _perfect_ way to determine a completely
arbitrary number.
Actually, you probably see that I just made up that number and tested a
few times, and it seemed to work reasonably well.
FWIW almost the same procedure led to the weights 0, 2, 1 and 4 that you
see in help.c. The weights are basically factors with which mistakes are
punished: if you just confuse two adjacent letters, such as "psuh" instead
of "push" (which can be quite common if you use two hands, one on the left
side, and one on the right side of the keyboard, with an en-US layout so
many of us use, myself included) it costs 0.
If you write a different character than what you intended, the cost is 2.
The idea behind it is that you're more likely to miss a key than to hit
the wrong key. With the laptop I am typing this email on, it is
particularly likely that I miss a key, because there are certain
key combinations where only the first key triggers an input event, but the
second only triggers an input event when it is _released_ after the first
one. So when I type "er" real fast and happen to release the "e" key
after the "r" key, no "r" appears on my screen.
Okay, so the weight for adding a character must be smaller than
substituting a character, but why is the cost for deletion so high?
Well, I really rarely type unnecessary characters (except when writing to
the Git mailing list, arguably) so those costs must be substantially
higher than for typing the wrong character.
My original plan was to log all my tyops into a log file and analyze those
errors later, but then my initial 0, 2, 1, 4 and 6 constants worked well
enough for me that I did not bother.
Satisfied?
Ciao,
Dscho
^ permalink raw reply
* Re: git-reflog 70 minutes at 100% cpu and counting
From: Jeff King @ 2009-12-14 21:23 UTC (permalink / raw)
To: Eric Paris; +Cc: git
In-Reply-To: <1260825629.9379.56.camel@localhost>
On Mon, Dec 14, 2009 at 04:20:29PM -0500, Eric Paris wrote:
> Updated to git-1.6.5.3-1 from Fedora rawhide and still git reflog ran
> for >5 minutes at 100% cpu (I killed it, it didn't finish)
>
> I'm pushing a copy of the whole repo (all 1.9G after bzip compression)
> to
>
> http://people.redhat.com/~eparis/git-tar/
Wowzers, that's big. Can you send just what's in .git?
-Peff
^ permalink raw reply
* Re: Help: approach for rebasing to older commits after merging more recent commits
From: Junio C Hamano @ 2009-12-14 21:33 UTC (permalink / raw)
To: Jay Soffian; +Cc: git
In-Reply-To: <76718490912141238k4ca1ba55jeff928efe875f020@mail.gmail.com>
Jay Soffian <jaysoffian@gmail.com> writes:
> [He asks again...]
>
> I have an interesting problem I'm not sure how best to tackle.
>
> A small development team is basing its product on an upstream git repo that is
> itself an svn clone. Currently the process looks like this:
>
> r1--r2--r3--r4--r5 upstream trunk (git svn clone)
> \ \ \
> A---B---C---D---E local trunk (git clone of upstream)
> \ /
> F---G developerN trunk (git clones of local)
>
> So local trunk has both daily merges from the local developers, as well as
> less periodic (typically weekly) merges from upstream trunk. The reason being
> that it is necessary to remain on top of the upstream bleeding edge.
>
> This works out okay, but there is a minor problem and a major problem.
>
> The minor problem is that the local trunk is cluttered with the developerN
> merges. That is easy to solve by having local developers rebase before pushing
> to local trunk. That would look like:
>
> r1--r2--r3--r4--r5 upstream trunk
> \ \ \
> A---B---C---D---E---F'---G' local trunk
>
> The major problem is that local trunk is also cluttered with merges from
> upstream. The is a problem because at some point in the future, upstream
> is going to declare some rN as being officially blessed. And we're going to
> want to rewind any rN changes past that point.
>
> So the question is, what's the best way to do this? Say r2 is blessed by
> upstream.
You should re-think your earlier claim:
The reason being that it is necessary to remain on top of the upstream
bleeding edge.
and then think about ways you can take advantage of the distributed nature
of git, especially the topic branch workflow it supports well.
The key realization is this:
It is necessary to remain on top of the upstream" does not mean "all
of the new development done by my group must build _on top of_ the
upstream" at all.
for one obvious reason: near the tip of the upstream is not necessarily
what you want in your final product. If the "blessed" point will be r2,
you won't want any topic to be based on r3 or later before forking your
release branch that contains the upstream up to r2.
You _would_ want to remain as compatible as possible with upstream later
than r2 even if the "blessed" point _for this cycle_ is r2, so you would
need a daily integration testing branch that is rebuilt by merging all the
topics from your group _and_ upstream tip and run your tests there. But
once you know which commit from the upstream side (r2 in your example)
will be the one your group will base your changes on and go into a pre-
release mode, you will likely to want to have _another_ test integration
branch to merge all the topics from your group that have forked from a
commit before r2, on top of r2. There _will_ be topics from some members
who unfortunately based their topic on commits newer than r2 and they need
to adjust their work on r2 or older. If they didn't depend on new API,
new features, or modified semantics r3 or later introduced, this rebase
should be trivial. On the other hand, if they did depend on new things,
the project management needs to make a decision. You may choose to
keep the topic out of your upcoming release (because it will depend on
features of upstream newer than the "blassed" upstream release your
product will be based on), or you may choose to "backport" the upstream
features the topic needs without slurping all the new and unstable
upstream features (e.g. if a topic based its work on a new feature
implemented in r5, but what r3 or r4 do are unnecessary for it, or any
other topics in your product, you will cherry-pick r5 on top of r2 and
then rebase the topic on top of the result).
It also means that your group members try not to base their work on the
daily integration branch which is your 'master' branch current is, nor
rebase their work on it before merging. Instead, fork from a known good
state where the result of building on it and then merging that with the
tip of the upstream and with other topics won't introduce excessive
conflicts.
The project manager may want to establish and maintain a reasonable
baseline that:
- merges from upstream trunk (but not its tip) only occasionally; Being
behind is perfectly fine and even preferable than being too hasty and
merging r3 and later into this branch.
- merges from group members' topics _only_ after they are _fully_
completed _and_ you are sure that you want these topics in your next
release.
and have group members fork from its tip. I'd recommend using 'master'
for that. Don't let random topics that are half-done (i.e. "this doesn't
do what it should do, but it hopefully doesn't break other things") merged
into this branch.
Then every test cycle (once a day, or continuous, it doesn't matter)
creates a throw-away test branch to the tip of this branch, merges all the
topics that are still cooking and testable, and merges from the updated
upstream. This may need conflict resolution, and rerere will help here.
And test the result. That way you will ensure that you will be "on top of
the bleeding edge".
When conflicts with the updated upstream become getting excessive, it is a
good sign that you would want to update the "baseline" (see above) with
(an early part of) the upstream.
If conflicts between topics of your group members become excessive, it is
a sign that you are not partitioning the work correctly. SCM is never a
substitute for developer communication and you may need to have the
members involved in the conflict talk with each other and resolve the
conflict between themselves. E.g. they may want to decide to merge their
work so far and build their topics upon a consolidated base.
And that "reasonable baseline", as long as it won't overstep 'r2' in the
end, will be directly the branch you cut your release from. On the day
you decide to base your next product on r2, you will fork 'release' branch
from the tip of 'master', and merge up to 'r2' from the upstream, and you
have your release with fully cooked topics. If on the other hand you have
already merged from the upstream up to 'r5' to 'master', then you would
need to build a separate 'release' branch, forking from some commits in
'master' before the branch merged commits later than 'r2' from the
upstream. Betweeen this fork-point and the tip of your 'master', there
will be some topics that have graduated to 'master'. You need to examine
them and see if they merge to 'release' without dragging 'r3' and later in
from the upstream, and make the management decision I mentioned 60 lines
ago.
^ permalink raw reply
* Re: [PATCH] help.autocorrect: do not run a command if the command given is junk
From: Junio C Hamano @ 2009-12-14 21:47 UTC (permalink / raw)
To: Johannes Schindelin; +Cc: Johannes Sixt, Git Mailing List, Alex Riesen
In-Reply-To: <alpine.DEB.1.00.0912142146590.4985@pacific.mpi-cbg.de>
Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:
> Satisfied?
Very much.
FWIW almost the same procedure led to the weights 0, 2, 1 and 4 that you
see in help.c. The weights are basically factors with which mistakes are
punished: if you just confuse two adjacent letters, such as "psuh" instead
of "push" (which can be quite common if you use two hands, one on the left
side, and one on the right side of the keyboard, with an en-US layout so
many of us use, myself included) it costs 0.
If you write a different character than what you intended, the cost is 2.
The idea behind it is that you're more likely to miss a key than to hit
the wrong key. With the laptop I am typing this email on, it is
particularly likely that I miss a key, because there are certain
key combinations where only the first key triggers an input event, but the
second only triggers an input event when it is _released_ after the first
one. So when I type "er" real fast and happen to release the "e" key
after the "r" key, no "r" appears on my screen.
Okay, so the weight for adding a character must be smaller than
substituting a character, but why is the cost for deletion so high?
Well, I really rarely type unnecessary characters (except when writing to
the Git mailing list, arguably) so those costs must be substantially
higher than for typing the wrong character.
These are actually very good justifications in the sense that people who
might want to tweak the heuristics can see the reason behind the current
choice and agree or disagree with it.
I somehow suspect that a good mathematician can come up with a rationale
for 6 after the fact that sounds convincing, along the lines of "the
average length of commands being N, and levenshtein penalties being
<0,2,1,4>, you can insert X mistaken keystroke and/or omit Y mistaken
keystroke per every correct keystroke without exceeding this value 6, and
the percentage X and/or Y represents is not too low to be practical but low
enough to reject false positives".
In any case, I'll further squash in the following. Thanks for an amusing
explanation ;-).
diff --git a/help.c b/help.c
index fbf80d9..de1e2ea 100644
--- a/help.c
+++ b/help.c
@@ -297,7 +297,7 @@ static void add_cmd_list(struct cmdnames *cmds, struct cmdnames *old)
old->names = NULL;
}
-/* how did we decide this is a good cutoff??? */
+/* An empirically derived magic number */
#define SIMILAR_ENOUGH(x) ((x) < 6)
const char *help_unknown_cmd(const char *cmd)
^ permalink raw reply related
* Re: git-reflog 70 minutes at 100% cpu and counting
From: Eric Paris @ 2009-12-14 21:56 UTC (permalink / raw)
To: Jeff King; +Cc: git
In-Reply-To: <20091214212343.GA11131@coredump.intra.peff.net>
On Mon, 2009-12-14 at 16:23 -0500, Jeff King wrote:
> On Mon, Dec 14, 2009 at 04:20:29PM -0500, Eric Paris wrote:
>
> > Updated to git-1.6.5.3-1 from Fedora rawhide and still git reflog ran
> > for >5 minutes at 100% cpu (I killed it, it didn't finish)
> >
> > I'm pushing a copy of the whole repo (all 1.9G after bzip compression)
> > to
> >
> > http://people.redhat.com/~eparis/git-tar/
>
> Wowzers, that's big. Can you send just what's in .git?
So I zipped up just .git 1.2G. I did a make clean and zipped up the
whole repo 1.3G.
Just started pushing the 1.3G file.
Maybe having a .git directory that large is the problem?
^ permalink raw reply
* Re: [PATCH] help.autocorrect: do not run a command if the command given is junk
From: Johannes Schindelin @ 2009-12-14 22:04 UTC (permalink / raw)
To: Junio C Hamano; +Cc: Johannes Sixt, Git Mailing List, Alex Riesen
In-Reply-To: <7vocm1grdb.fsf@alter.siamese.dyndns.org>
Hi,
On Mon, 14 Dec 2009, Junio C Hamano wrote:
> I somehow suspect that a good mathematician can come up with a rationale
> for 6 after the fact that sounds convincing, along the lines of "the
> average length of commands being N, and levenshtein penalties being
> <0,2,1,4>, you can insert X mistaken keystroke and/or omit Y mistaken
> keystroke per every correct keystroke without exceeding this value 6,
> and the percentage X and/or Y represents is not too low to be practical
> but low enough to reject false positives".
Being a mathematician, I was tempted to invent such a reasoning in
hindsight.
But I decided to be truthful instead,
Dscho
^ permalink raw reply
* Re: git-reflog 70 minutes at 100% cpu and counting
From: Sverre Rabbelier @ 2009-12-14 22:03 UTC (permalink / raw)
To: Eric Paris; +Cc: Jeff King, git
In-Reply-To: <1260827790.9379.59.camel@localhost>
Heya,
On Mon, Dec 14, 2009 at 22:56, Eric Paris <eparis@redhat.com> wrote:
> Just started pushing the 1.3G file.
>
> Maybe having a .git directory that large is the problem?
What did you say this repository contained again? Your home video's?
Ah, well that explains ;).
--
Cheers,
Sverre Rabbelier
^ permalink raw reply
* Re: git-reflog 70 minutes at 100% cpu and counting
From: Jeff King @ 2009-12-14 22:14 UTC (permalink / raw)
To: Eric Paris; +Cc: git
In-Reply-To: <1260827790.9379.59.camel@localhost>
On Mon, Dec 14, 2009 at 04:56:30PM -0500, Eric Paris wrote:
> So I zipped up just .git 1.2G. I did a make clean and zipped up the
> whole repo 1.3G.
>
> Just started pushing the 1.3G file.
>
> Maybe having a .git directory that large is the problem?
It could be, but I doubt it. If you have a lot of loose objects that
could make things slow due to the disk access, but it is not likely to
use that much CPU time (we do have to zlib uncompress more, but
still...70 minutes is a lot).
-Peff
^ permalink raw reply
* Re: [PATCH] help.autocorrect: do not run a command if the command given is junk
From: Johannes Sixt @ 2009-12-14 21:55 UTC (permalink / raw)
To: Junio C Hamano; +Cc: Johannes Schindelin, Git Mailing List, Alex Riesen
In-Reply-To: <7v7hspjp3q.fsf@alter.siamese.dyndns.org>
On Montag, 14. Dezember 2009, Junio C Hamano wrote:
> In the meantime, I think squashing the following in would help us keep the
> two magic numbers in sync.
I do not think that keeping the numbers in sync is necessary. For example, the
similarity requirement for commands that run automatically could be stricter
than for the list of suggestions. Then it would be possible that a unique
best candidate is not good enough to be run automatically; there would only
be a list of suggestions.
-- Hannes
^ permalink raw reply
* Giving command line parameter to textconv command?
From: Nanako Shiraishi @ 2009-12-14 22:17 UTC (permalink / raw)
To: git
Some text documents in my repository is encoded in EUC-JP
and I have this line in my .gitattributes file.
*.txt diff=eucjp
and these two lines in my .git/config file.
[diff "eucjp"]
textconv = nkf-w
And I have ~/bin/nkf-w script that is executable.
#!/bin/sh
nkf -w "$@"
The command takes a (Japanese) text file and converts it
into UTF-8 (it guesses the input encoding).
I need this extra script because setting 'nkf -w' for
textconv like this
[diff "eucjp"]
textconv = nkf -w
gives an error.
% diff --git a/hello.txt b/hello.txt
index 696acd7..f07aa1a 100644
error: cannot run nkf -w: No such file or directory
error: error running textconv command 'nkf -w'
fatal: unable to read files to diff
Could you fix textconv so that it can be given parameters?
--
Nanako Shiraishi
http://ivory.ap.teacup.com/nanako3/
^ permalink raw reply
* Re: am fails to apply patches for files with CRLF lineendings
From: Brandon Casey @ 2009-12-14 22:56 UTC (permalink / raw)
To: Junio C Hamano; +Cc: Björn Steinbrink, jk, git, Brandon Casey
In-Reply-To: <7vvdg9i9mn.fsf@alter.siamese.dyndns.org>
Junio C Hamano wrote:
> Björn Steinbrink <B.Steinbrink@gmx.de> writes:
>
>> Commit c2ca1d7 "Allow mailsplit ... to handle mails with CRLF line-endings"
>> seems to be responsible.
>
> Yes, that commit is not only responsible but was deliberate. For a better
> backstory, see:
>
> http://thread.gmane.org/gmane.comp.version-control.git/124718/focus=124721
>
> You'd notice that I was one of the people who didn't want to have this
> change, so you don't need to convince _me_ that this was not a change to
> keep everybody happy, but you'd need to try a better job than I did back
> then to convince people who thought that "am" should directly work on
> "Thunderbird saved mails" that what they want was a bad idea X-<.
My understanding of the problem is that rfc2822 dictates that CRLF is the
line ending in an email message for _every_ line, and that CR cannot
occur without LF and vice versa. So there is no reliable way to extract
patches from the body of an email and expect line endings to be conveyed
accurately. Some email clients save emails with the line-endings of the
platform, some save in what they call "raw" format with rfc2822's CRLF
line endings. So we have to _assume_ that the patch extracted from the
email has a particular line ending and make-it-so. For better or worse
(better for me), commit c2ca1d7 chose LF line-endings as the line-ending
of choice.
I agree that git-am should be able to apply everything that
git-format-patch produces. Perhaps the diff machinery should be modified
to treat files containing \r as binary when generating the output for
format-patch. Then we'd get a binary diff in the email.
-brandon
^ permalink raw reply
* Re: [PATCH 03/23] Introduce "skip-worktree" bit in index, teach Git to get/set this bit
From: Greg Price @ 2009-12-14 23:06 UTC (permalink / raw)
To: Nguyễn Thái Ngọc Duy; +Cc: git
In-Reply-To: <1260786666-8405-4-git-send-email-pclouds@gmail.com>
Hi Duy,
> +Skip-worktree bit
> +-----------------
> +
> +Skip-worktree bit can be defined in one (long) sentence: When reading
> +an entry, if it is marked as skip-worktree, then Git pretends its
> +working directory version is up to date and read the index version
> +instead.
> +
> +To elaborate, "reading" means checking for file existence, reading
> +file attributes or file content. The working directory version may be
> +present or absent. If present, its content may match against the index
> +version or not. Writing is not affected by this bit, content safety
> +is still first priority. Note that Git _can_ update working directory
> +file, that is marked skip-worktree, if it is safe to do so (i.e.
> +working directory version matches index version)
> +
> +Although this bit looks similar to assume-unchanged bit, its goal is
> +different from assume-unchanged bit's. Skip-worktree also takes
> +precedence over assume-unchanged bit when both are set.
I confess I can't tell how the skip-worktree bit does differ from
assume-unchanged. Is its 'goal' different only in that you have a
different motivation for introducing it, or does it actually have a
different effect -- and what is that different effect?
Looking forward to seeing sparse checkouts soon!
Cheers,
Greg
^ permalink raw reply
* Re: am fails to apply patches for files with CRLF lineendings
From: Junio C Hamano @ 2009-12-14 23:22 UTC (permalink / raw)
To: Brandon Casey; +Cc: Björn Steinbrink, jk, git, Brandon Casey
In-Reply-To: <tCQlJn153g8Oa6Z9HKe6xOUQJdcf2PCIVthlTrLgYE-wJ5jFyXVXWw@cipher.nrlssc.navy.mil>
Brandon Casey <brandon.casey.ctr@nrlssc.navy.mil> writes:
> My understanding of the problem is that rfc2822 dictates that...
I think the fundamental problem is that what MUA uses as the internal
storage format doesn't necessarily have to even be RFC-2822, which only
specifies what should be on-the-wire. The blamed commit took things too
far.
It actually is the norm to use LF as the line terminator in the body text
in saved messages (and trailing CR as a true part of the payload), and
"am" traditionally used that definition. It is meant to read from "mbox"
format to begin with.
Before the blamed commit, "am" took what was given literally, and it
treated the trailing CR as part of the payload in a text file, each of
whose line is LF terminated. This meant that if you sent and your MUA
didn't corrupt, or more importantly if you ran format-patch yourself to
produce a patch on content with CRLF line endings and fed it to am without
any e-mail involved, your CRLF would have been preserved. So in that
sense, unlike what you said in your message, the blamed commit didn't
decide that the line termination must be LF. It decided that the line
termination does not matter, which is a lot worse.
As long as the use of CR is an internal storage matter and "Save As..."
doesn't add extra CR that wasn't in the original contents, I wouldn't say
that such a MUA is broken. In the use case that led to the blamed commit,
the user is choosing to read directly from the internal storage of MUA,
bypassing its "Save As..." interface meant to be used to externalize the
messages, and the user is responsible for dealing with the fallout, hence
my "dos2unix" suggestion in the original thread.
Probably we should revert that commit, unless somebody comes up with a
better solution _or_ somebody convincingly argues that there shouldn't be
CRLF in your committed history.
^ permalink raw reply
* Re: Giving command line parameter to textconv command?
From: Junio C Hamano @ 2009-12-14 23:31 UTC (permalink / raw)
To: Nanako Shiraishi; +Cc: git, Jeff King
In-Reply-To: <20091215071735.6117@nanako3.lavabit.com>
Nanako Shiraishi <nanako3@lavabit.com> writes:
> I need this extra script because setting 'nkf -w' for
> textconv like this
>
> [diff "eucjp"]
> textconv = nkf -w
>
> gives an error.
>
> % diff --git a/hello.txt b/hello.txt
> index 696acd7..f07aa1a 100644
> error: cannot run nkf -w: No such file or directory
> error: error running textconv command 'nkf -w'
> fatal: unable to read files to diff
>
> Could you fix textconv so that it can be given parameters?
The change to do so looks like this; it has a few side effects:
- If somebody else were relying on the fact that 'nkf -w' names the
entire command, it now will run 'nkf' command with '-w' as an argument,
and it will break such a set-up. IOW, command that has an IFS white
space in its path will now need to be quoted from the shell.
You can see the fallout from this in the damage made to t/ hierarchy in
the attached patch.
- You can now use $HOME and other environment variables your shell
expands when defining your textconv command.
Overall I think it is a good direction to go, but we need to be careful
about how we transition the existing repositories that use the old
semantics.
We might need to introduce diff.*.xtextconv or something.
diff.c | 9 ++++-----
t/t4030-diff-textconv.sh | 2 +-
t/t4031-diff-rewrite-binary.sh | 2 +-
3 files changed, 6 insertions(+), 7 deletions(-)
diff --git a/diff.c b/diff.c
index 08bbd3e..64a1486 100644
--- a/diff.c
+++ b/diff.c
@@ -3760,15 +3760,14 @@ static char *run_textconv(const char *pgm, struct diff_filespec *spec,
size_t *outsize)
{
struct diff_tempfile *temp;
- const char *argv[3];
- const char **arg = argv;
+ const char *argv[4] = { "sh", "-c", NULL, NULL };
struct child_process child;
struct strbuf buf = STRBUF_INIT;
+ struct strbuf cmd = STRBUF_INIT;
temp = prepare_temp_file(spec->path, spec);
- *arg++ = pgm;
- *arg++ = temp->name;
- *arg = NULL;
+ strbuf_addf(&cmd, "%s %s", pgm, temp->name);
+ argv[2] = strbuf_detach(&cmd, NULL);
memset(&child, 0, sizeof(child));
child.argv = argv;
diff --git a/t/t4030-diff-textconv.sh b/t/t4030-diff-textconv.sh
index a3f0897..3468f77 100755
--- a/t/t4030-diff-textconv.sh
+++ b/t/t4030-diff-textconv.sh
@@ -48,7 +48,7 @@ test_expect_success 'file is considered binary by plumbing' '
test_expect_success 'setup textconv filters' '
echo file diff=foo >.gitattributes &&
- git config diff.foo.textconv "$PWD"/hexdump &&
+ git config diff.foo.textconv \""$PWD"/hexdump\" &&
git config diff.fail.textconv false
'
diff --git a/t/t4031-diff-rewrite-binary.sh b/t/t4031-diff-rewrite-binary.sh
index a894c60..e6cb30e 100755
--- a/t/t4031-diff-rewrite-binary.sh
+++ b/t/t4031-diff-rewrite-binary.sh
@@ -54,7 +54,7 @@ chmod +x dump
test_expect_success 'setup textconv' '
echo file diff=foo >.gitattributes &&
- git config diff.foo.textconv "$PWD"/dump
+ git config diff.foo.textconv \""$PWD"/dump\"
'
test_expect_success 'rewrite diff respects textconv' '
^ permalink raw reply related
* Re: [PATCH] help.autocorrect: do not run a command if the command given is junk
From: Junio C Hamano @ 2009-12-14 23:39 UTC (permalink / raw)
To: Johannes Sixt
Cc: Junio C Hamano, Johannes Schindelin, Git Mailing List,
Alex Riesen
In-Reply-To: <200912142255.36949.j.sixt@viscovery.net>
Johannes Sixt <j.sixt@viscovery.net> writes:
> On Montag, 14. Dezember 2009, Junio C Hamano wrote:
>> In the meantime, I think squashing the following in would help us keep the
>> two magic numbers in sync.
>
> I do not think that keeping the numbers in sync is necessary. For example, the
> similarity requirement for commands that run automatically could be stricter
> than for the list of suggestions. Then it would be possible that a unique
> best candidate is not good enough to be run automatically; there would only
> be a list of suggestions.
Well thought out. Would you want to reroll a patch with two symbolic
constants then?
^ permalink raw reply
* Re: git-reflog 70 minutes at 100% cpu and counting
From: Nicolas Pitre @ 2009-12-15 0:26 UTC (permalink / raw)
To: Eric Paris; +Cc: Jeff King, git
In-Reply-To: <1260827790.9379.59.camel@localhost>
On Mon, 14 Dec 2009, Eric Paris wrote:
> On Mon, 2009-12-14 at 16:23 -0500, Jeff King wrote:
> > On Mon, Dec 14, 2009 at 04:20:29PM -0500, Eric Paris wrote:
> >
> > > Updated to git-1.6.5.3-1 from Fedora rawhide and still git reflog ran
> > > for >5 minutes at 100% cpu (I killed it, it didn't finish)
> > >
> > > I'm pushing a copy of the whole repo (all 1.9G after bzip compression)
> > > to
> > >
> > > http://people.redhat.com/~eparis/git-tar/
> >
> > Wowzers, that's big. Can you send just what's in .git?
>
> So I zipped up just .git 1.2G. I did a make clean and zipped up the
> whole repo 1.3G.
>
> Just started pushing the 1.3G file.
>
> Maybe having a .git directory that large is the problem?
Shouldn't be, unless your repo is really badly packed.
What's the output of 'git count-objects -v' ?
Nicolas
^ permalink raw reply
* Re: git-reflog 70 minutes at 100% cpu and counting
From: Nicolas Pitre @ 2009-12-15 0:29 UTC (permalink / raw)
To: Sverre Rabbelier; +Cc: Eric Paris, Jeff King, git
In-Reply-To: <fabb9a1e0912141403hb728974sc50b9e8dbb08925d@mail.gmail.com>
On Mon, 14 Dec 2009, Sverre Rabbelier wrote:
> Heya,
>
> On Mon, Dec 14, 2009 at 22:56, Eric Paris <eparis@redhat.com> wrote:
> > Just started pushing the 1.3G file.
> >
> > Maybe having a .git directory that large is the problem?
>
> What did you say this repository contained again? Your home video's?
> Ah, well that explains ;).
That would explain the size, but not the reflog CPU time.
Nicolas
^ permalink raw reply
* Re: git-reflog 70 minutes at 100% cpu and counting
From: Junio C Hamano @ 2009-12-15 0:36 UTC (permalink / raw)
To: Nicolas Pitre; +Cc: Eric Paris, Jeff King, git
In-Reply-To: <alpine.LFD.2.00.0912141924030.23173@xanadu.home>
Nicolas Pitre <nico@fluxnic.net> writes:
> On Mon, 14 Dec 2009, Eric Paris wrote:
>
>> On Mon, 2009-12-14 at 16:23 -0500, Jeff King wrote:
>> > On Mon, Dec 14, 2009 at 04:20:29PM -0500, Eric Paris wrote:
>> >
>> > > Updated to git-1.6.5.3-1 from Fedora rawhide and still git reflog ran
>> > > for >5 minutes at 100% cpu (I killed it, it didn't finish)
>> > >
>> > > I'm pushing a copy of the whole repo (all 1.9G after bzip compression)
>> > > to
>> > >
>> > > http://people.redhat.com/~eparis/git-tar/
>> >
>> > Wowzers, that's big. Can you send just what's in .git?
>>
>> So I zipped up just .git 1.2G. I did a make clean and zipped up the
>> whole repo 1.3G.
>>
>> Just started pushing the 1.3G file.
>>
>> Maybe having a .git directory that large is the problem?
>
> Shouldn't be, unless your repo is really badly packed.
>
> What's the output of 'git count-objects -v' ?
Didn't somebody say that the trace hints an infinite loop not "slow
because of bad packing"?
^ permalink raw reply
* Re: am fails to apply patches for files with CRLF lineendings
From: Björn Steinbrink @ 2009-12-15 1:25 UTC (permalink / raw)
To: Brandon Casey; +Cc: Junio C Hamano, Brandon Casey, jk, git
In-Reply-To: <ee63ef30912141650ie05baf4kab8505adf160c62e@mail.gmail.com>
On 2009.12.14 18:50:44 -0600, Brandon Casey wrote:
> I think it is more correct to say that the line termination in an email is
> ambiguous. CRLF does not necessarily mean that the original had CRLF line
> termination if RFC-2822 is followed explicitly.
Right. And checking, after sending a patch containing CRs with mutt, it
lost those CRs. Even the local copy saved directly by mutt, which didn't
leave my box, lacks the CRs. So it seems basically impossible to send
patches to CRLF files inline.
RFC-822 still allowed bare CRs/LFs :-/
So the commit didn't break with anything mails conforming to RFC-2822,
those won't work for files with CR being patch. But it still breaks the
the raw format-patch generated patches, so even attaching them to the
actual email as a workaround won't do.
That makes a "use the first line to decide whether or not to strip CRs"
approach look like a good idea. Real mails are broken anyway, and the
format-patch output has LF on the first line, so mailsplit wouldn't mess
it up... Unless git on windows produces CRLF format-patch output...
Björn
^ permalink raw reply
* Fwd: am fails to apply patches for files with CRLF lineendings
From: Brandon Casey @ 2009-12-15 2:09 UTC (permalink / raw)
To: git
In-Reply-To: <ee63ef30912141650ie05baf4kab8505adf160c62e@mail.gmail.com>
Forwarding to the list. The original was bounced since gmail sent a
multipart mime version with html. Seems we can't disable html
composing in the gmail settings anymore (I thought we used to be able
to).
---------- Forwarded message ----------
From: Brandon Casey <drafnel@gmail.com>
Date: Mon, Dec 14, 2009 at 6:50 PM
Subject: Re: am fails to apply patches for files with CRLF lineendings
To: Junio C Hamano <gitster@pobox.com>
Cc: Brandon Casey <brandon.casey.ctr@nrlssc.navy.mil>, Björn
Steinbrink <B.Steinbrink@gmx.de>, jk@silentcow.com,
git@vger.kernel.org
On Mon, Dec 14, 2009 at 5:22 PM, Junio C Hamano <gitster@pobox.com> wrote:
>
> Brandon Casey <brandon.casey.ctr@nrlssc.navy.mil> writes:
>
> > My understanding of the problem is that rfc2822 dictates that...
>
> I think the fundamental problem is that what MUA uses as the internal
> storage format doesn't necessarily have to even be RFC-2822, which only
> specifies what should be on-the-wire.
If CRLF is what is on-the-wire, how can the MUA tell whether the
original was also CRLF or whether it was only LF? My assumption was
that the MUA cannot tell, and that things worked for most people
because those people who wanted LF terminated output were on platforms
that used LF termination and their MUA produced output using the
native line termination. Things broke recently for some people since
thunderbird devels decided to start saving emails with CRLF
termination on linux.
>
> The blamed commit took things too
> far.
>
> It actually is the norm to use LF as the line terminator in the body text
> in saved messages (and trailing CR as a true part of the payload), and
> "am" traditionally used that definition. It is meant to read from "mbox"
> format to begin with.
But isn't each email in the mbox file supposed to be RFC-2822
formatted anyway? If so, then my reading of RFC-2822 says that there
should only be CRLF everywhere and no bare CR or bare LF. But maybe
everyone has just been ignoring that part of RFC-2822? I'm not an
email expert, so I really don't know.
>
> Before the blamed commit, "am" took what was given literally, and it
> treated the trailing CR as part of the payload in a text file, each of
> whose line is LF terminated. This meant that if you sent and your MUA
> didn't corrupt, or more importantly if you ran format-patch yourself to
> produce a patch on content with CRLF line endings and fed it to am without
> any e-mail involved, your CRLF would have been preserved. So in that
> sense, unlike what you said in your message, the blamed commit didn't
> decide that the line termination must be LF. It decided that the line
> termination does not matter, which is a lot worse.
I think it is more correct to say that the line termination in an
email is ambiguous. CRLF does not necessarily mean that the original
had CRLF line termination if RFC-2822 is followed explicitly.
> As long as the use of CR is an internal storage matter and "Save As..."
> doesn't add extra CR that wasn't in the original contents, I wouldn't say
> that such a MUA is broken. In the use case that led to the blamed commit,
> the user is choosing to read directly from the internal storage of MUA,
> bypassing its "Save As..." interface meant to be used to externalize the
> messages,
No, we're using "Save As..." in thunderbird, and it saves with CRLF
line endings. I don't really care for thunderbird and its proclivity
for munging my emails and _not_ doing-the-right-thing in my opinion.
Maybe someone can suggest a mail client that can use imap and provide
the nice sorting of emails into folders like thunderbird does.
-brandon
^ permalink raw reply
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox