* Re: non-US-ASCII file names (e.g. Hiragana) on Windows
From: Johannes Sixt @ 2009-12-01 8:27 UTC (permalink / raw)
To: Thomas Singer; +Cc: git
In-Reply-To: <4B14CA79.6040408@syntevo.com>
Thomas Singer schrieb:
> Thomas Singer wrote:
>> Reece Dunn wrote:
>>> This is a bug in git's character encoding/conversion logic. It looks
>>> like git is taking the source string and converting it to ascii to be
>>> displayed on the console output (e.g. by using the WideCharToMultiByte
>>> conversion API) -- these APIs will use a '?' character for characters
>>> that it cannot map to the target character encoding (like the Hiragana
>>> characters that you are using).
>> I have a screenshot from a SmartGit user where 1) the console can show the
>> far-east-characters and 2) Git *can* show the characters escaped. Are there
>> two versions of Git available or does Gits behaviour depends somehow on the
>> system locale?
>
> Does no Git expert know what to do to get it working?
http://article.gmane.org/gmane.comp.version-control.git/133980 [*]
The possible reason why some one else is seeing correct glyphs with
SmartGit is because it is a Unicode application and the Windows box has
suitable fonts installed and the console is configured with a suitable
font as well.
-- Hannes
[*] I had a botch email infrastructure when I sent this message, and the
copy intended for you went to the waste bin, but I thought I had re-sent
to you in a private mail.
^ permalink raw reply
* Re: [PATCH] get_ref_states: strdup entries and free util in stale list
From: Johannes Schindelin @ 2009-12-01 8:35 UTC (permalink / raw)
To: Bert Wesarg; +Cc: Junio C Hamano, Jay Soffian, git
In-Reply-To: <0458f16c6ce906997aaf357c0c7368841ae83c36.1259625072.git.bert.wesarg@googlemail.com>
Hi,
On Tue, 1 Dec 2009, Bert Wesarg wrote:
> Signed-off-by: Bert Wesarg <bert.wesarg@googlemail.com>
Thanks. I trust you ran the test suite with valgrind just to make sure?
Ciao,
Dscho
^ permalink raw reply
* Re: Umlaut in filename makes troubles
From: Jochen @ 2009-12-01 8:26 UTC (permalink / raw)
To: git
In-Reply-To: <20091201074420.GC3618@triton>
Øyvind A. Holm wrote:
> On 2009-12-01 08:15:08, rick23@gmx.net wrote:
> > I have problems with my repository under slackware vs. windows. I
> > created a repo in linux and every time I use it under msysgit,
> > the files containing umlauts in the filename are marked as
> > deleted (and vice versa).
> >
> > For instance: the repo perfectly synced under msysgit leads to:
> >
> > user@sauron:/media/disk-2$ git status |grep Auszug
> > # deleted: "trunk/007_Literatur/Auszug aus Ergonomische
> > Untersuchung des Lenkgef\374hles.docx"
> > # "trunk/007_Literatur/Auszug aus Ergonomische Untersuchung
> > des Lenkgef\303\274hles.docx"
> >
> > in linux. But the file exists and is displayed correctly in the
> > shell or in dolphin (my filemanager under X):
> >
> > user@sauron:/media/disk-2$ ls trunk/007_Literatur/Auszug*
> > trunk/007_Literatur/Auszug aus Ergonomische Untersuchung des
> > Lenkgefühles.docx
> >
> > Can you please give me a hint what to do?
>
> Try to specify "utf8" as mount option under Linux.
The automount of KDE 4.2.4 already used utf-8 (I guess the filenames
would be garbled in dolphin otherwise)
> You can also try
> experimenting with the "nls" mount option, check out the mount(8)
> man page to see how it's used.
Ufff, I'm sorry - I'm not sure how to pass this to the automouter. I'd
tried to mount the stick manually (with and without utf) and without
utf8 the filenames are display strange from "ls".
> Additionally, I found that I need "shortname=mixed" when mounting
> USB memory cards. As filenames are case insensitive in Windowsworld
> and gadgets using vfat, Linux tend to treat them differently.
My automouter done it as
/dev/sde on /media/disk-2 type vfat
(rw,nosuid,nodev,uhelper=hal,uid=1000,utf8,shortname=mixed)
So the options look right to me.
Kindest regards
Jochen
^ permalink raw reply
* Re: non-US-ASCII file names (e.g. Hiragana) on Windows
From: Thomas Singer @ 2009-12-01 8:55 UTC (permalink / raw)
To: Johannes Sixt; +Cc: git
In-Reply-To: <4B14D381.3010706@viscovery.net>
Johannes Sixt wrote:
> Thomas Singer schrieb:
>> Thomas Singer wrote:
>>> Reece Dunn wrote:
>>>> This is a bug in git's character encoding/conversion logic. It looks
>>>> like git is taking the source string and converting it to ascii to be
>>>> displayed on the console output (e.g. by using the WideCharToMultiByte
>>>> conversion API) -- these APIs will use a '?' character for characters
>>>> that it cannot map to the target character encoding (like the Hiragana
>>>> characters that you are using).
>>> I have a screenshot from a SmartGit user where 1) the console can show the
>>> far-east-characters and 2) Git *can* show the characters escaped. Are there
>>> two versions of Git available or does Gits behaviour depends somehow on the
>>> system locale?
>> Does no Git expert know what to do to get it working?
>
> http://article.gmane.org/gmane.comp.version-control.git/133980 [*]
>
> The possible reason why some one else is seeing correct glyphs with
> SmartGit is because it is a Unicode application and the Windows box has
> suitable fonts installed and the console is configured with a suitable
> font as well.
I wasn't talking about SmartGit, but msysgit on the Windows console. Sorry,
if that wasn't clear.
Is it a German Windows limitation, that far-east characters are not
supported on it (but work fine on a Japanese Windows), are there different
(mysys)Git versions available or is this a configuration issue?
--
Tom
^ permalink raw reply
* Re: non-US-ASCII file names (e.g. Hiragana) on Windows
From: Thomas Singer @ 2009-12-01 8:57 UTC (permalink / raw)
To: Johannes Sixt; +Cc: git
In-Reply-To: <200911282100.23000.j6t@kdbg.org>
Johannes Sixt wrote:
> On Samstag, 28. November 2009, Thomas Singer wrote:
>> I've created a file with unicode characters in its name (using Java):
>>
>> new File(dir, "\u3041\u3042\u3043\u3044").createNewFile();
>> ...
>> $ git add .
>> fatal: unable to stat '????': No such file or directory
>>
>> What should I do to make Git recognize these characters?
>
> You cannot on a German Windows.
>
> You can switch your Windows to Japanese (not the UI, just the codepage
> aka "locale"; yes, that's possible, I have such a setup), but even then the
> characters of the file name will be recorded in Shift-JIS encoding, not UTF-8
> or Unicode. When you later switch back to German, these bytes will be
> interpreted as cp850 or cp1252 text and displayed accordingly.
Who is interpreting the file names? Windows or Git or Java?
--
Tom
^ permalink raw reply
* Re: non-US-ASCII file names (e.g. Hiragana) on Windows
From: Thomas Singer @ 2009-12-01 9:04 UTC (permalink / raw)
To: Johannes Sixt; +Cc: git
In-Reply-To: <4B14DA78.70906@syntevo.com>
Thomas Singer wrote:
> Johannes Sixt wrote:
>> You can switch your Windows to Japanese (not the UI, just the codepage
>> aka "locale"; yes, that's possible, I have such a setup), but even then the
>> characters of the file name will be recorded in Shift-JIS encoding, not UTF-8
>> or Unicode. When you later switch back to German, these bytes will be
>> interpreted as cp850 or cp1252 text and displayed accordingly.
>
> Who is interpreting the file names? Windows or Git or Java?
To be more precise: Who is interpreting the bytes in the file names as
characters? Windows, Git or Java?
--
Tom
^ permalink raw reply
* Re: [PATCH] get_ref_states: strdup entries and free util in stale list
From: Bert Wesarg @ 2009-12-01 9:05 UTC (permalink / raw)
To: Johannes Schindelin; +Cc: Junio C Hamano, Jay Soffian, git
In-Reply-To: <alpine.DEB.1.00.0912010934120.4985@pacific.mpi-cbg.de>
On Tue, Dec 1, 2009 at 09:35, Johannes Schindelin
<Johannes.Schindelin@gmx.de> wrote:
> Hi,
>
> On Tue, 1 Dec 2009, Bert Wesarg wrote:
>
>> Signed-off-by: Bert Wesarg <bert.wesarg@googlemail.com>
>
> Thanks. I trust you ran the test suite with valgrind just to make sure?
Not the test suite. But my use case where I found the problem (Ie.
cut-off branch names) which was 'git remote show <remote>'.
There are still invalid reads of size 4. I think the problem is the
flex array member of 'struct ref' and strlen(). If its worth I can
look into this.
Bert
>
> Ciao,
> Dscho
>
^ permalink raw reply
* Re: Umlaut in filename makes troubles
From: Jochen @ 2009-12-01 9:06 UTC (permalink / raw)
To: git
In-Reply-To: <4B14CBA9.6080104@drmicha.warpmail.net>
Michael J Gruber wrote:
> > I have problems with my repository under slackware vs. windows. I
> > created a repo in linux and every time I use it under msysgit,
> > the files containing umlauts in the filename are marked as
> > deleted (and vice versa).
> >
> > For instance: the repo perfectly synced under msysgit leads to:
> >
> > user@sauron:/media/disk-2$ git status |grep Auszug
> > # deleted: "trunk/007_Literatur/Auszug aus Ergonomische
> > Untersuchung des Lenkgef\374hles.docx"
> > # "trunk/007_Literatur/Auszug aus Ergonomische Untersuchung
> > des Lenkgef\303\274hles.docx"
> >
> > in linux. But the file exists and is displayed correctly in the
> > shell or in dolphin (my filemanager under X):
> >
> > user@sauron:/media/disk-2$ ls trunk/007_Literatur/Auszug*
> > trunk/007_Literatur/Auszug aus Ergonomische Untersuchung des
> > Lenkgefühles.docx
> >
> > Can you please give me a hint what to do?
>
> I would say you can give us more info about your setup. If I
> understand you correctly, you use the same repo and checkout under
> linux and msysgit.
Right - I created it under linux on a local directory, cloned it to an
usb-stick and cloned it again under windows.
I then worked under windows (msysgit) and pushed it to the stick
again.
Now I am back an my linux maschine and the files with umlauts in the
filenames are mared as deleted.
> > What kind of filesystem is this on, what are the
> mount options?
/dev/sde on /media/disk-2 type vfat
(rw,nosuid,nodev,uhelper=hal,uid=1000,utf8,shortname=mixed)
mounted by the automounter of KDE 4.2.4
> What is your locale ($LANG) in slackware?
LANG=de_AT.utf-8
Jochen
^ permalink raw reply
* Re: non-US-ASCII file names (e.g. Hiragana) on Windows
From: Erik Faye-Lund @ 2009-12-01 9:12 UTC (permalink / raw)
To: Thomas Singer; +Cc: Maximilien Noal, git
In-Reply-To: <4B123C80.30607@syntevo.com>
On Sun, Nov 29, 2009 at 10:18 AM, Thomas Singer
<thomas.singer@syntevo.com> wrote:
> Maximilien Noal wrote:
>> About the 'boxes' :
>>
>> The thing is, Windows' files for Asian languages are _not_ installed by
>> default.
>>
>> They can be installed (even while installing Windows), by checking the
>> two checkboxes under the "Supplemtal languages support" groupbox in the
>> "Languages" tab of the "Regional and language options" control panel.
>> *re-take some breath ;-) *
>>
>> It will remove the "boxes" in Explorer and display nice Asian characters.
>
> Thanks, now the characters are showing up fine in the Explorer.
>
> Reece Dunn wrote:
>> This is a bug in git's character encoding/conversion logic. It looks
>> like git is taking the source string and converting it to ascii to be
>> displayed on the console output (e.g. by using the WideCharToMultiByte
>> conversion API) -- these APIs will use a '?' character for characters
>> that it cannot map to the target character encoding (like the Hiragana
>> characters that you are using).
>
> I have a screenshot from a SmartGit user where 1) the console can show the
> far-east-characters and 2) Git *can* show the characters escaped. Are there
> two versions of Git available or does Gits behaviour depends somehow on the
> system locale?
Did you try to make sure your console window used a Unicode font on
your German Windows installation? Asian Windows installations might do
this by default, something at least neither English nor Norwegian
Windows installations seems to do...
You can change the console window font through the properties-menu
that appears when you right click the title-bar.
--
Erik "kusma" Faye-Lund
^ permalink raw reply
* [PATCH] Documentation: Fix a few i.e./e.g. mix-ups
From: Michael J Gruber @ 2009-12-01 9:19 UTC (permalink / raw)
To: git; +Cc: Junio C Hamano
A git bundle can be transported by several means (such as e-mail), not
only by snekaernet, so use e.g. instead of i.e.
The mix-up in git-bundle.txt is obvious.
Signed-off-by: Michael J Gruber <git@drmicha.warpmail.net>
---
Documentation/git-bundle.txt | 2 +-
Documentation/gitcore-tutorial.txt | 2 +-
2 files changed, 2 insertions(+), 2 deletions(-)
diff --git a/Documentation/git-bundle.txt b/Documentation/git-bundle.txt
index aee7e4a..c3a066e 100644
--- a/Documentation/git-bundle.txt
+++ b/Documentation/git-bundle.txt
@@ -24,7 +24,7 @@ ssh, rsync, http) cannot be used. This command provides support for
'git-fetch' and 'git-pull' to operate by packaging objects and references
in an archive at the originating machine, then importing those into
another repository using 'git-fetch' and 'git-pull'
-after moving the archive by some means (i.e., by sneakernet). As no
+after moving the archive by some means (e.g., by sneakernet). As no
direct connection between the repositories exists, the user must specify a
basis for the bundle that is held by the destination repository: the
bundle assumes that all objects in the basis are already in the
diff --git a/Documentation/gitcore-tutorial.txt b/Documentation/gitcore-tutorial.txt
index e237394..f762dca 100644
--- a/Documentation/gitcore-tutorial.txt
+++ b/Documentation/gitcore-tutorial.txt
@@ -602,7 +602,7 @@ $ git tag -s <tagname>
----------------
which will sign the current `HEAD` (but you can also give it another
-argument that specifies the thing to tag, i.e., you could have tagged the
+argument that specifies the thing to tag, e.g., you could have tagged the
current `mybranch` point by using `git tag <tagname> mybranch`).
You normally only do signed tags for major releases or things
--
1.6.6.rc0.274.g71380
^ permalink raw reply related
* Newbie "svn update" question
From: Mikko Oksalahti @ 2009-12-01 9:25 UTC (permalink / raw)
To: git
Hi,
I just started using git for my personal projects at home. Basic usage seems
pretty straight-forward as well as setting up everything. However, I have a
simple question about how do I mimic an "svn update" command on a locally
created repository. Here's what I do:
some_existing_project_dir> git init
some_existing_project_dir> git add .
(about 1000 files added...)
some_existing_project_dir> git commit -a -m "initial commit"
(now I edit 10 files and accidentally delete some files that I'm not aware of)
How do I now get the accidentally deleted files back from the repository without
losing local changes made to 10 files?
I've tried using: "git checkout HEAD ." but my local changes after last commit
will be lost.
I've tried using: "git pull ." but the deleted files are not restored.
So I'm looking for an "svn update" equivalent command that would semantically do
this: "Get the latest version of all files from the repository and merge them
with any local changes I've made to files."
I know a suitable command is available and I'm just a moron who can't read the
manual correctly but help me out anyway :P
Regards,
Mikko
^ permalink raw reply
* Re: [PATCH] get_ref_states: strdup entries and free util in stale list
From: Bert Wesarg @ 2009-12-01 9:32 UTC (permalink / raw)
To: Junio C Hamano; +Cc: Johannes Schindelin, Jay Soffian, git
In-Reply-To: <7vy6ln2llw.fsf@alter.siamese.dyndns.org>
On Tue, Dec 1, 2009 at 08:34, Junio C Hamano <gitster@pobox.com> wrote:
> Bert Wesarg <bert.wesarg@googlemail.com> writes:
>
>> On Tue, Dec 1, 2009 at 01:21, Junio C Hamano <gitster@pobox.com> wrote:
>> ...
>>> Hmm, the Subject: matches what the code does, but nobody mentions why it
>>> is necessary (iow, what breaks in the current code and what becomes better
>>> if the patch is applied). The blank space before your "S-o-b:" line is
>>> for you to describe these things.
>> Sure. unfortunately the code where the string list is filled is not in
>> the patch. But if you look at the code it should be self-explanatory.
>
> That is _exactly_ why I want the description in the commit log message. I
> don't want commit messages (or lack thereof) to force people to look at
> the code outside the patch.
>
> Otherwise I'll have to ask _you_ to personally give the 7-line explanation
> you just gave us to anybody who runs "git log -p" with the default context
> size after this patch is applied. I do not think you have the bandwidth
> to handle that ;-).
Yes. That makes perfectly sense. Sorry for the hassle.
>
>> There is actually also an other solution: we could first strdup the
>> ref name to the .util member and take this as the input for the
>> abbrev_ref()/string list entry and safe there the strdup.
>
> I too thought something like that may make sense, but it doesn't look like
> so, for two reasons:
>
> - string-list API is a bit cumbersome to use if you allocate the string
> yourself. You will be made responsible for freeing them, and often you
> do so by setting strdup_strings to true immediately before calling
> string_list_clear(), which is kind of ugly;
>
> - The ref abbrev_branch() is called and the address of whose substring is
> taken to be used as "name" in handle_one_branch() is refspec.src, but
> what goes to .util is refname that is refspec.dst---they are different
> strings and one is not a substring of the other.
I don't see you point here. The current code is:
for (ref = stale_refs; ref; ref = ref->next) {
struct string_list_item *item =
string_list_append(abbrev_branch(ref->name), &states->stale);
item->util = xstrdup(ref->name);
}
So 0 == strcmp(item->string, abbrev_ref(item->util, "refs/heads/"))
should be true, shouldn't it?
Bert
^ permalink raw reply
* Re: Newbie "svn update" question
From: Howard Miller @ 2009-12-01 9:45 UTC (permalink / raw)
To: Mikko Oksalahti; +Cc: git
In-Reply-To: <loom.20091201T101313-496@post.gmane.org>
2009/12/1 Mikko Oksalahti <mikko@azila.fi>:
> Hi,
>
> I just started using git for my personal projects at home. Basic usage seems
> pretty straight-forward as well as setting up everything. However, I have a
> simple question about how do I mimic an "svn update" command on a locally
> created repository. Here's what I do:
>
> some_existing_project_dir> git init
> some_existing_project_dir> git add .
>
> (about 1000 files added...)
>
> some_existing_project_dir> git commit -a -m "initial commit"
>
> (now I edit 10 files and accidentally delete some files that I'm not aware of)
>
> How do I now get the accidentally deleted files back from the repository without
> losing local changes made to 10 files?
>
> I've tried using: "git checkout HEAD ." but my local changes after last commit
> will be lost.
>
> I've tried using: "git pull ." but the deleted files are not restored.
>
> So I'm looking for an "svn update" equivalent command that would semantically do
> this: "Get the latest version of all files from the repository and merge them
> with any local changes I've made to files."
>
> I know a suitable command is available and I'm just a moron who can't read the
> manual correctly but help me out anyway :P
>
> Regards,
> Mikko
'git status' should show you what files you have deleted. 'git
checkout filename' should get them back. I can't think of a way of
recovering every file you have just deleted although - I suspect it
might be tricky. Thinks like 'git pull' only apply to remote
repositories and you don't have one of those. You're not thinking of
it the right way (yet) :-)
Howard
^ permalink raw reply
* Re: Newbie "svn update" question
From: Tay Ray Chuan @ 2009-12-01 9:49 UTC (permalink / raw)
To: Mikko Oksalahti; +Cc: git
In-Reply-To: <loom.20091201T101313-496@post.gmane.org>
Hi
On Tue, Dec 1, 2009 at 5:25 PM, Mikko Oksalahti <mikko@azila.fi> wrote:
> How do I now get the accidentally deleted files back from the repository without
> losing local changes made to 10 files?
>
> I've tried using: "git checkout HEAD ." but my local changes after last commit
> will be lost.
>
> I've tried using: "git pull ." but the deleted files are not restored.
>
> So I'm looking for an "svn update" equivalent command that would semantically do
> this: "Get the latest version of all files from the repository and merge them
> with any local changes I've made to files."
Are you looking for a command that
"Restore deleted files, without reverting local changes",
or
"Get the latest version of all files from the repository and merge
them with any local changes I've made to files."?
I would suggest adding the changed files, then doing a checkout.
git add changed.file1
git add changed.file2
git add -p #alternatively, select hunks/changes to add interactively
git checkout HEAD #although using . (current directory) should give
the same result
Note that your changes have only been added to the index, and you need
to commit them. The index/stage is a concept not in svn. See the user
manual for more on this.
I also suspect you are still thinking in svn mode when you said
getting the latest version from the repository. You already have a
repository locally. It is more an issue of syncing your local
repository with the one that you're following. Unlike svn, every time
you commit, your local repository is updated, not the remote one, and
vice-versa.
--
Cheers,
Ray Chuan
^ permalink raw reply
* Re: Umlaut in filename makes troubles
From: Jochen @ 2009-12-01 9:58 UTC (permalink / raw)
To: git
In-Reply-To: <200912010815.14515.rick23@gmx.net>
I found another strange effect...
I made a file with "touch aöäü.txt" and from "git status" I get
# "a\303\266\303\244\303\274.txt"
reported as untracked. But when I start "git gui" I get file displayed with
it's correct name...
^ permalink raw reply
* Re: non-US-ASCII file names (e.g. Hiragana) on Windows
From: Johannes Sixt @ 2009-12-01 10:00 UTC (permalink / raw)
To: Thomas Singer; +Cc: git
In-Reply-To: <4B14DA1A.4060505@syntevo.com>
Thomas Singer schrieb:
> Is it a German Windows limitation, that far-east characters are not
> supported on it (but work fine on a Japanese Windows), are there different
> (mysys)Git versions available or is this a configuration issue?
It is a matter of configuration.
Since 8 bits are not sufficient to support Japanese alphabet in addition
to the German alphabet, programs that are not Unicode aware -- such as git
-- have to make a decision which alphabet they support. The decision is
made by picking a "codepage".
On German Windows, you are in codepage 850 (in the console). The filenames
(that actually are in Unicode) are converted to bytes according to
codepage 850 *before* git sees them. If your filenames contain Hiragana,
they are substituted by the "unknown character" marker because there is no
place for them in codepage 850.
However, you can install Japanese language support on German Windows. Then
you can change your console to codepage 932:
chcp 932
When you run git from *this* console, Hiragana in the filenames are
converted to cp932 before git sees them. The resulting byte sequence is
different from the one in cp850, but git will be able to see that the file
exists and was modified, and you can 'git add' it.
But if you have files with umlauts, they will not be recognized anymore
because umlauts have no place in cp932.
In neither case can you exchange the repository with Linux if you have
your locale set to UTF-8 on Linux, because neither byte sequence (umlauts
from cp850 or Hiragana from cp932) are valid UTF-8 sequences, let alone
result in the expected glyphs.
Corollary: Stick to ASCII file names.
There have been suggestions to switch the console to codepage 65001
(UTF-8), but I have never heard of success reports. I'm not saying it does
not work, though.
-- Hannes
^ permalink raw reply
* Re: non-US-ASCII file names (e.g. Hiragana) on Windows
From: Johannes Sixt @ 2009-12-01 10:08 UTC (permalink / raw)
To: Thomas Singer; +Cc: git
In-Reply-To: <4B14DC20.6040808@syntevo.com>
Thomas Singer schrieb:
> To be more precise: Who is interpreting the bytes in the file names as
> characters? Windows, Git or Java?
In the case of git: Windows does it, using the console's codepage to
convert between bytes and Unicode.
I don't know about Java, but I guess that no conversion is necessary
because Java is Unicode-aware.
-- Hannes
^ permalink raw reply
* Re: [PATCH 0/2] http: allow multi-pass authentication
From: Martin Storsjö @ 2009-12-01 10:28 UTC (permalink / raw)
To: Tay Ray Chuan
Cc: git, Nicholas Miell, gsky51, Clemens Buchacher, Mark Lodato,
Johannes Schindelin
In-Reply-To: <20091127234110.7b7e9993.rctay89@gmail.com>
On Fri, 27 Nov 2009, Tay Ray Chuan wrote:
> This patch series applies on top of master. It enables fetching and
> pushing over http with the most suitable authentication scheme chosen
> by curl when the http.authAny or GIT_HTTP_AUTH_ANY is set.
I also tested this, and things generally seem to work fine.
Thanks to the "maintain curl sessions" patch, only the first request needs
to be redone after getting the 401 error containing the authentication
challenge, the later ones work fine on the first try. However,
theoretically, I guess we can't be certain that the curl session really is
initialized for the later requests (we could be given a new fresh curl
session for some reason), or the first request could perhaps be a large,
(currently) non-rewindable POST.
Avoiding redoing large POST requests is generally accomplished by adding a
Expect: 100-continue header, and then waiting for a reply (either 100
continue or 401 unauthorized) to that header before actually sending the
POST body data. If the server doesn't support the Expect header (e.g.
Lighttpd doesn't support it), the client starts sending the POST body
after a timeout (1 second in libcurl).
(As a side note, chunked POST requests without a content-length header
isn't supported by lighttpd at all at the moment, neither in the stable
1.4 version nor in the new upcoming 1.5 branch.)
Normally, libcurl should add the Expect: 100-continue header
automatically, but for some reason
(http://article.gmane.org/gmane.comp.web.curl.library/25992) it doesn't,
so that's probably why we're manually adding that header in
remote-curl.c:371 at the moment. libcurl doesn't detect this at the moment
(http://article.gmane.org/gmane.comp.web.curl.library/25991) so it won't
wait for the 100 continue response before starting to send the body data.
So, with a server supporting Expect, the 401 error response may come after
sending a few KB of POST data (corresponding to the roundtrip delay for
the server to respond to the header) - if the server doesn't support
Expect at all, the whole request will be sent and may need to be rewound.
To clarify - this only happens if the curl authentication isn't
initialized yet, for the first request of every curl session. The
"maintain curl sessions" patch makes sure this isn't needed in the normal
case.
I've experimented with two solutions to this, which add partial and full
rewind solutions to the chunked POST requests - I'll send them as
follow-ups to this mail.
// Martin
^ permalink raw reply
* [PATCH/RFC] Allow curl to rewind the RPC read buffer
From: Martin Storsjö @ 2009-12-01 10:33 UTC (permalink / raw)
To: Tay Ray Chuan
Cc: git, Nicholas Miell, gsky51, Clemens Buchacher, Mark Lodato,
Johannes Schindelin
In-Reply-To: <alpine.DEB.2.00.0912011208160.5582@cone.home.martin.st>
When using multi-pass authentication methods, the curl library may need
to rewind the read buffers used for providing data to HTTP POST, if data
has been output before a 401 error is received.
This is needed only when the first request (when the multi-pass
authentication method isn't initialized and hasn't received its challenge
yet) for a certain curl session is a chunked HTTP POST.
As long as the current rpc read buffer is the first one, we're able to
rewind without need for additional buffering.
The curl library currently starts sending data without waiting for a
response to the Expect: 100-continue header, due to a bug in curl that
exists up to curl version 7.19.7.
If the HTTP server doesn't handle Expect: 100-continue headers properly
(e.g. Lighttpd), the library has to start sending data without knowing
if the request will be successfully authenticated. In this case, this
rewinding solution is not sufficient - the whole request will be sent
before the 401 error is received.
Signed-off-by: Martin Storsjo <martin@martin.st>
---
The curl bug is yet unconfirmed upstream, discussed here:
http://article.gmane.org/gmane.comp.web.curl.library/25991
remote-curl.c | 30 ++++++++++++++++++++++++++++++
1 files changed, 30 insertions(+), 0 deletions(-)
diff --git a/remote-curl.c b/remote-curl.c
index a331bae..28b2a31 100644
--- a/remote-curl.c
+++ b/remote-curl.c
@@ -290,6 +290,7 @@ struct rpc_state {
int out;
struct strbuf result;
unsigned gzip_request : 1;
+ unsigned initial_buffer : 1;
};
static size_t rpc_out(void *ptr, size_t eltsize,
@@ -300,6 +301,7 @@ static size_t rpc_out(void *ptr, size_t eltsize,
size_t avail = rpc->len - rpc->pos;
if (!avail) {
+ rpc->initial_buffer = 0;
avail = packet_read_line(rpc->out, rpc->buf, rpc->alloc);
if (!avail)
return 0;
@@ -314,6 +316,29 @@ static size_t rpc_out(void *ptr, size_t eltsize,
return avail;
}
+#ifndef NO_CURL_IOCTL
+curlioerr rpc_ioctl(CURL *handle, int cmd, void *clientp)
+{
+ struct rpc_state *rpc = clientp;
+
+ switch (cmd) {
+ case CURLIOCMD_NOP:
+ return CURLIOE_OK;
+
+ case CURLIOCMD_RESTARTREAD:
+ if (rpc->initial_buffer) {
+ rpc->pos = 0;
+ return CURLIOE_OK;
+ }
+ fprintf(stderr, "Unable to rewind rpc post data - try increasing http.postBuffer\n");
+ return CURLIOE_FAILRESTART;
+
+ default:
+ return CURLIOE_UNKNOWNCMD;
+ }
+}
+#endif
+
static size_t rpc_in(const void *ptr, size_t eltsize,
size_t nmemb, void *buffer_)
{
@@ -370,8 +395,13 @@ static int post_rpc(struct rpc_state *rpc)
*/
headers = curl_slist_append(headers, "Expect: 100-continue");
headers = curl_slist_append(headers, "Transfer-Encoding: chunked");
+ rpc->initial_buffer = 1;
curl_easy_setopt(slot->curl, CURLOPT_READFUNCTION, rpc_out);
curl_easy_setopt(slot->curl, CURLOPT_INFILE, rpc);
+#ifndef NO_CURL_IOCTL
+ curl_easy_setopt(slot->curl, CURLOPT_IOCTLFUNCTION, rpc_ioctl);
+ curl_easy_setopt(slot->curl, CURLOPT_IOCTLDATA, rpc);
+#endif
if (options.verbosity > 1) {
fprintf(stderr, "POST %s (chunked)\n", rpc->service_name);
fflush(stderr);
--
1.6.4.4
^ permalink raw reply related
* [PATCH/RFC] Allow curl to rewind the RPC read buffer at any time
From: Martin Storsjö @ 2009-12-01 10:37 UTC (permalink / raw)
To: Tay Ray Chuan
Cc: git, Nicholas Miell, gsky51, Clemens Buchacher, Mark Lodato,
Johannes Schindelin
In-Reply-To: <alpine.DEB.2.00.0912011208160.5582@cone.home.martin.st>
When using multi-pass authentication methods, the curl library may
need to rewind the read buffers used for providing data to HTTP POST,
if data has been output before a 401 error is received.
This solution buffers all data read by the curl library, in order to allow
it to rewind the reading buffer at any time later.
If communicating with a HTTP server that doesn't support the
Expect: 100-continue headers, all HTTP POST data will be sent before
the server replies with the 401 error containing the authentication
challenge.
The buffering is enabled only if the rpc_service function allocates a
buffer - this should perhaps be limited to the cases where http.authAny
is enabled.
Signed-off-by: Martin Storsjo <martin@martin.st>
---
remote-curl.c | 53 +++++++++++++++++++++++++++++++++++++++++++++++++++++
1 files changed, 53 insertions(+), 0 deletions(-)
diff --git a/remote-curl.c b/remote-curl.c
index a331bae..c1c5ccd 100644
--- a/remote-curl.c
+++ b/remote-curl.c
@@ -286,6 +286,10 @@ struct rpc_state {
size_t alloc;
size_t len;
size_t pos;
+ char *rewind_buf;
+ size_t rewind_buf_size;
+ size_t rewind_buf_write_pos;
+ size_t rewind_buf_read_pos;
int in;
int out;
struct strbuf result;
@@ -299,12 +303,28 @@ static size_t rpc_out(void *ptr, size_t eltsize,
struct rpc_state *rpc = buffer_;
size_t avail = rpc->len - rpc->pos;
+ if (rpc->rewind_buf && rpc->rewind_buf_read_pos < rpc->rewind_buf_write_pos) {
+ avail = rpc->rewind_buf_write_pos - rpc->rewind_buf_read_pos;
+ if (max < avail)
+ avail = max;
+ memcpy(ptr, rpc->rewind_buf + rpc->rewind_buf_read_pos, avail);
+ rpc->rewind_buf_read_pos += avail;
+ return avail;
+ }
+
if (!avail) {
avail = packet_read_line(rpc->out, rpc->buf, rpc->alloc);
if (!avail)
return 0;
rpc->pos = 0;
rpc->len = avail;
+
+ if (rpc->rewind_buf) {
+ ALLOC_GROW(rpc->rewind_buf, rpc->rewind_buf_write_pos + avail, rpc->rewind_buf_size);
+ memcpy(rpc->rewind_buf + rpc->rewind_buf_write_pos, rpc->buf, avail);
+ rpc->rewind_buf_write_pos += avail;
+ rpc->rewind_buf_read_pos += avail;
+ }
}
if (max < avail)
@@ -314,6 +334,26 @@ static size_t rpc_out(void *ptr, size_t eltsize,
return avail;
}
+#ifndef NO_CURL_IOCTL
+curlioerr rpc_ioctl(CURL *handle, int cmd, void *clientp)
+{
+ struct rpc_state *rpc = clientp;
+
+ switch (cmd) {
+ case CURLIOCMD_NOP:
+ return CURLIOE_OK;
+
+ case CURLIOCMD_RESTARTREAD:
+ rpc->rewind_buf_read_pos = 0;
+ rpc->pos = rpc->len;
+ return CURLIOE_OK;
+
+ default:
+ return CURLIOE_UNKNOWNCMD;
+ }
+}
+#endif
+
static size_t rpc_in(const void *ptr, size_t eltsize,
size_t nmemb, void *buffer_)
{
@@ -370,8 +410,18 @@ static int post_rpc(struct rpc_state *rpc)
*/
headers = curl_slist_append(headers, "Expect: 100-continue");
headers = curl_slist_append(headers, "Transfer-Encoding: chunked");
+ if (rpc->rewind_buf) {
+ ALLOC_GROW(rpc->rewind_buf, rpc->rewind_buf_write_pos + rpc->len, rpc->rewind_buf_size);
+ memcpy(rpc->rewind_buf, rpc->buf, rpc->len);
+ rpc->rewind_buf_read_pos = rpc->len;
+ rpc->rewind_buf_write_pos = rpc->len;
+ }
curl_easy_setopt(slot->curl, CURLOPT_READFUNCTION, rpc_out);
curl_easy_setopt(slot->curl, CURLOPT_INFILE, rpc);
+#ifndef NO_CURL_IOCTL
+ curl_easy_setopt(slot->curl, CURLOPT_IOCTLFUNCTION, rpc_ioctl);
+ curl_easy_setopt(slot->curl, CURLOPT_IOCTLDATA, rpc);
+#endif
if (options.verbosity > 1) {
fprintf(stderr, "POST %s (chunked)\n", rpc->service_name);
fflush(stderr);
@@ -472,6 +522,8 @@ static int rpc_service(struct rpc_state *rpc, struct discovery *heads)
rpc->buf = xmalloc(rpc->alloc);
rpc->in = client.in;
rpc->out = client.out;
+ rpc->rewind_buf_size = 100;
+ rpc->rewind_buf = xmalloc(rpc->rewind_buf_size);
strbuf_init(&rpc->result, 0);
strbuf_addf(&buf, "%s/%s", url, svc);
@@ -503,6 +555,7 @@ static int rpc_service(struct rpc_state *rpc, struct discovery *heads)
free(rpc->hdr_content_type);
free(rpc->hdr_accept);
free(rpc->buf);
+ free(rpc->rewind_buf);
strbuf_release(&buf);
return err;
}
--
1.6.4.4
^ permalink raw reply related
* Transplant branch from another repository
From: Jeenu V @ 2009-12-01 11:01 UTC (permalink / raw)
To: git
Hi,
Say, I have two repositories A and B (local, independent, but similar
- they are for content tracking and not collaboration purposes). A has
a branch 'a', which I want to have in B. What I mean is that I'd like
to have the sequence of changes in the branch 'a' to be present in B,
thus creating an independent branch 'b' in B.
Is there any way to achieve this? One thing that I could think of is
to use 'format-patch' to generate the list of patch files from A. But
I don't see how to convert those patches to a sequence of commits in
repo B. I could do a 'git apply patches/*' but then all patches
collapse to one single commit. If format-patch is a/the way, could
somebody tell me how to get this done? Or are there any alternatives?
FWIW: I'm running Git under Cygwin, and sendmail isn't configured.
Thanks
Jeenu
^ permalink raw reply
* Re: equal-tree-merges as way to make rebases fast-forward-able
From: Michael Haggerty @ 2009-12-01 11:50 UTC (permalink / raw)
To: Bernhard R. Link; +Cc: git
In-Reply-To: <cover.1259524136.git.brlink@debian.org>
Bernhard R. Link wrote:
> Example 1:
>
> Let's assume you maintain such a regularily-rebased branch that you
> want to be able to publish (or pull from other repositories for example
> on your laptop):
>
> o=m=o=o=master
> \
> a=b=c=d=e=feature
>
> with this patch you can do "git rebase -eqt master" and get:
>
> a'=b'=c'=d'=e'=feature'=eqt
> / /
> o=m=o=o=master-------- /
> \ \ /
> a=b=c=d=e=feature--merge-------
Actually, there is more information that can be retained about this
rebase operation. Your scheme records the fact that (a+b+c+d+e+merge)
== (o+o+a'+b'+c'+d'+e'), which is certainly true. But in the process of
rebasing, the user has (implicitly or explicitly) resolved conflicts in
transforming each of the patches a -> a', b -> b', etc. In fact, the
patch a' is itself a merge between a and master; b' is a merge between b
and a'; etc. If you record each of these merges individually, the
result looks like this:
o=m=o=o=master
\ \
\ a'=b'=c'=d'=e'=feature'
\ / / / / /
---a==b==c==d==e==feature
There are advantages to retaining all of this history:
* It faithfully represents intermediate steps of the rebase.
* There is no need for special "merge" and "eqt" merge commits affecting
an arbitrary group of feature patches; each of the rebased patches is
treated identically.
* There is a direct ancestry connection from the "new version" to the
"old version" of each patch; for example, it is easy to see that c' is a
new version of c and to compute the corresponding interdiffs.
* There are situations where the additional info can help git choose
better merge bases in the case of merge/rebases across three or more
repositories. For example, somebody who is developing a subfeature
based on the feature branch can merge/rebase changes from both feature
and master without causing utter chaos.
The "historical" version of the feature branch should be omitted from
most git output as you have suggested, but this would be best
implemented by marking the "historical" ancestor with some extra flag in
each merge commit.
> Example 2:
>
> Let's assume you have a feature branch like
>
> o=master
> \
> a=b=c=d=e=f
>
> Assume you just commited "f" which fixes a bug introduced by "b". [...]
>
> So with this patches you can do "git rebase -i --eqt" and squash f into b
> and get:
>
> o=master
> \
> a=b=c=d=e=f---
> \ \
> b+f=c'=d'=e'=eqt
This case can also record additional information:
o=master
\
a=b===c==d=e=f
\ \ \ \
b+f=c'=d'==e'
Here the new DAG cannot represent *all* ancestry information (namely,
that b+f, c', and d' also include the original patch f), but it does
accurately reflect useful information such as that c' includes c and
that e' includes e and f.
I wrote some blog entries about rebasing-with-history that might be
interesting [1-3].
Michael
[1]
http://softwareswirl.blogspot.com/2009/04/truce-in-merge-vs-rebase-war.html
[2]
http://softwareswirl.blogspot.com/2009/08/upstream-rebase-just-works-if-history.html
[3]
http://softwareswirl.blogspot.com/2009/08/rebase-with-history-implementation.html
^ permalink raw reply
* Re: Newbie
From: Mikko Oksalahti @ 2009-12-01 12:00 UTC (permalink / raw)
To: git
In-Reply-To: <26ae428a0912010145k61dbfc41l8243363493918445@mail.gmail.com>
Howard Miller <howardsmiller <at> googlemail.com> writes:
>
> 2009/12/1 Mikko Oksalahti <mikko <at> azila.fi>:
> > How do I now get the accidentally deleted files back from the repository
without
> > losing local changes made to 10 files?
>
> 'git status' should show you what files you have deleted. 'git
> checkout filename' should get them back. I can't think of a way of
> recovering every file you have just deleted although - I suspect it
> might be tricky. Thinks like 'git pull' only apply to remote
> repositories and you don't have one of those.
>
> Howard
>
Ok. That helps. I just assumed the 'git pull' would work same way on local and
remote repositories but I guess not then...
> ....You're not thinking of it the right way (yet)
I hate when you say that :P
/Mikko
^ permalink raw reply
* Re: non-US-ASCII file names (e.g. Hiragana) on Windows
From: Thomas Singer @ 2009-12-01 12:08 UTC (permalink / raw)
To: Johannes Sixt; +Cc: git
In-Reply-To: <4B14E934.9090304@viscovery.net>
Johannes Sixt wrote:
> Thomas Singer schrieb:
>> Is it a German Windows limitation, that far-east characters are not
>> supported on it (but work fine on a Japanese Windows), are there different
>> (mysys)Git versions available or is this a configuration issue?
>
> It is a matter of configuration.
>
> Since 8 bits are not sufficient to support Japanese alphabet in addition
> to the German alphabet, programs that are not Unicode aware -- such as git
> -- have to make a decision which alphabet they support. The decision is
> made by picking a "codepage".
>
> On German Windows, you are in codepage 850 (in the console). The filenames
> (that actually are in Unicode) are converted to bytes according to
> codepage 850 *before* git sees them. If your filenames contain Hiragana,
> they are substituted by the "unknown character" marker because there is no
> place for them in codepage 850.
>
> However, you can install Japanese language support on German Windows. Then
> you can change your console to codepage 932:
>
> chcp 932
>
> When you run git from *this* console, Hiragana in the filenames are
> converted to cp932 before git sees them. The resulting byte sequence is
> different from the one in cp850, but git will be able to see that the file
> exists and was modified, and you can 'git add' it.
>
> But if you have files with umlauts, they will not be recognized anymore
> because umlauts have no place in cp932.
>
> In neither case can you exchange the repository with Linux if you have
> your locale set to UTF-8 on Linux, because neither byte sequence (umlauts
> from cp850 or Hiragana from cp932) are valid UTF-8 sequences, let alone
> result in the expected glyphs.
>
> Corollary: Stick to ASCII file names.
>
> There have been suggestions to switch the console to codepage 65001
> (UTF-8), but I have never heard of success reports. I'm not saying it does
> not work, though.
Thanks for the detailed explanation. I know the differences between bytes
and characters and the needed *encoding* to convert from one to another, but
I did not know how Git handles it. I'm quite surprised, that -- as I
understand you -- msys-Git (or Git at all?) is not able to handle all
characters (aka unicode) at the same time. I expected it would be better
than older tools, e.g. SVN.
BTW, we are invoking the Git executable from Java. Is there automatically a
console "around" Git? Should we invoke a shell-script (which sets the
console's code page) instead of the Git executable directly?
--
Tom
^ permalink raw reply
* Re: non-US-ASCII file names (e.g. Hiragana) on Windows
From: Thomas Singer @ 2009-12-01 12:11 UTC (permalink / raw)
To: kusmabite; +Cc: Maximilien Noal, git
In-Reply-To: <40aa078e0912010112u4205452as3627ba019544c5fe@mail.gmail.com>
Erik Faye-Lund wrote:
> Did you try to make sure your console window used a Unicode font on
> your German Windows installation? Asian Windows installations might do
> this by default, something at least neither English nor Norwegian
> Windows installations seems to do...
>
> You can change the console window font through the properties-menu
> that appears when you right click the title-bar.
I've tried to change the console font (there is just one alternative), but
without any change.
--
Tom
^ permalink raw reply
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox