* Query for certain branches, but not the rest
From: Felipe Contreras @ 2012-11-25 9:24 UTC (permalink / raw)
To: git
Hi,
Suppose I have these branches:
fc/feature-a
fc/feature-b
master
next
I want to show this: fc/feature-a fc/feature-b ^master ^next. I can do
'git log --branches=fc' to show the branches that begin with fc/, but
there's no way to specify the rest. If they were under a prefix, I
could do '--not --branches=prefix', but they are not.
Anybody knows a way to query branches that don't have any prefix?
It appears this works:
% git for-each-ref refs/heads/*
But I would like something parsable by rev-parse.
Any ideas?
Cheers.
--
Felipe Contreras
^ permalink raw reply
* Re: Python extension commands in git - request for policy change
From: Eric S. Raymond @ 2012-11-25 9:53 UTC (permalink / raw)
To: Felipe Contreras; +Cc: git
In-Reply-To: <CAMP44s18MzmWRNRiRjL6hvpK1cm=S-97fB2ep-_0RAhnfs5cvA@mail.gmail.com>
Felipe Contreras <felipe.contreras@gmail.com>:
> If your friends jump off a bridge, would you? Yes, using python has
> served them well, but as opposed to what? Other scripting languages? I
> don't think so.
The competition that Python won was *precisely* against other scripting
languages, notably shell and Perl. Both used to be much more heavily
used in system scripting than they are now.
> What if my extension only supports python 2.7? Or what if my extension
> wants to support 2.0?
I propose that if 2.6 can't support it, then that should be considered
grounds to reject it.
> Yes, they should _if_ they know what version they need. In my
> extensions I really have no idea.
Then you shouldn't submit those extensions to be folded into core git.
> > 3) We should be unconditionally be encouraging extensions to move
> > from shell and Perl to Python. This would be a clear net gain is
> > portability and maintainability.
>
> NO! It's up to the developer to choose what language to use,
I agree. You seem to be raising a lot of straw men. 'Encouragement'
does not equate to beating anyone who makes an unpopular choice over
the head.
I am also not suggesting that the whole git core ought to be hoicked
over to Python. I was thinking mainly about extension subcommands,
not what's in libgit now.
--
<a href="http://www.catb.org/~esr/">Eric S. Raymond</a>
^ permalink raw reply
* Re: Python extension commands in git - request for policy change
From: Eric S. Raymond @ 2012-11-25 9:54 UTC (permalink / raw)
To: Felipe Contreras; +Cc: Nguyen Thai Ngoc Duy, git, msysGit
In-Reply-To: <CAMP44s0r1J=aOuEpKQ1+ew9FzODwLX-w5z9rG-WN6AjU0o97yw@mail.gmail.com>
Felipe Contreras <felipe.contreras@gmail.com>:
> On Sun, Nov 25, 2012 at 6:18 AM, Eric S. Raymond <esr@thyrsus.com> wrote:
> > Nguyen Thai Ngoc Duy <pclouds@gmail.com>:
> >> These may apply to other languages as well. Where do we draw a line?
> >
> > I'm in favor of the general policy of avoiding scripting languages
> > other than the top three most widely deployed. At the moment that
> > means shell, Python, Perl; on present trends, in a few years Perl
> > (dropping in popularity) might be passed by Ruby on the way up.
>
> Top three according to whom?
According to the LOC counts in git's codebase.
--
<a href="http://www.catb.org/~esr/">Eric S. Raymond</a>
^ permalink raw reply
* Re: git bash does not access drive f:
From: Sebastian Leske @ 2012-11-23 0:58 UTC (permalink / raw)
To: git
In-Reply-To: <CAB9Jk9Ae46PNRex9QrEy9gTgqAbn8KUFifmxQU4s5K5mDDmDZQ@mail.gmail.com>
On 2012-11-22, Angelo Borsotti <angelo.borsotti@gmail.com> wrote:
> Hi
>
> I have attached an external disc, which appears on Windows as drive f:
> in Windows Explorer.
> Right-clicking on it displays a context menu showing (among other
> items) Git Init Here, Git Gui and
> Git Bash. The first two work properly on that drive.
> However, the git bash does not. Not even the one that is run from the icon:
>
> $ cd f:
> sh.exe": cd: f:: No such file or directory
This is probably a known limitation of MSYS (the shell environment that
Git Bash uses, which is part of MingW):
Drive letters added after MSYS has started only become visible if all
MSYS processes are stopped and restarted. See e.g.
How to register newly mounted drive in git bash?
http://stackoverflow.com/questions/5197540/
for some more details.
It seems there is no easy workaround, so you'll need to close all Git
Bash windows and reopen them.
^ permalink raw reply
* Re: Re: Python extension commands in git - request for policy change
From: Pat Thoyts @ 2012-11-25 10:26 UTC (permalink / raw)
To: Nguyen Thai Ngoc Duy; +Cc: Eric S. Raymond, git, msysGit
In-Reply-To: <CACsJy8BbUjrJtfpEvbcK==Y2gFNsFhFBN93CL36J5uVe=Ca4wQ@mail.gmail.com>
On 25 November 2012 03:15, Nguyen Thai Ngoc Duy <pclouds@gmail.com> wrote:
> CCing msysgit. I vaguely remember they had problems with building
> Python on Windows. I don't know if it's still an issue.
>
> On Sun, Nov 25, 2012 at 9:44 AM, Eric S. Raymond <esr@thyrsus.com> wrote:
>> git presently contains one Python extension command, Pete Wycoff's p4
>> importer. If my git-weave code is merged it will acquire another.
>> I think we can expect more submissions of Python extensions in the
>> future, for two good reasons:
>>
>> 1. Python has a much richer type ontology than shell; there are many
>> things this makes relatively easy that are quite painful in shell.
>>
>> 2. While Perl shares advantage #1, compared to Python it's a
>> maintainability mess - much more difficult to read 6 months later.
>>
>> On the other hand,
>>
>> 3. Attitudes in the git dev group seem to be influenced by a
>> perception that up-to-date Python versions are not as reliably present
>> on our target platforms as Perl is.
>>
>> 4. Python has the disadvantage that comes with robust growth; you have
>> to specify "version x.y or later" as a dependency, mainly because new
>> modules keep getting getting folded into the stock Python environment.
>
> These may apply to other languages as well. Where do we draw a line?
>
>
>> Previous conversation on the list suggests that there has been a tacit
>> policy of managing these problems by (a) discouraging (though not entirely
>> forbidding) Python extensions, and (b) requiring extension submitters to
>> document some dependency on language version.
>>
>> I think this is suboptimal. By not forbidding the Python language
>> entirely, we guarantee having to deal with problems 3 and 4 anyway -
>> but by discouraging it, we're buying significant long-term
>> maintainability costs. It especially disturbed me to hear of Python
>> commands being recoded in C - that is definitely not the right
>> direction for reducing expected defect counts, if only because of
>> memory-management issues.
>>
>> We're behind the best-practices curve here. The major Linux
>> distributions, which have to deal with almost the same set of
>> tradeoffs we do, went to Python for pretty much all glue and
>> administration scripts outside /etc a decade ago, and the decision has
>> served them well.
>>
>> That, among other things, means up-to-date versions of Python are
>> ubiquitous unless we're looking at Windows - in which case Perl and
>> shell actually become much bigger portability problems. Mac OS X
>> has kept up to date, too; Lion shipped 2.7.1 and that was a major
>> release back at this point.
>>
>> To be fair, there was a time when being a bit twitchy about Python
>> version skew and deployment breadth was justified, but I believe that
>> time is now well past us. My basis for believing this is very simple -
>> I maintain a lot of Python code for systems programmers with stiff
>> portability requirements (things like reposurgeon, coverity-submit,
>> freecode-submit, shipper, and the Python tools in gpsd). I know what
>> kinds of bug reports I get and what kinds I don't, and in the last
>> few years "this breaks on my Python version" has gone from unusual
>> to doesn't-happen.
>>
>> I think my experience with gpsd is particularly instructive. Like
>> git, that project has a C core with Python wrappers and extension
>> components. Like git, it gets deployed in a lot of odd places by people
>> who cannot afford the time to be tolerant about cross-platform
>> problems and are quite willing to hit the maintainer with a clue-bat
>> when they encounter them. The good news is - they don't have to.
>>
>> I should also point out that none of Mercurial's problems seem to
>> have anything to do with the fact that it's written in Python...
>>
>> I think we can choose a better policy based on some simple premises.
>>
>> 1) In 2012, we can specify a "floor" Python version of 2.6 (shipped in
>> 2008) and be pretty much guaranteed it will be anywhere we want to
>> deploy except Windows. Windows will remain a problem because Python
>> isn't part of the stock install, but that's an equal or worse problem
>> for shell and Perl - and at least the Python project ships a binary
>> installer for Windows.
>>
>> 2) Python extension commands should test the Python version on startup
>> and die loudly but gracefully in the rare case that they don't find
>> what they need.
>>
>> 3) We should be unconditionally be encouraging extensions to move
>> from shell and Perl to Python. This would be a clear net gain is
>> portability and maintainability.
>>
>> 4) We should be encouraging C code to move to Python, too. There's
>> little gain in portability on this path because modern C has cleaned
>> up its act a lot, but the drop in expected bug loads would be well
>> worth the porting effort. Segfaults are not your friend, and the x2 to
>> x5 drop in line count would do very good things for long-term
>> maintainability.
Git for Windows simply ships everything we need to run git - so if a
desirable module requires a version of python, we will add that
version plus any required modules into the installer. We already have
a patch to provide python in the msysgit tree - it would just require
polishing up a little. I'm certain this is no problem for the other
windows port (cygwin) either.
--
*** Please reply-to-all at all times ***
*** (do not pretend to know who is subscribed and who is not) ***
*** Please avoid top-posting. ***
The msysGit Wiki is here: https://github.com/msysgit/msysgit/wiki - Github accounts are free.
You received this message because you are subscribed to the Google
Groups "msysGit" group.
To post to this group, send email to msysgit@googlegroups.com
To unsubscribe from this group, send email to
msysgit+unsubscribe@googlegroups.com
For more options, and view previous threads, visit this group at
http://groups.google.com/group/msysgit?hl=en_US?hl=en
^ permalink raw reply
* Re: Python extension commands in git - request for policy change
From: Eric S. Raymond @ 2012-11-25 10:25 UTC (permalink / raw)
To: Johannes Sixt; +Cc: git
In-Reply-To: <50B1DD78.5040907@kdbg.org>
Johannes Sixt <j6t@kdbg.org>:
> Am 25.11.2012 03:44, schrieb Eric S. Raymond:
> > That, among other things, means up-to-date versions of Python are
> > ubiquitous unless we're looking at Windows - in which case Perl and
> > shell actually become much bigger portability problems.
>
> You seem to ignore that more than a quater of users are on Windows[1].
> This is not negligible.
I'm not ignoring that at all. There are questions of fact here:
Are Perl and a POSIX shell part of the stock installation of Windows?
I believe the answer is "no". You are free to correct me, but if that's
true they don't have any obvious portability advantage over Python.
That means the 25% percent of Windows users are not actually a reason
to prefer them.
> Absolutely not. To achieve best portability, all code should move to C
> instead.
I wrote the (first) book on C portability. I mean that literally -
"Portable C and Unix Systems Programming", Prentice-Hall 1987. Please
don't feel insulted when I point out that over the last 25 years I
have probably forgotten more about this topic than you know. Just
listen when I tell you that it is not at all obvious that raw C is the
maximally portable language.
It may very well be the case that some random scripting language (not
necessarily Python) achieves greater portability simply because its
maintainers get to pay more concentrated attention to the portability
of the environment bindings at the bottom of their C implementation than
we can.
In any case, I don't believe the difference in portability between raw
C and Python is large enough in either direction to be a reason to
favor either, and I speak as a domain expert on this issue. This is
not Python advocacy talking; the same could be said of Perl or Ruby.
The real advantages of a scripting language are in maintainability and
expected defect rates, not portability. The three relevant things we kbnow
from large-scale studies of software defect patterns are these:
1) Expected defect counts are predictable from LOC.
2) Moving to any given scripting language from C dramatically reduces LOC,
and thus expected defects over time.
3) Moving to any scripting language from C eliminates a class of
memory-management problems that dominate C defect statistics.
--
<a href="http://www.catb.org/~esr/">Eric S. Raymond</a>
^ permalink raw reply
* Re: Re: Python extension commands in git - request for policy change
From: Eric S. Raymond @ 2012-11-25 10:33 UTC (permalink / raw)
To: Pat Thoyts; +Cc: Nguyen Thai Ngoc Duy, git, msysGit
In-Reply-To: <CABNJ2G+CevGU=-DjC073yGv0gupd9QK6eyjhrrQTNNmTkq_fxg@mail.gmail.com>
Pat Thoyts <patthoyts@gmail.com>:
> Git for Windows simply ships everything we need to run git - so if a
> desirable module requires a version of python, we will add that
> version plus any required modules into the installer. We already have
> a patch to provide python in the msysgit tree - it would just require
> polishing up a little. I'm certain this is no problem for the other
> windows port (cygwin) either.
Thank you - I think this completely disposes of the "Windows is a blocker
for scripting language X" argument, with the case X = Python in point.
--
<a href="http://www.catb.org/~esr/">Eric S. Raymond</a>
--
*** Please reply-to-all at all times ***
*** (do not pretend to know who is subscribed and who is not) ***
*** Please avoid top-posting. ***
The msysGit Wiki is here: https://github.com/msysgit/msysgit/wiki - Github accounts are free.
You received this message because you are subscribed to the Google
Groups "msysGit" group.
To post to this group, send email to msysgit@googlegroups.com
To unsubscribe from this group, send email to
msysgit+unsubscribe@googlegroups.com
For more options, and view previous threads, visit this group at
http://groups.google.com/group/msysgit?hl=en_US?hl=en
^ permalink raw reply
* [PATCH 4/4] git-svn: Note about tags.
From: Sebastian Leske @ 2012-11-23 7:29 UTC (permalink / raw)
Cc: Eric Wong
In-Reply-To: <b115a546fa783b4121d118bb8fdb9270443f90fa.1353691892.git.Sebastian.Leske@sleske.name>
Document that 'git svn' will import SVN tags as branches.
Signed-off-by: Sebastian Leske <sebastian.leske@sleske.name>
---
Documentation/git-svn.txt | 6 ++++++
1 file changed, 6 insertions(+)
diff --git a/Documentation/git-svn.txt b/Documentation/git-svn.txt
index 92780ef..6212b24 100644
--- a/Documentation/git-svn.txt
+++ b/Documentation/git-svn.txt
@@ -938,6 +938,12 @@ the possible corner cases (git doesn't do it, either). Committing
renamed and copied files is fully supported if they're similar enough
for git to detect them.
+In SVN, it is possible (though discouraged) to commit changes to a tag
+(because a tag is just a directory copy, thus technically the same as a
+branch). When cloning an SVN repository, 'git svn' cannot know if such a
+commit to a tag will happen in the future. Thus it acts conservatively
+and imports all SVN tags as branches, prefixing the tag name with 'tags/'.
+
CONFIGURATION
-------------
--
1.7.10.4
^ permalink raw reply related
* [PATCH 3/4] git-svn: Expand documentation for --follow-parent
From: Sebastian Leske @ 2012-11-23 7:13 UTC (permalink / raw)
Cc: Eric Wong
In-Reply-To: <b115a546fa783b4121d118bb8fdb9270443f90fa.1353691892.git.Sebastian.Leske@sleske.name>
Describe what the option --follow-parent does, and what happens if it is
set or unset.
Signed-off-by: Sebastian Leske <sebastian.leske@sleske.name>
---
Documentation/git-svn.txt | 16 ++++++++++++----
1 file changed, 12 insertions(+), 4 deletions(-)
diff --git a/Documentation/git-svn.txt b/Documentation/git-svn.txt
index 928a961..92780ef 100644
--- a/Documentation/git-svn.txt
+++ b/Documentation/git-svn.txt
@@ -628,11 +628,19 @@ ADVANCED OPTIONS
Default: "svn"
--follow-parent::
+ This option is only relevant if we are tracking branches (using
+ one of the repository layout options --trunk, --tags,
+ --branches, --stdlayout). For each tracked branch, try to find
+ out where its revision was copied (i.e. branched) from, and set
+ a suitable parent in the first git commit for the branch.
This is especially helpful when we're tracking a directory
- that has been moved around within the repository, or if we
- started tracking a branch and never tracked the trunk it was
- descended from. This feature is enabled by default, use
- --no-follow-parent to disable it.
+ that has been moved around within the repository (note that you
+ must track both the old and the new name for the whole history
+ to be imported). If this feature is disabled, the branches
+ created by 'git svn' will all be linaear and not share any
+ history, meaning that there will be no information on where
+ branches where branched off or merged. This feature is enabled
+ by default, use --no-follow-parent to disable it.
+
[verse]
config key: svn.followparent
--
1.7.10.4
^ permalink raw reply related
* [PATCH 1/4] git-svn: Document branches with at-sign(@).
From: Sebastian Leske @ 2012-11-18 17:08 UTC (permalink / raw)
Cc: Eric Wong
git svn will sometimes create branches with an at-sign in the name
(branchname@revision). These branches confuse many users and it is a FAQ
why they are created. Document when git svn will create them.
Signed-off-by: Sebastian Leske <sebastian.leske@sleske.name>
---
I found various important features of git-svn that are not documented.
Thus I created a series of patches to add and update documentation for
git-svn. I am sending this as four patches (of which this mail is the
first). I have tried my best to make the patches ready for inclusion
into git, but if there's any feedback, I'll gladly incorporate it.
Documentation/git-svn.txt | 26 ++++++++++++++++++++++++++
1 file changed, 26 insertions(+)
diff --git a/Documentation/git-svn.txt b/Documentation/git-svn.txt
index 8b0d3ad..482d60d 100644
--- a/Documentation/git-svn.txt
+++ b/Documentation/git-svn.txt
@@ -881,6 +881,32 @@ different name spaces. For example:
branches = stable/*:refs/remotes/svn/stable/*
branches = debug/*:refs/remotes/svn/debug/*
+If 'git svn' is configured to fetch branches (and --follow-branches
+is in effect), it will sometimes create multiple branches for one SVN
+branch, where the addtional branches have names of the form
+'branchname@nnn' (with nnn an SVN revision number). These additional
+branches are created if 'git svn' cannot find a parent commit for the
+first commit in the branch, to connect the branch to the history of the
+other branches. Normally, 'git svn' will find the git commit that
+corresponds to the SVN revision a branch was created (copied) from and
+use that as the parent. However, it is possible that there is no
+suitable git commit to serve as parent. This will happen, among other
+reasons, if the SVN branch is a copy of a revision that was not fetched
+by 'git svn' (e.g. because it is an old revision that was skipped with
+'--revision'), or if in SVN a directory was copied that is not tracked
+by 'git svn' (a branch that is not tracked at all, or a subdirectory of
+a tracked branch). In these cases, 'git svn' will still create a git
+branch, but the branch will not be connected to the history of the other
+branches. 'git svn' will also create a commit from the SVN revision the
+branch was copied from, and use that as the parent commit of the branch
+(this is indicated by the message "Initializing parent: <branchname>").
+Additionally, it will create a special branch named
+'<branchname>@<SVN-Revision>', where <SVN-Revision> is the SVN revision
+number the branch was copied from. This branch will point to the
+specially created parent commit.
+If in SVN the branch was deleted and later recreated from a different
+version, there will be multiple such branches with an '@'.
+
BUGS
----
--
1.7.10.4
^ permalink raw reply related
* [PATCH 2/4] Recommend use of structure options for git svn.
From: Sebastian Leske @ 2012-11-21 7:13 UTC (permalink / raw)
Cc: Eric Wong
In-Reply-To: <b115a546fa783b4121d118bb8fdb9270443f90fa.1353691892.git.Sebastian.Leske@sleske.name>
Document that when using git svn, one should usually either use the
directory structure options to import branches as branches, or only
import one subdirectory. The default behaviour of cloning all branches
and tags as subdirectories in the working copy is usually not what the
user wants.
Signed-off-by: Sebastian Leske <sebastian.leske@sleske.name>
---
Documentation/git-svn.txt | 10 ++++++++++
1 file changed, 10 insertions(+)
diff --git a/Documentation/git-svn.txt b/Documentation/git-svn.txt
index 482d60d..928a961 100644
--- a/Documentation/git-svn.txt
+++ b/Documentation/git-svn.txt
@@ -871,6 +871,16 @@ already dcommitted. It is considered bad practice to --amend commits
you've already pushed to a remote repository for other users, and
dcommit with SVN is analogous to that.
+When cloning an SVN repository, if none of the options for describing
+the repository layout is used (--trunk, --tags, --branches,
+--stdlayout), 'git svn clone' will create a git repository with
+completely linear history, where branches and tags appear as separate
+folders in the working copy. For projects with many branches this will
+lead to a working copy many times larger than just the trunk. It is
+recommended to either use the the options for trunk / tag / branch
+directory, or to only clone the trunk (or a subdirectory of the
+trunk).
+
When using multiple --branches or --tags, 'git svn' does not automatically
handle name collisions (for example, if two branches from different paths have
the same name, or if a branch and a tag have the same name). In these cases,
--
1.7.10.4
^ permalink raw reply related
* Re: Python extension commands in git - request for policy change
From: Michael Haggerty @ 2012-11-25 10:44 UTC (permalink / raw)
To: Felipe Contreras; +Cc: Eric S. Raymond, git
In-Reply-To: <CAMP44s18MzmWRNRiRjL6hvpK1cm=S-97fB2ep-_0RAhnfs5cvA@mail.gmail.com>
On 11/25/2012 09:53 AM, Felipe Contreras wrote:
> On Sun, Nov 25, 2012 at 3:44 AM, Eric S. Raymond <esr@thyrsus.com> wrote:
>> 1) In 2012, we can specify a "floor" Python version of 2.6 (shipped in
>> 2008) and be pretty much guaranteed it will be anywhere we want to
>> deploy except Windows. Windows will remain a problem because Python
>> isn't part of the stock install, but that's an equal or worse problem
>> for shell and Perl - and at least the Python project ships a binary
>> installer for Windows.
>
> What if my extension only supports python 2.7? Or what if my extension
> wants to support 2.0?
There would obviously have to be a policy like "all Python code in core
git must run on any Python interpreter with 2.6 <= version < 3.0", just
as there are policies about what C and shell features are allowed. If
you happen to want to support earlier versions of Python, I don't see
why anybody would stop you as long as your code also runs in the
mandated versions.
(In practice, backwards compatibility within Python versions 2.x is very
good and almost any code that runs in Python 2.6 would automatically run
in all later 2.x versions. Moreover, the Python documentation covering
what is available in each version and the deltas between versions is
high-quality and easily available online.)
There is, of course, the awkward issue of how/when to transition to
Python 3.x, which is *not* backwards compatible with Python 2.x. I
expect that when the time comes there will be volunteers (myself
included) willing to help adapt Python scripts to the new version, but
the problem shouldn't be minimized.
Of course Perl will have the same problem if Perl6 ever materializes.
>> 2) Python extension commands should test the Python version on startup
>> and die loudly but gracefully in the rare case that they don't find
>> what they need.
>
> Yes, they should _if_ they know what version they need. In my
> extensions I really have no idea.
Then simply (with the help of the mailing list) ensure that your
extensions run under 2.6 (or whatever the chosen minimum version is) and
everything will be OK. It is not an error to specify 2.6 as the minimum
version even though your script happens also to run on older versions :-)
>> 3) We should be unconditionally be encouraging extensions to move
>> from shell and Perl to Python. This would be a clear net gain is
>> portability and maintainability.
>
> NO! It's up to the developer to choose what language to use, and I
> find it very chauvinist of you to say "python is better, so let's all
> use python". So far you have listed a few advantages of python, but
> you haven't explained so far what is wrong with shell and perl.
Given that some languages are accepted in git-core and others are not,
it's already not "up to the developer to choose what language to use".
At best there is a short list of "blessed" languages, and the developer
can choose among only those.
> In fact, while advancing python you have made clear a problem with
> python; the version requirements. So far I have *never* encountered a
> problem with git because of my bash version, or my perl version. And
> we haven't touched to the python3 mess yet. To me, those are
> advantages of shell and perl.
On the contrary, there is *constant* traffic on the mailing list about
incompatibilities between different shell implementations (sh, dash,
bash, etc), not to mention those in other utilities (sed, grep, etc)
that one is forced to work with in shell scripts. Compatibility is a
*huge* pain when developing shell code for git. The fact that users
typically don't encounter such problems is due to the hard work of POSIX
lawyers on the mailing list correcting the compatibility errors of
mortal programmers.
> Actually, I don't care if 'git foo' is written in perl, or shell, or
> c; as long as it *works*. And I would hate it if 'git rebase' ever
> told me that I need a newer version of python, or worst; that I don't
> have python in my system (Arch Linux ships 'python2', not 'python').
The configure script would locate the correct interpreter and the build
would adjust the scripts' shebang lines, just as things are tweaked
within Perl scripts at build time.
>> 4) We should be encouraging C code to move to Python, too. There's
>> little gain in portability on this path because modern C has cleaned
>> up its act a lot, but the drop in expected bug loads would be well
>> worth the porting effort. Segfaults are not your friend, and the x2 to
>> x5 drop in line count would do very good things for long-term
>> maintainability.
>
> Definitely NO! I really really doubt git in python would be able to
> achieve the same performance as git in c, but to show me wrong, it
> wouldn't be very difficult to run a few measurements with python
> dulwich *if* we are even to begin considering this point.
>
> And are segmentation faults really that different from python's
> exceptions? Not to the user.
There is one huge difference: it C it is all too easy to write code that
leads to a security hole due to buffer overflows and other memory
management errors. Code written in a scripting language are largely
immune to such problems (except of course for any such bugs in the
interpreter itself, but the testing of the interpreter is shared across
many projects and users).
It would be insane to rewrite performance-critical C code in any
scripting language, but there is a huge penumbra of code that is not
performance critical and that mutates rapidly. Such code is much easier
to write and maintain in a sane scripting language if the portability
issues can be mastered.
The most important issues to consider when imagining a future with a
hybrid of code in C and some scripting language "X" are:
* Portability: is "X" available on all platforms targeted by git, in
usable and mutually-compatible versions?
* Startup time: Is the time to start the "X" interpreter prohibitive?
(On my computer, "python -c pass", which starts the Python
interpreter and does nothing, takes about 24ms.) This overhead would
be incurred by every command that is not pure C.
* Should the scripting language access the C functionality only by
calling pure-C executables or by dynamically or statically linking to
a binary module interface? If the former, then the granularity of
interactions between "X" and C is necessarily coarse, and "X" cannot
be used to implement anything but the outermost layer of
functionality. If the latter, then the way would be clear to
implement much more of git in "X" (and lua would also be worth
considering).
* Learning curve for developers: how difficult is it for a typical git
developer to become conversant with "X", considering both (1) how
likely is it that the typical git developer already knows "X" and
(2) how straightforward and predictable is the language "X"?
In this category I think that Python has a huge advantage over
Perl, though certainly opinions will differ and Ruby would also be
a contender.
Personally, I regret wasting my time programming pointer arithmetic in
git modules that are not performance-critical (and correcting bugs by
others in these areas). And I'm tired of having an idea to improve a
git feature only to find that it is implemented in shell, where not even
arrays are available. I would therefore welcome more friendliness
towards a decent scripting language in the git project.
Michael
--
Michael Haggerty
mhagger@alum.mit.edu
http://softwareswirl.blogspot.com/
^ permalink raw reply
* Re: Python extension commands in git - request for policy change
From: Eric S. Raymond @ 2012-11-25 10:57 UTC (permalink / raw)
To: Michael Haggerty; +Cc: Felipe Contreras, git
In-Reply-To: <50B1F684.5020805@alum.mit.edu>
Michael Haggerty <mhagger@alum.mit.edu>:
> There is, of course, the awkward issue of how/when to transition to
> Python 3.x, which is *not* backwards compatible with Python 2.x. I
> expect that when the time comes there will be volunteers (myself
> included) willing to help adapt Python scripts to the new version, but
> the problem shouldn't be minimized.
2to3 actually does a pretty good job. It doesn't reduce the
transition cost to zero, but I find it does reduce that cost to an
easily manageable level even on quite large codebases.
> It would be insane to rewrite performance-critical C code in any
> scripting language, but there is a huge penumbra of code that is not
> performance critical and that mutates rapidly.
Indeed. In the git architecture there is a pretty clear dividing line -
to a first approximation, plumbing should remain C but porcelain should
probably not. (Not that I am advocating forcing such a move - but it would
be good to allow it to happen.)
The 80-20 rule (80% of the execution time is spent in 20% of the code)
helps us here. The *other* 80% of the code can move to a scripting
language with no significant performance loss. To find out what needs
to stay in C, run a profiler!
--
<a href="http://www.catb.org/~esr/">Eric S. Raymond</a>
^ permalink raw reply
* [PATCH 0/8] Add function strbuf_addstr_xml_quoted() and more
From: Michael Haggerty @ 2012-11-25 11:08 UTC (permalink / raw)
To: Junio C Hamano; +Cc: git, Jeremy White, Johannes Schindelin, Michael Haggerty
There were two functions doing almost the same XML quoting of
character entities, so implement a library function
strbuf_addstr_xml_quoted() and use that in both places.
Along the way, do a lot of simplification within imap-send.c, which
was doing a lot of its own string management instead of using strbuf.
Please note that "git imap-send" is utterly absent from the test
suite, probably due to the difficulty of testing without a real IMAP
server. I ran some manual tests after my changes and didn't find any
problems.
The bug that I reported on 2012-11-12, namely that
git format-patch --signoff --stdout --attach origin | git imap-send
is broken, is not addressed by these patches.
Michael Haggerty (8):
Add new function strbuf_add_xml_quoted()
xml_entities(): use function strbuf_addstr_xml_quoted()
lf_to_crlf(): NUL-terminate msg_data::data
imap-send: store all_msgs as a strbuf
imap-send: correctly report errors reading from stdin
imap-send: change msg_data from storing (char *, len) to storing
strbuf
wrap_in_html(): use strbuf_addstr_xml_quoted()
wrap_in_html(): process message in bulk rather than line-by-line
http-push.c | 23 +--------
imap-send.c | 157 +++++++++++++++++++++++++++---------------------------------
strbuf.c | 26 ++++++++++
strbuf.h | 6 +++
4 files changed, 104 insertions(+), 108 deletions(-)
--
1.8.0
^ permalink raw reply
* [PATCH 1/8] Add new function strbuf_add_xml_quoted()
From: Michael Haggerty @ 2012-11-25 11:08 UTC (permalink / raw)
To: Junio C Hamano; +Cc: git, Jeremy White, Johannes Schindelin, Michael Haggerty
In-Reply-To: <1353841721-16269-1-git-send-email-mhagger@alum.mit.edu>
Substantially the same code is present in http-push.c and imap-send.c,
so make a library function out of it.
Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
---
strbuf.c | 26 ++++++++++++++++++++++++++
strbuf.h | 6 ++++++
2 files changed, 32 insertions(+)
diff --git a/strbuf.c b/strbuf.c
index 05d0693..9a373be 100644
--- a/strbuf.c
+++ b/strbuf.c
@@ -425,6 +425,32 @@ void strbuf_add_lines(struct strbuf *out, const char *prefix,
strbuf_complete_line(out);
}
+void strbuf_addstr_xml_quoted(struct strbuf *buf, const char *s)
+{
+ while (*s) {
+ size_t len = strcspn(s, "\"<>&");
+ strbuf_add(buf, s, len);
+ s += len;
+ switch (*s) {
+ case '"':
+ strbuf_addstr(buf, """);
+ break;
+ case '<':
+ strbuf_addstr(buf, "<");
+ break;
+ case '>':
+ strbuf_addstr(buf, ">");
+ break;
+ case '&':
+ strbuf_addstr(buf, "&");
+ break;
+ case 0:
+ return;
+ }
+ s++;
+ }
+}
+
static int is_rfc3986_reserved(char ch)
{
switch (ch) {
diff --git a/strbuf.h b/strbuf.h
index aa386c6..ecae4e2 100644
--- a/strbuf.h
+++ b/strbuf.h
@@ -136,6 +136,12 @@ extern void strbuf_vaddf(struct strbuf *sb, const char *fmt, va_list ap);
extern void strbuf_add_lines(struct strbuf *sb, const char *prefix, const char *buf, size_t size);
+/*
+ * Append s to sb, with the characters '<', '>', '&' and '"' converted
+ * into XML entities.
+ */
+extern void strbuf_addstr_xml_quoted(struct strbuf *sb, const char *s);
+
static inline void strbuf_complete_line(struct strbuf *sb)
{
if (sb->len && sb->buf[sb->len - 1] != '\n')
--
1.8.0
^ permalink raw reply related
* [PATCH 3/8] lf_to_crlf(): NUL-terminate msg_data::data
From: Michael Haggerty @ 2012-11-25 11:08 UTC (permalink / raw)
To: Junio C Hamano; +Cc: git, Jeremy White, Johannes Schindelin, Michael Haggerty
In-Reply-To: <1353841721-16269-1-git-send-email-mhagger@alum.mit.edu>
Through the rest of the file, the data member of struct msg_data is
kept NUL-terminated, and that fact is relied upon in a couple of
places. Change lf_to_crlf() to preserve this invariant.
In fact, there are no execution paths in which lf_to_crlf() is called
and then its data member is required to be NUL-terminated, but it is
better to be consistent to prevent future confusion.
Document the invariant in the struct msg_data definition.
Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
---
imap-send.c | 7 ++++++-
1 file changed, 6 insertions(+), 1 deletion(-)
diff --git a/imap-send.c b/imap-send.c
index d42e471..c818b0c 100644
--- a/imap-send.c
+++ b/imap-send.c
@@ -69,8 +69,12 @@ struct store {
};
struct msg_data {
+ /* NUL-terminated data: */
char *data;
+
+ /* length of data (not including NUL): */
int len;
+
unsigned char flags;
};
@@ -1276,7 +1280,7 @@ static void lf_to_crlf(struct msg_data *msg)
lfnum++;
}
- new = xmalloc(msg->len + lfnum);
+ new = xmalloc(msg->len + lfnum + 1);
if (msg->data[0] == '\n') {
new[0] = '\r';
new[1] = '\n';
@@ -1297,6 +1301,7 @@ static void lf_to_crlf(struct msg_data *msg)
/* otherwise it already had CR before */
new[j++] = '\n';
}
+ new[j] = '\0';
msg->len += lfnum;
free(msg->data);
msg->data = new;
--
1.8.0
^ permalink raw reply related
* [PATCH 2/8] xml_entities(): use function strbuf_addstr_xml_quoted()
From: Michael Haggerty @ 2012-11-25 11:08 UTC (permalink / raw)
To: Junio C Hamano; +Cc: git, Jeremy White, Johannes Schindelin, Michael Haggerty
In-Reply-To: <1353841721-16269-1-git-send-email-mhagger@alum.mit.edu>
Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
---
http-push.c | 23 +----------------------
1 file changed, 1 insertion(+), 22 deletions(-)
diff --git a/http-push.c b/http-push.c
index 8701c12..9923441 100644
--- a/http-push.c
+++ b/http-push.c
@@ -172,28 +172,7 @@ enum dav_header_flag {
static char *xml_entities(const char *s)
{
struct strbuf buf = STRBUF_INIT;
- while (*s) {
- size_t len = strcspn(s, "\"<>&");
- strbuf_add(&buf, s, len);
- s += len;
- switch (*s) {
- case '"':
- strbuf_addstr(&buf, """);
- break;
- case '<':
- strbuf_addstr(&buf, "<");
- break;
- case '>':
- strbuf_addstr(&buf, ">");
- break;
- case '&':
- strbuf_addstr(&buf, "&");
- break;
- case 0:
- return strbuf_detach(&buf, NULL);
- }
- s++;
- }
+ strbuf_addstr_xml_quoted(&buf, s);
return strbuf_detach(&buf, NULL);
}
--
1.8.0
^ permalink raw reply related
* [PATCH 5/8] imap-send: correctly report errors reading from stdin
From: Michael Haggerty @ 2012-11-25 11:08 UTC (permalink / raw)
To: Junio C Hamano; +Cc: git, Jeremy White, Johannes Schindelin, Michael Haggerty
In-Reply-To: <1353841721-16269-1-git-send-email-mhagger@alum.mit.edu>
Previously, read_message() didn't distinguish between an error and eof
when reading its input. This could have resulted in incorrect
behavior if there was an error: (1) reporting "nothing to send" if no
bytes were read or (2) sending an incomplete message if some bytes
were read before the error.
Change read_message() to return -1 on ferror()s and 0 on success, so
that the caller can recognize that an error occurred. (The return
value used to be the length of the input read, which was redundant
because that is already available as the strbuf length.
Change the caller to report errors correctly.
Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
---
imap-send.c | 9 +++++++--
1 file changed, 7 insertions(+), 2 deletions(-)
diff --git a/imap-send.c b/imap-send.c
index 50e223a..86cf603 100644
--- a/imap-send.c
+++ b/imap-send.c
@@ -1398,7 +1398,7 @@ static int read_message(FILE *f, struct strbuf *all_msgs)
break;
} while (!feof(f));
- return all_msgs->len;
+ return ferror(f) ? -1 : 0;
}
static int count_messages(struct strbuf *all_msgs)
@@ -1537,7 +1537,12 @@ int main(int argc, char **argv)
}
/* read the messages */
- if (!read_message(stdin, &all_msgs)) {
+ if (read_message(stdin, &all_msgs)) {
+ fprintf(stderr, "error reading input\n");
+ return 1;
+ }
+
+ if (all_msgs.len == 0) {
fprintf(stderr, "nothing to send\n");
return 1;
}
--
1.8.0
^ permalink raw reply related
* [PATCH 4/8] imap-send: store all_msgs as a strbuf
From: Michael Haggerty @ 2012-11-25 11:08 UTC (permalink / raw)
To: Junio C Hamano; +Cc: git, Jeremy White, Johannes Schindelin, Michael Haggerty
In-Reply-To: <1353841721-16269-1-git-send-email-mhagger@alum.mit.edu>
all_msgs is only used as a glorified string, therefore there is no
reason to declare it as a struct msg_data.
Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
---
imap-send.c | 23 +++++++++--------------
1 file changed, 9 insertions(+), 14 deletions(-)
diff --git a/imap-send.c b/imap-send.c
index c818b0c..50e223a 100644
--- a/imap-send.c
+++ b/imap-send.c
@@ -1391,26 +1391,20 @@ static void wrap_in_html(struct msg_data *msg)
#define CHUNKSIZE 0x1000
-static int read_message(FILE *f, struct msg_data *msg)
+static int read_message(FILE *f, struct strbuf *all_msgs)
{
- struct strbuf buf = STRBUF_INIT;
-
- memset(msg, 0, sizeof(*msg));
-
do {
- if (strbuf_fread(&buf, CHUNKSIZE, f) <= 0)
+ if (strbuf_fread(all_msgs, CHUNKSIZE, f) <= 0)
break;
} while (!feof(f));
- msg->len = buf.len;
- msg->data = strbuf_detach(&buf, NULL);
- return msg->len;
+ return all_msgs->len;
}
-static int count_messages(struct msg_data *msg)
+static int count_messages(struct strbuf *all_msgs)
{
int count = 0;
- char *p = msg->data;
+ char *p = all_msgs->buf;
while (1) {
if (!prefixcmp(p, "From ")) {
@@ -1431,7 +1425,7 @@ static int count_messages(struct msg_data *msg)
return count;
}
-static int split_msg(struct msg_data *all_msgs, struct msg_data *msg, int *ofs)
+static int split_msg(struct strbuf *all_msgs, struct msg_data *msg, int *ofs)
{
char *p, *data;
@@ -1439,7 +1433,7 @@ static int split_msg(struct msg_data *all_msgs, struct msg_data *msg, int *ofs)
if (*ofs >= all_msgs->len)
return 0;
- data = &all_msgs->data[*ofs];
+ data = &all_msgs->buf[*ofs];
msg->len = all_msgs->len - *ofs;
if (msg->len < 5 || prefixcmp(data, "From "))
@@ -1509,7 +1503,8 @@ static int git_imap_config(const char *key, const char *val, void *cb)
int main(int argc, char **argv)
{
- struct msg_data all_msgs, msg;
+ struct strbuf all_msgs = STRBUF_INIT;
+ struct msg_data msg;
struct store *ctx = NULL;
int ofs = 0;
int r;
--
1.8.0
^ permalink raw reply related
* [PATCH 6/8] imap-send: change msg_data from storing (char *, len) to storing strbuf
From: Michael Haggerty @ 2012-11-25 11:08 UTC (permalink / raw)
To: Junio C Hamano; +Cc: git, Jeremy White, Johannes Schindelin, Michael Haggerty
In-Reply-To: <1353841721-16269-1-git-send-email-mhagger@alum.mit.edu>
struct msg_data stored (char *, len) of the data to be included in a
message, kept the character data NUL-terminated, etc., much like a
strbuf would do. So change it to use a struct strbuf. This makes the
code clearer and reduces copying a little bit.
A side effect of this change is that the memory for each message is
freed after it is used rather than leaked, though that detail is
unimportant given that imap-send is a top-level command.
--
For some reason, there is a bunch of infrastructure in this file for
dealing with IMAP flags, although there is nothing in the code that
actually allows any flags to be set. If there is no plan to add
support for flags in the future, a bunch of code could be ripped out
and "struct msg_data" could be completely replaced with strbuf.
Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
---
imap-send.c | 92 +++++++++++++++++++++++++++++++------------------------------
1 file changed, 47 insertions(+), 45 deletions(-)
diff --git a/imap-send.c b/imap-send.c
index 86cf603..a5e0e33 100644
--- a/imap-send.c
+++ b/imap-send.c
@@ -69,12 +69,7 @@ struct store {
};
struct msg_data {
- /* NUL-terminated data: */
- char *data;
-
- /* length of data (not including NUL): */
- int len;
-
+ struct strbuf data;
unsigned char flags;
};
@@ -1268,46 +1263,49 @@ static int imap_make_flags(int flags, char *buf)
return d;
}
-static void lf_to_crlf(struct msg_data *msg)
+static void lf_to_crlf(struct strbuf *msg)
{
+ size_t new_len;
char *new;
int i, j, lfnum = 0;
- if (msg->data[0] == '\n')
+ if (msg->buf[0] == '\n')
lfnum++;
for (i = 1; i < msg->len; i++) {
- if (msg->data[i - 1] != '\r' && msg->data[i] == '\n')
+ if (msg->buf[i - 1] != '\r' && msg->buf[i] == '\n')
lfnum++;
}
- new = xmalloc(msg->len + lfnum + 1);
- if (msg->data[0] == '\n') {
+ new_len = msg->len + lfnum;
+ new = xmalloc(new_len + 1);
+ if (msg->buf[0] == '\n') {
new[0] = '\r';
new[1] = '\n';
i = 1;
j = 2;
} else {
- new[0] = msg->data[0];
+ new[0] = msg->buf[0];
i = 1;
j = 1;
}
for ( ; i < msg->len; i++) {
- if (msg->data[i] != '\n') {
- new[j++] = msg->data[i];
+ if (msg->buf[i] != '\n') {
+ new[j++] = msg->buf[i];
continue;
}
- if (msg->data[i - 1] != '\r')
+ if (msg->buf[i - 1] != '\r')
new[j++] = '\r';
/* otherwise it already had CR before */
new[j++] = '\n';
}
- new[j] = '\0';
- msg->len += lfnum;
- free(msg->data);
- msg->data = new;
+ strbuf_attach(msg, new, new_len, new_len + 1);
}
-static int imap_store_msg(struct store *gctx, struct msg_data *data)
+/*
+ * Store msg to IMAP. Also detach and free the data from msg->data,
+ * leaving msg->data empty.
+ */
+static int imap_store_msg(struct store *gctx, struct msg_data *msg)
{
struct imap_store *ctx = (struct imap_store *)gctx;
struct imap *imap = ctx->imap;
@@ -1316,16 +1314,15 @@ static int imap_store_msg(struct store *gctx, struct msg_data *data)
int ret, d;
char flagstr[128];
- lf_to_crlf(data);
+ lf_to_crlf(&msg->data);
memset(&cb, 0, sizeof(cb));
- cb.dlen = data->len;
- cb.data = xmalloc(cb.dlen);
- memcpy(cb.data, data->data, data->len);
+ cb.dlen = msg->data.len;
+ cb.data = strbuf_detach(&msg->data, NULL);
d = 0;
- if (data->flags) {
- d = imap_make_flags(data->flags, flagstr);
+ if (msg->flags) {
+ d = imap_make_flags(msg->flags, flagstr);
flagstr[d++] = ' ';
}
flagstr[d] = 0;
@@ -1356,7 +1353,8 @@ static void encode_html_chars(struct strbuf *p)
strbuf_splice(p, i, 1, """, 6);
}
}
-static void wrap_in_html(struct msg_data *msg)
+
+static void wrap_in_html(struct strbuf *msg)
{
struct strbuf buf = STRBUF_INIT;
struct strbuf **lines;
@@ -1366,9 +1364,7 @@ static void wrap_in_html(struct msg_data *msg)
static char *pre_close = "</pre>\n";
int added_header = 0;
- strbuf_attach(&buf, msg->data, msg->len, msg->len);
- lines = strbuf_split(&buf, '\n');
- strbuf_release(&buf);
+ lines = strbuf_split(msg, '\n');
for (p = lines; *p; p++) {
if (! added_header) {
if ((*p)->len == 1 && *((*p)->buf) == '\n') {
@@ -1385,8 +1381,8 @@ static void wrap_in_html(struct msg_data *msg)
}
strbuf_addstr(&buf, pre_close);
strbuf_list_free(lines);
- msg->len = buf.len;
- msg->data = strbuf_detach(&buf, NULL);
+ strbuf_release(msg);
+ *msg = buf;
}
#define CHUNKSIZE 0x1000
@@ -1425,34 +1421,39 @@ static int count_messages(struct strbuf *all_msgs)
return count;
}
-static int split_msg(struct strbuf *all_msgs, struct msg_data *msg, int *ofs)
+/*
+ * Copy the next message from all_msgs, starting at offset *ofs, to
+ * msg. Update *ofs to the start of the following message. Return
+ * true iff a message was successfully copied.
+ */
+static int split_msg(struct strbuf *all_msgs, struct strbuf *msg, int *ofs)
{
char *p, *data;
+ size_t len;
- memset(msg, 0, sizeof *msg);
if (*ofs >= all_msgs->len)
return 0;
data = &all_msgs->buf[*ofs];
- msg->len = all_msgs->len - *ofs;
+ len = all_msgs->len - *ofs;
- if (msg->len < 5 || prefixcmp(data, "From "))
+ if (len < 5 || prefixcmp(data, "From "))
return 0;
p = strchr(data, '\n');
if (p) {
- p = &p[1];
- msg->len -= p-data;
- *ofs += p-data;
+ p++;
+ len -= p - data;
+ *ofs += p - data;
data = p;
}
p = strstr(data, "\nFrom ");
if (p)
- msg->len = &p[1] - data;
+ len = &p[1] - data;
- msg->data = xmemdupz(data, msg->len);
- *ofs += msg->len;
+ strbuf_add(msg, data, len);
+ *ofs += len;
return 1;
}
@@ -1504,7 +1505,7 @@ static int git_imap_config(const char *key, const char *val, void *cb)
int main(int argc, char **argv)
{
struct strbuf all_msgs = STRBUF_INIT;
- struct msg_data msg;
+ struct msg_data msg = {STRBUF_INIT, 0};
struct store *ctx = NULL;
int ofs = 0;
int r;
@@ -1564,11 +1565,12 @@ int main(int argc, char **argv)
ctx->name = imap_folder;
while (1) {
unsigned percent = n * 100 / total;
+
fprintf(stderr, "%4u%% (%d/%d) done\r", percent, n, total);
- if (!split_msg(&all_msgs, &msg, &ofs))
+ if (!split_msg(&all_msgs, &msg.data, &ofs))
break;
if (server.use_html)
- wrap_in_html(&msg);
+ wrap_in_html(&msg.data);
r = imap_store_msg(ctx, &msg);
if (r != DRV_OK)
break;
--
1.8.0
^ permalink raw reply related
* [PATCH 7/8] wrap_in_html(): use strbuf_addstr_xml_quoted()
From: Michael Haggerty @ 2012-11-25 11:08 UTC (permalink / raw)
To: Junio C Hamano; +Cc: git, Jeremy White, Johannes Schindelin, Michael Haggerty
In-Reply-To: <1353841721-16269-1-git-send-email-mhagger@alum.mit.edu>
Use the new function to quote characters as they are being added to
buf, rather than quoting them in *p and then copying them into buf.
This increases code sharing, and changes the algorithm from O(N^2) to
O(N) in the number of characters in a line.
Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
---
imap-send.c | 23 ++++-------------------
1 file changed, 4 insertions(+), 19 deletions(-)
diff --git a/imap-send.c b/imap-send.c
index a5e0e33..b73c913 100644
--- a/imap-send.c
+++ b/imap-send.c
@@ -1339,21 +1339,6 @@ static int imap_store_msg(struct store *gctx, struct msg_data *msg)
return DRV_OK;
}
-static void encode_html_chars(struct strbuf *p)
-{
- int i;
- for (i = 0; i < p->len; i++) {
- if (p->buf[i] == '&')
- strbuf_splice(p, i, 1, "&", 5);
- if (p->buf[i] == '<')
- strbuf_splice(p, i, 1, "<", 4);
- if (p->buf[i] == '>')
- strbuf_splice(p, i, 1, ">", 4);
- if (p->buf[i] == '"')
- strbuf_splice(p, i, 1, """, 6);
- }
-}
-
static void wrap_in_html(struct strbuf *msg)
{
struct strbuf buf = STRBUF_INIT;
@@ -1372,12 +1357,12 @@ static void wrap_in_html(struct strbuf *msg)
strbuf_addbuf(&buf, *p);
strbuf_addstr(&buf, pre_open);
added_header = 1;
- continue;
+ } else {
+ strbuf_addbuf(&buf, *p);
}
+ } else {
+ strbuf_addstr_xml_quoted(&buf, (*p)->buf);
}
- else
- encode_html_chars(*p);
- strbuf_addbuf(&buf, *p);
}
strbuf_addstr(&buf, pre_close);
strbuf_list_free(lines);
--
1.8.0
^ permalink raw reply related
* Re: Python extension commands in git - request for policy change
From: Felipe Contreras @ 2012-11-25 11:19 UTC (permalink / raw)
To: esr; +Cc: git
In-Reply-To: <20121125095356.GA22279@thyrsus.com>
On Sun, Nov 25, 2012 at 10:53 AM, Eric S. Raymond <esr@thyrsus.com> wrote:
> Felipe Contreras <felipe.contreras@gmail.com>:
>> If your friends jump off a bridge, would you? Yes, using python has
>> served them well, but as opposed to what? Other scripting languages? I
>> don't think so.
>
> The competition that Python won was *precisely* against other scripting
> languages, notably shell and Perl. Both used to be much more heavily
> used in system scripting than they are now.
Against shell and perl yes, not against the rest.
>> What if my extension only supports python 2.7? Or what if my extension
>> wants to support 2.0?
>
> I propose that if 2.6 can't support it, then that should be considered
> grounds to reject it.
Seems sensible, but I don't know what "rejection" would actually mean.
My "extensions" are on the way to the contrib area. Is the contrib
area supposed to have different rules? I don't know.
Either way, making a script work on python 2.6 is probably easier than
trying to "reject" it.
>> Yes, they should _if_ they know what version they need. In my
>> extensions I really have no idea.
>
> Then you shouldn't submit those extensions to be folded into core git.
Too late.
>> > 3) We should be unconditionally be encouraging extensions to move
>> > from shell and Perl to Python. This would be a clear net gain is
>> > portability and maintainability.
>>
>> NO! It's up to the developer to choose what language to use,
>
> I agree. You seem to be raising a lot of straw men. 'Encouragement'
> does not equate to beating anyone who makes an unpopular choice over
> the head.
I don't see what this means in practical terms. People are going to
write code in whatever language they want to write code in. How
exactly are "we" going to "encourage" them not to do that is not
entirely clear to me.
I don't think there's such a thing as "git leadership" that would be
able to take these policy decisions, and if there was one, I don't
think the evidence presented would be enough to weigh in either way.
> I am also not suggesting that the whole git core ought to be hoicked
> over to Python. I was thinking mainly about extension subcommands,
> not what's in libgit now.
Subcommands are also probably more efficient in c. And lets remember
that most people use git through the *official* subcommands.
Cheers.
--
Felipe Contreras
^ permalink raw reply
* Re: Python extension commands in git - request for policy change
From: Nguyen Thai Ngoc Duy @ 2012-11-25 11:25 UTC (permalink / raw)
To: Michael Haggerty; +Cc: Felipe Contreras, Eric S. Raymond, git
In-Reply-To: <50B1F684.5020805@alum.mit.edu>
On Sun, Nov 25, 2012 at 5:44 PM, Michael Haggerty <mhagger@alum.mit.edu> wrote:
> On the contrary, there is *constant* traffic on the mailing list about
> incompatibilities between different shell implementations (sh, dash,
> bash, etc), not to mention those in other utilities (sed, grep, etc)
> that one is forced to work with in shell scripts. Compatibility is a
> *huge* pain when developing shell code for git. The fact that users
> typically don't encounter such problems is due to the hard work of POSIX
> lawyers on the mailing list correcting the compatibility errors of
> mortal programmers.
I think we still are in the process of moving away from shell-based
commands (not the shell interface), just not enough man power to do it
fast. The only shell-based command with active development is
git-submodule. So most shell PITA is in the test suite.
> The most important issues to consider when imagining a future with a
> hybrid of code in C and some scripting language "X" are:
>
> * Portability: is "X" available on all platforms targeted by git, in
> usable and mutually-compatible versions?
>
> * Startup time: Is the time to start the "X" interpreter prohibitive?
> (On my computer, "python -c pass", which starts the Python
> interpreter and does nothing, takes about 24ms.) This overhead would
> be incurred by every command that is not pure C.
>
> * Should the scripting language access the C functionality only by
> calling pure-C executables or by dynamically or statically linking to
> a binary module interface? If the former, then the granularity of
> interactions between "X" and C is necessarily coarse, and "X" cannot
> be used to implement anything but the outermost layer of
> functionality. If the latter, then the way would be clear to
> implement much more of git in "X" (and lua would also be worth
> considering).
>
> * Learning curve for developers: how difficult is it for a typical git
> developer to become conversant with "X", considering both (1) how
> likely is it that the typical git developer already knows "X" and
> (2) how straightforward and predictable is the language "X"?
> In this category I think that Python has a huge advantage over
> Perl, though certainly opinions will differ and Ruby would also be
> a contender.
* We might also need an embedded language variant, like Jeff's lua
experiment. I'd be nice if "X" can also take this role.
--
Duy
^ permalink raw reply
* Re: Python extension commands in git - request for policy change
From: Felipe Contreras @ 2012-11-25 11:40 UTC (permalink / raw)
To: Michael Haggerty; +Cc: Eric S. Raymond, git
In-Reply-To: <50B1F684.5020805@alum.mit.edu>
On Sun, Nov 25, 2012 at 11:44 AM, Michael Haggerty <mhagger@alum.mit.edu> wrote:
> On 11/25/2012 09:53 AM, Felipe Contreras wrote:
>> On Sun, Nov 25, 2012 at 3:44 AM, Eric S. Raymond <esr@thyrsus.com> wrote:
>>> 1) In 2012, we can specify a "floor" Python version of 2.6 (shipped in
>>> 2008) and be pretty much guaranteed it will be anywhere we want to
>>> deploy except Windows. Windows will remain a problem because Python
>>> isn't part of the stock install, but that's an equal or worse problem
>>> for shell and Perl - and at least the Python project ships a binary
>>> installer for Windows.
>>
>> What if my extension only supports python 2.7? Or what if my extension
>> wants to support 2.0?
>
> There would obviously have to be a policy like "all Python code in core
> git must run on any Python interpreter with 2.6 <= version < 3.0", just
> as there are policies about what C and shell features are allowed. If
> you happen to want to support earlier versions of Python, I don't see
> why anybody would stop you as long as your code also runs in the
> mandated versions.
Of course, but there are experts in C and shell around, not so many
python experts. So if somebody sneaks in a python program that makes
use of features specific to python 2.7, I doubt anybody would notice.
And if they did, I doubt that would be reason enough for rejection,
supposing that porting to 2.6 would be difficult enough.
Anyway, I think this is all guesswork.
> Of course Perl will have the same problem if Perl6 ever materializes.
It *might*, it might not be as severe.
>>> 2) Python extension commands should test the Python version on startup
>>> and die loudly but gracefully in the rare case that they don't find
>>> what they need.
>>
>> Yes, they should _if_ they know what version they need. In my
>> extensions I really have no idea.
>
> Then simply (with the help of the mailing list) ensure that your
> extensions run under 2.6 (or whatever the chosen minimum version is) and
> everything will be OK. It is not an error to specify 2.6 as the minimum
> version even though your script happens also to run on older versions :-)
Who would do that? I don't see a lot of people.
>>> 3) We should be unconditionally be encouraging extensions to move
>>> from shell and Perl to Python. This would be a clear net gain is
>>> portability and maintainability.
>>
>> NO! It's up to the developer to choose what language to use, and I
>> find it very chauvinist of you to say "python is better, so let's all
>> use python". So far you have listed a few advantages of python, but
>> you haven't explained so far what is wrong with shell and perl.
>
> Given that some languages are accepted in git-core and others are not,
> it's already not "up to the developer to choose what language to use".
> At best there is a short list of "blessed" languages, and the developer
> can choose among only those.
They are not because they haven't been proposed. Things change.
>> In fact, while advancing python you have made clear a problem with
>> python; the version requirements. So far I have *never* encountered a
>> problem with git because of my bash version, or my perl version. And
>> we haven't touched to the python3 mess yet. To me, those are
>> advantages of shell and perl.
>
> On the contrary, there is *constant* traffic on the mailing list about
> incompatibilities between different shell implementations (sh, dash,
> bash, etc), not to mention those in other utilities (sed, grep, etc)
> that one is forced to work with in shell scripts. Compatibility is a
> *huge* pain when developing shell code for git. The fact that users
> typically don't encounter such problems is due to the hard work of POSIX
> lawyers on the mailing list correcting the compatibility errors of
> mortal programmers.
*Theoretical* incompatibilities on probably obscure systems. *I* have
never seen such compatibility issues *in practice*.
>> Actually, I don't care if 'git foo' is written in perl, or shell, or
>> c; as long as it *works*. And I would hate it if 'git rebase' ever
>> told me that I need a newer version of python, or worst; that I don't
>> have python in my system (Arch Linux ships 'python2', not 'python').
>
> The configure script would locate the correct interpreter and the build
> would adjust the scripts' shebang lines, just as things are tweaked
> within Perl scripts at build time.
Arch Linux doesn't use no configure script. And what if I'm building
git myself (I've hit the issue multiple times)? Perl might have
similar issues on other systems, but not on Arch Linux; /usr/bin/perl
is there.
>>> 4) We should be encouraging C code to move to Python, too. There's
>>> little gain in portability on this path because modern C has cleaned
>>> up its act a lot, but the drop in expected bug loads would be well
>>> worth the porting effort. Segfaults are not your friend, and the x2 to
>>> x5 drop in line count would do very good things for long-term
>>> maintainability.
>>
>> Definitely NO! I really really doubt git in python would be able to
>> achieve the same performance as git in c, but to show me wrong, it
>> wouldn't be very difficult to run a few measurements with python
>> dulwich *if* we are even to begin considering this point.
>>
>> And are segmentation faults really that different from python's
>> exceptions? Not to the user.
>
> There is one huge difference: it C it is all too easy to write code that
> leads to a security hole due to buffer overflows and other memory
> management errors. Code written in a scripting language are largely
> immune to such problems (except of course for any such bugs in the
> interpreter itself, but the testing of the interpreter is shared across
> many projects and users).
>
> It would be insane to rewrite performance-critical C code in any
> scripting language, but there is a huge penumbra of code that is not
> performance critical and that mutates rapidly. Such code is much easier
> to write and maintain in a sane scripting language if the portability
> issues can be mastered.
I think git developers are perfectly able to write such a code.
> The most important issues to consider when imagining a future with a
> hybrid of code in C and some scripting language "X" are:
>
> * Portability: is "X" available on all platforms targeted by git, in
> usable and mutually-compatible versions?
>
> * Startup time: Is the time to start the "X" interpreter prohibitive?
> (On my computer, "python -c pass", which starts the Python
> interpreter and does nothing, takes about 24ms.) This overhead would
> be incurred by every command that is not pure C.
Agree.
> * Should the scripting language access the C functionality only by
> calling pure-C executables or by dynamically or statically linking to
> a binary module interface? If the former, then the granularity of
> interactions between "X" and C is necessarily coarse, and "X" cannot
> be used to implement anything but the outermost layer of
> functionality. If the latter, then the way would be clear to
> implement much more of git in "X" (and lua would also be worth
> considering).
I think this is very far fetched at the moment. Proposals such as
libgit2 are moving things forward, but we are pretty far from a goal
like that.
> * Learning curve for developers: how difficult is it for a typical git
> developer to become conversant with "X", considering both (1) how
> likely is it that the typical git developer already knows "X" and
> (2) how straightforward and predictable is the language "X"?
> In this category I think that Python has a huge advantage over
> Perl, though certainly opinions will differ and Ruby would also be
> a contender.
Right, but I have the feeling that most git developers are perfectly
familiar with C already. In order to move to something else, and all
the necessary burden of learning, or becoming more familiar with X, a
compelling argument must be put forward, and I haven't seen such an
argument.
> Personally, I regret wasting my time programming pointer arithmetic in
> git modules that are not performance-critical (and correcting bugs by
> others in these areas). And I'm tired of having an idea to improve a
> git feature only to find that it is implemented in shell, where not even
> arrays are available. I would therefore welcome more friendliness
> towards a decent scripting language in the git project.
Me too, if ruby was one of them.
Cheers.
--
Felipe Contreras
^ permalink raw reply
* Re: Python extension commands in git - request for policy change
From: Felipe Contreras @ 2012-11-25 11:48 UTC (permalink / raw)
To: esr; +Cc: Nguyen Thai Ngoc Duy, git, msysGit
In-Reply-To: <20121125095429.GB22279@thyrsus.com>
On Sun, Nov 25, 2012 at 10:54 AM, Eric S. Raymond <esr@thyrsus.com> wrote:
> Felipe Contreras <felipe.contreras@gmail.com>:
>> On Sun, Nov 25, 2012 at 6:18 AM, Eric S. Raymond <esr@thyrsus.com> wrote:
>> > Nguyen Thai Ngoc Duy <pclouds@gmail.com>:
>> >> These may apply to other languages as well. Where do we draw a line?
>> >
>> > I'm in favor of the general policy of avoiding scripting languages
>> > other than the top three most widely deployed. At the moment that
>> > means shell, Python, Perl; on present trends, in a few years Perl
>> > (dropping in popularity) might be passed by Ruby on the way up.
>>
>> Top three according to whom?
>
> According to the LOC counts in git's codebase.
Not according to ohloh:
1) shell 33%
2) tcl 9%
3) perl 9.7%
4) python 1.8%
And this is a non-sequitur; you are proposing to change git policies
based on numbers that are a direct result of git's policies?
https://www.ohloh.net/p/git/analyses/latest/languages_summary
--
Felipe Contreras
^ permalink raw reply
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox