Mercurial's only true "plugin" extension: inotify... and can it be done in Git?

Mercurial's only true "plugin" extension: inotify... and can it be done in Git?

[Jakub Narebski]

When reading "Distributed Revision Control with Mercurial"[1] to see
how Mercurial differs from Git, in the chapter 14 "Adding functionality 
with extensions" there is description (in section 14.1) of one 
interesting extension (see also [2]), namely 'inotify' extension. 

This extension[3][4] optimises "hg status" command by asking file 
notification daemon about changed files instead of doing stat. I'm not 
sure how useful this extension is: inotify is Linux specific, and Linux 
has fast stat... nevertheless this is one example where extension 
(plug-in) framework shows that it can do more than good scriptability.

I wonder if, and how, such 'extension' could be done with Git...

References:
===========
[1] http://hgbook.red-bean.com
[2] http://www.selenic.com/mercurial/wiki/index.cgi/UsingExtensions
[3] http://hgbook.red-bean.com/hgbookch14.html#x18-31700014.1
[4] http://www.selenic.com/mercurial/wiki/index.cgi/InotifyExtension
-- 
Jakub Narebski
Poland

Re: Mercurial's only true "plugin" extension: inotify... and can it be done in Git?

[Jakub Narebski]

On Sun, 9 Mar 2008, Jakub Narebski wrote:

> When reading "Distributed Revision Control with Mercurial" to
> see how Mercurial differs from Git, in the chapter 14 "Adding
> functionality  with extensions" there is description
> (in section 14.1) of one  interesting extension,
> namely 'inotify' extension. 

By the way, when browsing list of available Mercurial extensions
http://www.selenic.com/mercurial/wiki/index.cgi/UsingExtensions
inotify extension was single extension which doesn't have less
or more direct equivalent in Git, or cannot be easy done in Git.

See the table below for details:

Mercurial extension | git equivalent (or proposal)
--------------------+------------------------------------
acl                 | update-paranoid contrib hook
alias               | alias.<cmd> config
bisect              | git-bisect
bugzilla            | possible as hook
children            | possible command, cf. git-name-rev
convert (!)         | <possible, would be nice to have>
extdiff             | diff.external / GIT_EXTERNAL_DIFF
fetch               | git-pull (sic!)
gpg                 | signed tags
graphlog            | <possible, see: tig, git-forest>
hgk (*)             | gitk
imerge              | see git-mergetool
keyword expansion   | check possible gitattributes
mq                  | Guilt, StGit
notify              | post-receive-mail contrib hook,
                    | default update hook
graphviz            | <can be done>
send                | <possible, wrapper around git-bundle>
rdiff (!)           | <a trick for local repo in GitTips>
localbranch (*)     | git-branch, git-checkout
parentRevspec (*)   | rev^, rev^1 or rev^2, rev~n
patchbomb           | git-send-email
record              | git add -i, git commit -i
transplant          | git-rebase
win32ext            | core.autocrlf config
churn               | see git-shortlog
purge               | git-clean
cvscommit           | see git-cvsexportcommit, git-cvsserver
config              | compare git-config
digest              | see git-ls-remote for bundle
diffstat            | git diff --diffstat, default for git-pull
easy*               | see tig
forest              | git-submodule
hgcia               | CIA hook mentioned on GitWiki
inotify (!!!)       |

Footnotes:
==========
(*) Directly inspired by git feature
(!) No direct equivalent, would be quite a work to add
(!!!) No equivalent, don't know if possible

-- 
Jakub Narebski
Poland

Re: Mercurial's only true "plugin" extension: inotify... and can it be done in Git?

[Geert Bosch]

On Mar 9, 2008, at 09:59, Jakub Narebski wrote:

> This extension[3][4] optimises "hg status" command by asking file
> notification daemon about changed files instead of doing stat. I'm not
> sure how useful this extension is: inotify is Linux specific, and  
> Linux
> has fast stat... nevertheless this is one example where extension
> (plug-in) framework shows that it can do more than good scriptability.

FWIW, OS X 10.5 has a similar functionality, called "fsevents",
as well as a much slower stat.

   -Geert

Re: Mercurial's only true "plugin" extension: inotify... and can it be done in Git?

[Shawn O. Pearce]

Geert Bosch <bosch@adacore.com> wrote:
> On Mar 9, 2008, at 09:59, Jakub Narebski wrote:
> 
> >This extension[3][4] optimises "hg status" command by asking file
> >notification daemon about changed files instead of doing stat. I'm not
> >sure how useful this extension is: inotify is Linux specific, and  
> >Linux
> >has fast stat... nevertheless this is one example where extension
> >(plug-in) framework shows that it can do more than good scriptability.
> 
> FWIW, OS X 10.5 has a similar functionality, called "fsevents",
> as well as a much slower stat.

Windows NT on NTFS has a file monitor system that works somewhat
like an inotify, but different enough that you gotta write totally
different code for it.  It also has a much slower stat.

I have wanted to put something like this into git-gui, so that a
running git-gui session can see changes made to your working dir
"live".  But I haven't gotten around to writing it.  I did do some
early experiments on Windows NT + NTFS with a small C daemon that
watches the working directory, but it ran into issues about also
need to know what is inside .gitignore to properly avoid sending
events that git-gui doesn't want to know about.  Like say build
object files.  :-)

I think the reason Git has never had such an "extension" is we are
just that freaking fast when it comes to comparing the working tree
to the index.  hg wasn't in the early days and that may have been
what gave rise to the plugin.

-- 
Shawn.

Re: Mercurial's only true "plugin" extension: inotify... and can it be done in Git?

[J. Bruce Fields]

On Mon, Mar 10, 2008 at 06:43:18PM -0400, Shawn O. Pearce wrote:
> I think the reason Git has never had such an "extension" is we are
> just that freaking fast when it comes to comparing the working tree
> to the index.

NFS is an exception there; a git diff takes about 12 seconds for me.
It's easy to see why--ping time to the server is about a quarter of a
millisecond, so with some 24000 files to stat, and a roundtrip to the
server required for each, it would take at least 6 seconds even if there
was no other contribution to the stat latency.

But of course there's not much by way of file change notifications in
the current NFS protocol either, so that doesn't help.

At some point I tried patching git to do the stat's in parallel (the
totally lazy way--instead of modifying the existing code I just did an
extra parallel stat of everything first, then threw the results away, so
the real work would be done with a hot cache on the client).  I could
get it down to about 3.5 seconds that way, better though still annoying.

--b.

Re: Mercurial's only true "plugin" extension: inotify... and can it be done in Git?

[Andreas Ericsson]

Jakub Narebski wrote:
> 
> Mercurial extension | git equivalent (or proposal)
> --------------------+------------------------------------
> bisect              | git-bisect
> 
> Footnotes:
> ==========
> (*) Directly inspired by git feature
> (!) No direct equivalent, would be quite a work to add
> (!!!) No equivalent, don't know if possible
> 

IIRC, bisect should have a star here (or five, as it's a killer feature).

-- 
Andreas Ericsson                   andreas.ericsson@op5.se
OP5 AB                             www.op5.se
Tel: +46 8-230225                  Fax: +46 8-230231

Re: Mercurial's only true "plugin" extension: inotify... and can it be done in Git?

[Junio C Hamano]

Andreas Ericsson <ae@op5.se> writes:

> Jakub Narebski wrote:
>>
>> Mercurial extension | git equivalent (or proposal)
>> --------------------+------------------------------------
>> bisect              | git-bisect
>>
>> Footnotes:
>> ==========
>> (*) Directly inspired by git feature
>> (!) No direct equivalent, would be quite a work to add
>> (!!!) No equivalent, don't know if possible
>>
>
> IIRC, bisect should have a star here (or five, as it's a killer feature).

Heh, why do we even care?  It is a norm that good features cross pollinate
across systems and who borrows ideas from whom does not really matter
much.

Re: Mercurial's only true "plugin" extension: inotify... and can it be done in Git?

[Jakub Narebski]

Andreas Ericsson wrote:
> Jakub Narebski wrote:
> > 
> > Mercurial extension | git equivalent (or proposal)
> > --------------------+------------------------------------
> > bisect              | git-bisect
> > 
> > Footnotes:
> > ==========
> > (*) Directly inspired by git feature
> > (!) No direct equivalent, would be quite a work to add
> > (!!!) No equivalent, don't know if possible
> > 
> 
> IIRC, bisect should have a star here (or five, as it's a killer
> feature). 

I was not sure if bisect extension didn't come to Mercurial from/via  
BitKeeper; IIRC it was added at request of Linux kernel, but I might be 
mistaken...

So in short: the star isn't here because I was not sure.

-- 
Jakub Narebski
Poland

Re: Mercurial's only true "plugin" extension: inotify... and can it be done in Git?

[Andreas Ericsson]

Junio C Hamano wrote:
> Andreas Ericsson <ae@op5.se> writes:
> 
>> Jakub Narebski wrote:
>>> Mercurial extension | git equivalent (or proposal)
>>> --------------------+------------------------------------
>>> bisect              | git-bisect
>>>
>>> Footnotes:
>>> ==========
>>> (*) Directly inspired by git feature
>>> (!) No direct equivalent, would be quite a work to add
>>> (!!!) No equivalent, don't know if possible
>>>
>> IIRC, bisect should have a star here (or five, as it's a killer feature).
> 
> Heh, why do we even care?  It is a norm that good features cross pollinate
> across systems and who borrows ideas from whom does not really matter
> much.

Because companies such as the one where I work pride themselves in using
the most innovative tools. Clearly, git is the most innovative tool here,
since its developers managed to cook up something so immensely useful as
"git bisect" (which by the way is well-nigh single-handedly responsible
for reducing our average bugreport-to-fix time from 4 days to 6 hours).

-- 
Andreas Ericsson                   andreas.ericsson@op5.se
OP5 AB                             www.op5.se
Tel: +46 8-230225                  Fax: +46 8-230231

Re: Mercurial's only true "plugin" extension: inotify... and can it be done in Git?

[Jakub Narebski]

[Cc: git mailing list, mercurial mailing list]

On Thu, 13 March 2008, Zdeněk Vráblík wrote:
> On Mon, Mar 10, 2008, Jakub Narębski <jnareb@gmail.com> wrote:
>>
> ...
>>  cvscommit           | see git-cvsexportcommit, git-cvsserver

First, a word of warning. I use Git (and sometimes even contribute
to it), and watch its mailing list; I know Mercurial only from 
documentation and a few discussions on #revctrl IRC channel. So now you 
should know which way I tend to be biased ;-)
 
> I have looked at git-cvsserver
> http://www.kernel.org/pub/software/scm/git/docs/git-cvsserver.html
> It doesn't put back changes from git to cvs server.
> It uses cvs client to work with GIT repository.

On http://www.selenic.com/mercurial/wiki/index.cgi/UsingExtensions

   cvscommit            | Push changesets to CVS

But on the link on its wiki page we have

  From: Simon 'corecode' Schubert
  Imbeault, Jean-Christian wrote:
  >
  > Are there any tools available that can push an hg repository
  > into CVS? 
  I've written the cvscommit extension, but it might be outdated and
  maybe could use some structural changes, but I've used it for some
  commits.

There is no documentation for this extension, unfortunately, so I don't 
know what exactly it does.


From listed git "equivalents"

  git-cvsexportcommit  (1)  - Export a single commit to a CVS checkout
  git-cvsserver        (1)  - A CVS server emulator for git

it looks like git-cvsexportcommit is more equivalent to hg.ext.cvscommit
Mercurial extension.

> Does Mercurial has something like this? I use Mercurial personally and
> I like it. But to migrate from cvs I need persuade all team. It will
> be easier to do it with such cvs emulator.

I don't know of such extension / tool

> Thanks for the comparison.

You are welcome.
-- 
Jakub Narebski
Poland