How it was at GitTogether'08 ?

How it was at GitTogether'08 ?

[Jakub Narebski]

GitTogether 2008, which took place October 27(Mon)-29(Wed), has ended more
than week ago.  Therefore I'd like to ask you impressions (while it is still
fresh) from GitTogether, describe talks which are not described below,
correct wrong information in below, etc.

Please reply to this email while GitTogether is on front of your mind...


Talks at GitTogether 2008
=========================

Mon, Oct 27, 2008
-----------------
* Dscho: Contributing with Git
  http://www.youtube.com/watch?v=j45cs5_nY2k

* Junio: Git Chronicle

  blog: Junio went though a sort of statistical history of the Git project
  that was fascinating (turns out there are still about 220 lines of code
  still around from Linus original first commit).

  http://userweb.kernel.org/~junio/200810-Chron.pdf

* Jeff: Helping new contributors join

* Petr: Renames Again and Again and Again

  IRC: detection of wholesame renames of directories (WIP) and '--follow'
  limitation were mentioned, but outcome is unclear; pasky plans to hack
  together some patch implementing explicit renames hinting

* Sam: GitTorrent

  IRC: briefly the history if GitTorrent: way underannounced, not completely
  finished, GTP/0.1 was "published" long ago. There was a GSoC project this
  year, mentored by mugwump. Unfortunately, the student turned out to be a
  procrastinator or something, and this project was mainly pushed forward by
  the mentor. Now after the summer, mugwump took a step back and analyzed
  the protocol with all the experience won from implementing a large part of
  it. Turns out that a lot of the stuff was not really necessary, but copied
  because the BitTorrent protocol had it all.  So, plan is to really strip
  down GTP.  And not have a separate protocol for GitTorrent, but rather
  have it as part of git-daemon. [...] Turns out that warthog9 had a few
  really important things to say that should end up (at least in the 2nd
  version) in having a more robust discovery/exchange.

  http://utsl.gen.nz/talks/gittorrent/start.html

* Tom: GitHub

  IRC: a tour, some history, and insight into how it works; some nice
  gimmicks, such as "gist" (a git backed pastebin), or like network graphs
  (look graphically at forks of a repository).

  blog: Tom got to demonstrate GitHub and Gist to the group, most of whom
  are very command line oriented and had not used either before.

* Shawn: JGit

* Scott: Linkable Library

  blog: got to talk about the need for a linkable git library

  http://thread.gmane.org/gmane.comp.version-control.git/99608

* Sam: perl.git

  blog: Sam demonstrated the work he went through to import 20 years of Perl
  history into the git repository that the Perl team is just now finishing
  transitioning to from Perforce.

  http://utsl.gen.nz/talks/perl-history/slides/


Tue, Oct 28, 2008
-----------------
* Shawn: Pack v4

  IRC: Shawn gave a pretty good summary of packv4, and he said what are the
  major blocks: we need a refactoring of the tree walker first, and then it
  would add a pretty complex heap of code to make use of the packv4 format
  for tree walking.  Main concern was _not_ the size of pack (for spearce at
  least), but the speed of rev-list parsing with path delimiters [...] The
  tree walking needs a different interface. (Path-delimiting must be a part
  of it, not an afterthought. Mode filtering must be a part of it, not an
  afterthought.)  Estimations for packv4: 10% decrease in size, 25% increase
  in speed.  [...] reworking packv4 to do it for OBJ_BLOB too (and the
  obvious path to speed-optimize git grep -w) has not been touched at all.

* Jeff: What needs refactoring?

* David Brown: Life with Git
  http://www.davidb.org/git/git-corp.pdf

* Lighting Round Talks
  - Scott: iGitHub - git daemon and repository browser on the iPhone
  - Jeff: Portability autobuilders
  - Petr: Quick TopGit introduction, problems, future plans

  IRC: iGitHub - its a tiny app you run on your iphone to make your iphone
  act as a git server/client. since apple doesn't let you tether your laptop
  through your iphone to get wireless internet you can instead push a
  repository to the iphone via the laptop, then push from there to the
  world, or fetch from the world to the iphone, then from the iphone to the
  laptop. it also has a small history browser and stuff built in.

* Jeffrey Altman: Discussion with OpenAFS

  IRC: was about them trying to convert their franken-CVS repo to
  Git. Basically, they use CVS, but have a layer on top of it that logically
  combines two related commits into a single "delta", and they want to
  preseve those as single git commits to the extent possible (sometimes it
  is not possible because of conflicts). They are looking for ways to script
  the conversion.

  The OpenAFS guys were also interested in helping with libgit and windows
  TortoiseGit work (separate conversation).

* Tim: Git as a Media Repository
  http://www.thousandparsec.net/~tim/media+git.pdf

* Petr: Git UI

  IRC: pasky talked a little bit about what he still misses in git.git CLI:
  commit -m piping through fmt; the "reset and checkout" vs "switch, reset
  and restore" paradigm; add vs stage; a bit about submodules [...] and
  spent most of the time in a bikeshed argument about
  add/rm/stage/unstage/staged/revert/restore/switch/. [...] Basically, reset
  and checkout sucks and has chaotic behaviour with various switches and
  argument combinations (files given vs not given etc.) So in ideal world,
  reset and checkout might've been deprecated and instead, switch, restore
  and unstage would be introduced... but that's not gonna happen.

  blog: The other important, highly visual thing that was discussed, and
  even a few patches are already in for, is for little improvements to the
  UI. The full planning document is on Gist, but already things like making
  use of the term 'stage' for things that happen in the index (such as using
  "git diff --staged" instead of "git diff --cached") is being worked
  on. I'm excited that staging files may soon be done via "git stage"
  rather-than/in-addition-to "git add". This is nice for new users who often
  have a hard time seeing why you have to keep "git add"ing to stage your
  changes.

  http://pasky.or.cz/~pasky/cp/gittogether-ui.sxi
  http://gist.github.com/20818

  http://thread.gmane.org/gmane.comp.version-control.git/99572
  http://thread.gmane.org/gmane.comp.version-control.git/99541

* Steven: Life with git-svn


Wed, Oct 29, 2008
-----------------
* Shawn: Bundle Project

  IRC: Gerrit is issue tracker / code review tool, which takes full bundles
  and moves them around, and only when it was reviewed, it is actually
  applied.

  http://android.git.kernel.org/?p=tools/gerrit.git

* Discussion on notes

* Petr: introduce the secret pickaxe project
  http://repo.or.cz/w/giddy.git

* Petr: Git in Pharma Corp
  http://pasky.or.cz/~pasky/cp/gittogether-novartis.sxi
  http://pasky.or.cz/g/gitweb.cgi?p=templatedemo.git;a=summary

* Tim: Large media in Git (Repeat)

  IRC: Large = 100M and more (e.g. audio or video assets for a game). the
  idea was that you would have shallow _and_ lazy clones. There was not much
  discussion about implementation details, though; except that there should
  be a mode where blobs are _not_ fetched, and that every non-blob has to be
  local.

  IRC: Tim Ansell (mithro) of Thousand Parsec still uses SVN for this
  because he can checkout just one file, edit it, and push it
  back. Basically he wants to clone the metadata (commits and trees and
  small blobs) and have large blobs (e.g. >25 MB or some limit) kept on the
  origin, lazily fetched over a git transport on demand. Tim started to work
  up a prototype using HTTP. folks thought the idea showed promise, but we
  need to see code and the impacts it has and where it breaks down to really
  understand if it is worth doing or not.

  blog: Tim talked about something that I think will be one of the next huge
  (highly visible) changes in Git you\u2019re likely to see in the next year
  \u2013 handling large meda well, and being able to do narrow and sparse
  clones, (and shallow clones better). This means being able to clone part
  of a Git repository, such as just the last revision (shallow), just the
  \u2018lib\u2019 directory (narrow) or just a single file
  (sparse). Importantly, you would be able to see the history of everything
  still (it would download the commit and tree objects, which are generally
  small, but not the larger blobs), and you would be able to do pushes back
  (which shallow clones can\u2019t currently do).

  http://www.thousandparsec.net/~tim/media+git.pdf  

* JH: $$$ discussion

  IRC: It was about the Google Summer of Code 2008 mentor money. It was
  decided (given most of the mentors were there) to use it to help a few
  folks cover their travel expenses to the GitTogether. Most folks actually
  had their costs covered by their employer (yay for some big companies!)
  but a few paid out of pocket, so we are helping them defray it a bit. and
  the rest is being used to order and distribute some git t-shirts.

* Sam: Git as a DB Backend

  IRC: mugwump had this idea of running a sql sort of database in git, using
  the git logs as a way to implement an ACID complaint database. to be
  honest i was busy reading email or something and didn't pay much attention
  to his talk, but folks seemed to think it was an abuse of git

* Petr: GUI Even a Designer Uses

  IRC: it was rather short and ambiguous; mostly arguing whether to and in
  what language/toolkit to reimplement git-gui


Sources:
========
* http://git.or.cz/gitwiki/GitTogether
* http://google-opensource.blogspot.com/2008/11/gittogether-08.html
* http://github.com/blog/196-gittogether-2008
* Calendar ID: amj5rbmljs66k9rtibs34kc2c8@group.calendar.google.com
* http://colabti.org/irclogger/irclogger_log_search/git
  #git channel on irc.freenode.org


-- 
Jakub Narebski

Re: How it was at GitTogether'08 ?

[Johan Herland]

On Saturday 08 November 2008, Jakub Narebski wrote:
> GitTogether 2008, which took place October 27(Mon)-29(Wed), has ended
> more than week ago.  Therefore I'd like to ask you impressions (while it
> is still fresh) from GitTogether, describe talks which are not described
> below, correct wrong information in below, etc.

Thanks for providing this summary to those of us that didn't attend.

> * Discussion on notes

Can someone elaborate on this? AFAIK, notes have popped up on this list 
often enough that I'm convinced it would be a _really_ useful feature. The 
only drawback I was aware of, was the lack of an efficient implementation, 
but then Jeff comes out of the blue and posts some interesting numbers [1] 
a week or so ago. Does this mean there are no remaining obstacles?

[1]: http://article.gmane.org/gmane.comp.version-control.git/99415


Have fun!

...Johan

-- 
Johan Herland, <johan@herland.net>
www.herland.net

Re: How it was at GitTogether'08 ?

[Jeff King]

On Sat, Nov 08, 2008 at 04:41:25AM +0100, Johan Herland wrote:

> > * Discussion on notes
> 
> Can someone elaborate on this? AFAIK, notes have popped up on this list 
> often enough that I'm convinced it would be a _really_ useful feature. The 
> only drawback I was aware of, was the lack of an efficient implementation, 
> but then Jeff comes out of the blue and posts some interesting numbers [1] 
> a week or so ago. Does this mean there are no remaining obstacles?
> 
> [1]: http://article.gmane.org/gmane.comp.version-control.git/99415

The discussion was along the lines of "here are some more cool things we
could do, if we had notes." I don't remember the specifics of the cool
things, but they were related to annotating patches with review
information. Shawn can probably elaborate more.

That led to a "notes as a tree are nice, but too slow because looking up
a tree entry is linear" (and obviously you do a ton of lookups in the
notes tree during "git log"). Dscho had posted an implementation with a
persistent notes cache long ago. Since I failed to actually look at
that, I started on a slightly different approach, which is simply doing
an in-memory hash table to speedup the notes tree. And those are the
numbers and patch I posted.

My eventual plan was to re-work Dscho's patches with this performance
approach. But it is not at the top of my queue, so if somebody else
wanted to pick it up, I would be very happy. Everything I have done so
far is in the post you referenced.

The only other thing I remember discussing was notes namespaces. The two
obvious approaches are:

 1. a separate ref for each notes namespace, with each note ending up a
    blob in a tree. So you might have refs/notes/acked-by:$SHA1 as a
    blob.

 2. one notes ref, with the notes tree pointing a sub-tree that has
    named entries, one for each note type. So you might have
    refs/notes:$SHA1/acked-by as a blob.

The advantage of '1' is that it keeps your different note types
separate, which means it is easy to distribute one type but not the
other. The advantage of '2' is that I do one lookup per-commit, and then
I can see all of the notes, which keeps performance nice when you want
to annotate with several note types.

After some discussion, I think Dscho and I came to the conclusion that
supporting both might be desirable. And it should be pretty
straightforward. You can just have multiple note refs (but default to a
"main" one), and within each one, either point to a tree or blob (and we
will see which and use it appropriately).

And then depending on which notes the user wants, they can refer to them
appropriately. My suggestion for naming (and this wasn't discussed
earlier, so Dscho has not endorsed this) would be something like
"$X:$Y", which would mean "to get the notes for $SHA1, look at the tree
in refs/notes/$X for the file $SHA1/$Y". If $Y is empty, then expect
$SHA1 to be a blob (if it's a tree, maybe look at $SHA1/default). If
"$X" is empty, then use "refs/notes/default". If there is no colon,
assume we have "$Y".

So you could have a bunch of notes in some "main" namespace just by
calling them some name; without a name, you get some "default" note. But
if you wanted a separate database (say, for SVN information), you could
use "svn:" or "svn:name".

-Peff

Re: How it was at GitTogether'08 ?

[David Symonds]

On Fri, Nov 7, 2008 at 5:54 PM, Jakub Narebski <jnareb@gmail.com> wrote:

Here's my thoughts on some of these talks.

> Mon, Oct 27, 2008
> -----------------
> * Dscho: Contributing with Git
>  http://www.youtube.com/watch?v=j45cs5_nY2k

It was a good intro, but I was expecting a few more non-GitTogether
people. We had quite a large room, but there was only about a dozen
other people who came along. I don't know whether that was the fault
of the timing, lack of advertising, or a lack of interest.

> * Junio: Git Chronicle
>
>  blog: Junio went though a sort of statistical history of the Git project
>  that was fascinating (turns out there are still about 220 lines of code
>  still around from Linus original first commit).

This was really interesting. It would be great to put this on a
general web page instead of in a PDF.

> * Petr: Renames Again and Again and Again
>
>  IRC: detection of wholesame renames of directories (WIP) and '--follow'
>  limitation were mentioned, but outcome is unclear; pasky plans to hack
>  together some patch implementing explicit renames hinting

One thing I didn't get around to bringing up: one of the benefits of
diff-time rename detection that is often touted is that algorithms can
improve over time. Do folk here know whether that has actually
happened recently, in a general way? Do people actually expect major
improvements in the future?

> * Tom: GitHub
>
>  IRC: a tour, some history, and insight into how it works; some nice
>  gimmicks, such as "gist" (a git backed pastebin), or like network graphs
>  (look graphically at forks of a repository).
>
>  blog: Tom got to demonstrate GitHub and Gist to the group, most of whom
>  are very command line oriented and had not used either before.

The demo of iGitHub (an iPhone app that can act as a clone/push
target) looked really cool, if it can get further development. It
could potentially be really handy for travellers who could push to
their iPhone, and then push from there to an internet server.

> * Scott: Linkable Library
>
>  blog: got to talk about the need for a linkable git library
>
>  http://thread.gmane.org/gmane.comp.version-control.git/99608

It's good to see this starting to get wider traction. I think we
discussed that there could be benefits to git itself, beyond just
helping other programs access git repositories faster than fork/exec.

> * Sam: perl.git
>
>  blog: Sam demonstrated the work he went through to import 20 years of Perl
>  history into the git repository that the Perl team is just now finishing
>  transitioning to from Perforce.
>
>  http://utsl.gen.nz/talks/perl-history/slides/

It was very cool to see old-school email addresses like <isis!aburt>
in git, handled just fine.

> * Tim: Git as a Media Repository
>  http://www.thousandparsec.net/~tim/media+git.pdf

This has kicked off some mailing list discussion; I think this can be
a major weak point for git, since checking out only a subtree (and
only the latest revision) is the common SVN way, which copes with
media repositories and the like just fine.


> * JH: $$$ discussion
>
>  IRC: It was about the Google Summer of Code 2008 mentor money. It was
>  decided (given most of the mentors were there) to use it to help a few
>  folks cover their travel expenses to the GitTogether. Most folks actually
>  had their costs covered by their employer (yay for some big companies!)
>  but a few paid out of pocket, so we are helping them defray it a bit. and
>  the rest is being used to order and distribute some git t-shirts.

Yes, how are the t-shirts going? I seem to remember JH had volunteered
to do the logistics there.


Dave.

Re: How it was at GitTogether'08 ?

[Jakub Narebski]

On Sat, 8 Nov 2008, David Symonds wrote:
> On Fri, Nov 7, 2008 at 5:54 PM, Jakub Narebski <jnareb@gmail.com> wrote:
> 
> Here's my thoughts on some of these talks.
> 
> > Mon, Oct 27, 2008
> > -----------------
> > * Dscho: Contributing with Git
> >  http://www.youtube.com/watch?v=j45cs5_nY2k
> 
> It was a good intro, but I was expecting a few more non-GitTogether
> people. We had quite a large room, but there was only about a dozen
> other people who came along. I don't know whether that was the fault
> of the timing, lack of advertising, or a lack of interest.

By the way, it would be nice to have transcript for this talk, just
like there is for Linus talk:
  http://git.or.cz/gitwiki/LinusTalk200705Transcript
(but this would take some doing).
 
It would be also nice to have slides for the talk available somewhere,
just like slides for "Git Chronicle".

> > * Junio: Git Chronicle
> >
> >  blog: Junio went though a sort of statistical history of the Git project
> >  that was fascinating (turns out there are still about 220 lines of code
> >  still around from Linus original first commit).
> 
> This was really interesting. It would be great to put this on a
> general web page instead of in a PDF.

Something like Sam Vilain slides from "perl.git" talk?, 
  http://utsl.gen.nz/talks/perl-history/slides/

It shouldn't be that hard, depending on the original program the slides
were made... well, it was made in Impress from OpenOffice.org 2.4; it
might have export to (X)HTML + images, and to SWF (Flash presentation).

> > * Petr: Renames Again and Again and Again
> >
> >  IRC: detection of wholesame renames of directories (WIP) and '--follow'
> >  limitation were mentioned, but outcome is unclear; pasky plans to hack
> >  together some patch implementing explicit renames hinting
> 
> One thing I didn't get around to bringing up: one of the benefits of
> diff-time rename detection that is often touted is that algorithms can
> improve over time. Do folk here know whether that has actually
> happened recently, in a general way? Do people actually expect major
> improvements in the future?

If I remember correctly there was at least one improvement in rename
detection, namely better talking into account filename similarity score,
so for example similar files moved (or copied) didn't get marked as
coming from one source (and rest deleted).

> > * Tom: GitHub
> >
> >  IRC: a tour, some history, and insight into how it works; some nice
> >  gimmicks, such as "gist" (a git backed pastebin), or like network graphs
> >  (look graphically at forks of a repository).
> >
> >  blog: Tom got to demonstrate GitHub and Gist to the group, most of whom
> >  are very command line oriented and had not used either before.
> 
> The demo of iGitHub (an iPhone app that can act as a clone/push
> target) looked really cool, if it can get further development. It
> could potentially be really handy for travellers who could push to
> their iPhone, and then push from there to an internet server.

iGitHub has nothing to do with GitHub; I think you put the comment in
a wrong place; the iGitHub (or iGit / iGitRouter) was a separate talk
in "Lighting Round Talks" next day.

> > * Scott: Linkable Library
> >
> >  blog: got to talk about the need for a linkable git library
> >
> >  http://thread.gmane.org/gmane.comp.version-control.git/99608
> 
> It's good to see this starting to get wider traction. I think we
> discussed that there could be benefits to git itself, beyond just
> helping other programs access git repositories faster than fork/exec.

What benefits would be those? Current design of "fire and forget",
which stopped libification efforts till now was used for a reason...

> > * Sam: perl.git
> >
> >  blog: Sam demonstrated the work he went through to import 20 years of Perl
> >  history into the git repository that the Perl team is just now finishing
> >  transitioning to from Perforce.
> >
> >  http://utsl.gen.nz/talks/perl-history/slides/
> 
> It was very cool to see old-school email addresses like <isis!aburt>
> in git, handled just fine.

This is not suprising, as Git treats committer and author email data
as opaque data, not analysing it at all (some commits from early
versions of git might not have this data at all, IIRC).
 
> > * Tim: Git as a Media Repository
> >  http://www.thousandparsec.net/~tim/media+git.pdf
> 
> This has kicked off some mailing list discussion; I think this can be
> a major weak point for git, since checking out only a subtree (and
> only the latest revision) is the common SVN way, which copes with
> media repositories and the like just fine.

Well, you can workaround this weakness by (ab)using submodules...
...and one should always remember that casual partial checkouts
interfere a bit with whole-tree commits.

-- 
Jakub Narebski
Poland

Re: How it was at GitTogether'08 ?

[Kai Blin]

[-- Attachment #1: Type: text/plain, Size: 1142 bytes --]

On Saturday 08 November 2008 16:31:04 Jakub Narebski wrote:

> > > * Tim: Git as a Media Repository
> > >  http://www.thousandparsec.net/~tim/media+git.pdf
> >
> > This has kicked off some mailing list discussion; I think this can be
> > a major weak point for git, since checking out only a subtree (and
> > only the latest revision) is the common SVN way, which copes with
> > media repositories and the like just fine.
>
> Well, you can workaround this weakness by (ab)using submodules...
> ...and one should always remember that casual partial checkouts
> interfere a bit with whole-tree commits.

Interesting. How would you use submodules to work around the fact that binary 
file changes diff very bad and produce huge histories with basically no value 
for the user of the working copy? Can you do this from a GUI, easily? We're 
talking about media repositories here, so our users are artists.

Cheers,
Kai


-- 
Kai Blin
WorldForge developer  http://www.worldforge.org/
Wine developer        http://wiki.winehq.org/KaiBlin
Samba team member     http://www.samba.org/samba/team/
--
Will code for cotton.

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: How it was at GitTogether'08 ?

[Jakub Narebski]

On Sun, 9 Nov 2008, Kai Blin wrote:
> On Saturday 08 November 2008 16:31:04 Jakub Narebski wrote:

>>>> * Tim: Git as a Media Repository
>>>>  http://www.thousandparsec.net/~tim/media+git.pdf
>>>
>>> This has kicked off some mailing list discussion; I think this can be
>>> a major weak point for git, since checking out only a subtree (and
>>> only the latest revision) is the common SVN way, which copes with
>>> media repositories and the like just fine.
>>
>> Well, you can workaround this weakness by (ab)using submodules...
>> ...and one should always remember that casual partial checkouts
>> interfere a bit with whole-tree commits.
> 
> Interesting. How would you use submodules to work around the fact that binary 
> file changes diff very bad and produce huge histories with basically no value 
> for the user of the working copy? Can you do this from a GUI, easily? We're 
> talking about media repositories here, so our users are artists.

What I meant here (but perhaps was not clear) was (ab)using submodules
to allow to have full working repository without large [media] files
both in object database (repository) and without them checked out.

The workaround is to put all large files for example in 'media/' folder,
and make this folder be submodule. Each clone of repository can have
this 'media' submodule either present (both in object database, although
usually separate from main project object database), or not present
(not cloned and not checked out).

As to submodules UI and GUI support for submodules... currently it is
unfortunately lacking.


Note that I explicitly mentioned that (ab)using submodules to better
deal with large files is _workaround_, and not _solution_. Lazy clone
in version proposed by Tim is IMHO correct solution.

P.S. Could anybody document at last `delta' gitattribute?
P.P.S. You can have separate diff driver for binary files, but I don't
       know anyone who uses for example some such for images...
-- 
Jakub Narebski
Poland

Re: How it was at GitTogether'08 ?

[Kai Blin]

[-- Attachment #1: Type: text/plain, Size: 937 bytes --]

On Sunday 09 November 2008 17:31:47 Jakub Narebski wrote:


> The workaround is to put all large files for example in 'media/' folder,
> and make this folder be submodule. Each clone of repository can have
> this 'media' submodule either present (both in object database, although
> usually separate from main project object database), or not present
> (not cloned and not checked out).

Tim was talking about that media/ folder and managing that in git. If you want 
to work on the media, you might end up getting hundreds of gigabytes of data 
to get that folder, even if you only need to change one single file.

That's the issue we're running into, and I don't thing submodules solve this 
at all.

Cheers,
Kai

-- 
Kai Blin
WorldForge developer  http://www.worldforge.org/
Wine developer        http://wiki.winehq.org/KaiBlin
Samba team member     http://www.samba.org/samba/team/
--
Will code for cotton.

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: Large media in git (was: How it was at GitTogether'08)?

[Jakub Narebski]

On Sun, 9 Nov 2008, Kai Blin wrote:
> On Sunday 09 November 2008 17:31:47 Jakub Narebski wrote:
> 
> > The workaround is to put all large files for example in 'media/' folder,
> > and make this folder be submodule. Each clone of repository can have
> > this 'media' submodule either present (both in object database, although
> > usually separate from main project object database), or not present
> > (not cloned and not checked out).
> 
> Tim was talking about that media/ folder and managing that in git. If you want 
> to work on the media, you might end up getting hundreds of gigabytes of data 
> to get that folder, even if you only need to change one single file.
> 
> That's the issue we're running into, and I don't thing submodules solve this 
> at all.

Ah, well... Submodules cannot be workaround for _this_ issue. You can
have only all or nothing: either all files in media/ or none of them,
both in working directory like in repository object database... well
unless you subdivide further.

I guess that mentioned work on the media is in remote setting (you
cannot have main repository on network drive) so Dana How's proposed
solution would not work for you, is it?

-- 
Jakub Narebski
Poland

Re: Large media in git (was: How it was at GitTogether'08)?

[Kai Blin]

[-- Attachment #1: Type: text/plain, Size: 1736 bytes --]

On Monday 10 November 2008 10:30:53 Jakub Narebski wrote:
> On Sun, 9 Nov 2008, Kai Blin wrote:
> > On Sunday 09 November 2008 17:31:47 Jakub Narebski wrote:
> > > The workaround is to put all large files for example in 'media/'
> > > folder, and make this folder be submodule. Each clone of repository can
> > > have this 'media' submodule either present (both in object database,
> > > although usually separate from main project object database), or not
> > > present (not cloned and not checked out).
> >
> > Tim was talking about that media/ folder and managing that in git. If you
> > want to work on the media, you might end up getting hundreds of gigabytes
> > of data to get that folder, even if you only need to change one single
> > file.
> >
> > That's the issue we're running into, and I don't thing submodules solve
> > this at all.
>
> Ah, well... Submodules cannot be workaround for _this_ issue. You can
> have only all or nothing: either all files in media/ or none of them,
> both in working directory like in repository object database... well
> unless you subdivide further.
>
> I guess that mentioned work on the media is in remote setting (you
> cannot have main repository on network drive) so Dana How's proposed
> solution would not work for you, is it?

If my google-fu worked, the proposed solution you're talking about involves 
simply hiding large blobs from pack files, right? In that case it won't work, 
as the users of the repository indeed are remote.

Cheers,
Kai

-- 
Kai Blin
WorldForge developer  http://www.worldforge.org/
Wine developer        http://wiki.winehq.org/KaiBlin
Samba team member     http://www.samba.org/samba/team/
--
Will code for cotton.

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: How it was at GitTogether'08 ?

[Johannes Schindelin]

Hi,

On Sun, 9 Nov 2008, Kai Blin wrote:

> On Sunday 09 November 2008 17:31:47 Jakub Narebski wrote:
> 
> > The workaround is to put all large files for example in 'media/' 
> > folder, and make this folder be submodule. Each clone of repository 
> > can have this 'media' submodule either present (both in object 
> > database, although usually separate from main project object 
> > database), or not present (not cloned and not checked out).
> 
> Tim was talking about that media/ folder and managing that in git. If 
> you want to work on the media, you might end up getting hundreds of 
> gigabytes of data to get that folder, even if you only need to change 
> one single file.
> 
> That's the issue we're running into, and I don't thing submodules solve 
> this at all.

You'd have to have a single repository for each and every media file, and 
you'd need to use shallow clones and shallow fetches.

However, a push-conflict will probably be beyond any non-programmer 
skillz.

I'd rather propose to have a different interface, like through a web 
server, where the user can say "I have some cool new graphics, in this 
.zip file" together with a commit message.

Kind of a git-gui via browser.

Ciao,
Dscho

Re: How it was at GitTogether'08 ?

[Kai Blin]

[-- Attachment #1: Type: text/plain, Size: 1641 bytes --]

On Monday 10 November 2008 10:58:05 Johannes Schindelin wrote:
Hi Dscho,

> > Tim was talking about that media/ folder and managing that in git. If
> > you want to work on the media, you might end up getting hundreds of
> > gigabytes of data to get that folder, even if you only need to change
> > one single file.
> >
> > That's the issue we're running into, and I don't thing submodules solve
> > this at all.
>
> You'd have to have a single repository for each and every media file, and
> you'd need to use shallow clones and shallow fetches.
>
> However, a push-conflict will probably be beyond any non-programmer
> skillz.

Ok, I agree. But you could work around that by teaching the artists to 
fetch/rebase/push instead of just pushing, or hiding this in the GUI. If 
there's a conflict on a binary data file you're screwed anyway. :)

> I'd rather propose to have a different interface, like through a web
> server, where the user can say "I have some cool new graphics, in this
> .zip file" together with a commit message.
>
> Kind of a git-gui via browser.

Incidentally I'm currently working on something like this, just aimed at 
the "artist side", instead of the VCS side. This certainly is a useable 
solution for artists. But at some point a developer will want to check out 
the repository to cut a release tarball, and we're back to wanting shallow 
and narrow clones. :)

Cheers,
Kai

-- 
Kai Blin
WorldForge developer  http://www.worldforge.org/
Wine developer        http://wiki.winehq.org/KaiBlin
Samba team member     http://www.samba.org/samba/team/
--
Will code for cotton.

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: How it was at GitTogether'08 ?

[Johannes Schindelin]

Hi,

On Mon, 10 Nov 2008, Kai Blin wrote:

> On Monday 10 November 2008 10:58:05 Johannes Schindelin wrote:
> 
> > > Tim was talking about that media/ folder and managing that in git. 
> > > If you want to work on the media, you might end up getting hundreds 
> > > of gigabytes of data to get that folder, even if you only need to 
> > > change one single file.
> > >
> > > That's the issue we're running into, and I don't thing submodules 
> > > solve this at all.
> >
> > You'd have to have a single repository for each and every media file, 
> > and you'd need to use shallow clones and shallow fetches.
> >
> > However, a push-conflict will probably be beyond any non-programmer 
> > skillz.
> 
> Ok, I agree. But you could work around that by teaching the artists to 
> fetch/rebase/push instead of just pushing, or hiding this in the GUI. If 
> there's a conflict on a binary data file you're screwed anyway. :)

A fetch in that case would make the artist download things she does not 
need, right?  So maybe that is not the way to go.

> > I'd rather propose to have a different interface, like through a web 
> > server, where the user can say "I have some cool new graphics, in this 
> > .zip file" together with a commit message.
> >
> > Kind of a git-gui via browser.
> 
> Incidentally I'm currently working on something like this, just aimed at 
> the "artist side", instead of the VCS side. This certainly is a useable 
> solution for artists.

Seems like a nice starting point for a git-gui-in-a-browser.

> But at some point a developer will want to check out the repository to 
> cut a release tarball, and we're back to wanting shallow and narrow 
> clones. :)

Shallow clones are not an issue.  They _should_ work.

And for releasing you do not want narrow clones.

Ciao,
Dscho

Re: How it was at GitTogether'08 ?

[Johannes Schindelin]

Hi,

On Sat, 8 Nov 2008, Jakub Narebski wrote:

> On Sat, 8 Nov 2008, David Symonds wrote:
> > On Fri, Nov 7, 2008 at 5:54 PM, Jakub Narebski <jnareb@gmail.com> wrote:
> > 
> > > Mon, Oct 27, 2008
> > > -----------------
> > > * Dscho: Contributing with Git
> > >  http://www.youtube.com/watch?v=j45cs5_nY2k
> > 
> > It was a good intro, but I was expecting a few more non-GitTogether 
> > people. We had quite a large room, but there was only about a dozen 
> > other people who came along. I don't know whether that was the fault 
> > of the timing, lack of advertising, or a lack of interest.
> 
> By the way, it would be nice to have transcript for this talk, just
> like there is for Linus talk:
>   http://git.or.cz/gitwiki/LinusTalk200705Transcript
> (but this would take some doing).
>
> It would be also nice to have slides for the talk available somewhere, 
> just like slides for "Git Chronicle".

Voila:

http://repo.or.cz/w/git/dscho.git?a=blob_plain;f=all-your-rebase.pdf;hb=f23c1c9868256cae8591bddafd64422f95d67b02

Ciao,
Dscho

Re: How it was at GitTogether'08 ?

[Jakub Narebski]

On Mon, 10 Nov 2008, Johannes Schindelin wrote:
> On Sat, 8 Nov 2008, Jakub Narebski wrote:
>> On Sat, 8 Nov 2008, David Symonds wrote:
>>> On Fri, Nov 7, 2008 at 5:54 PM, Jakub Narebski <jnareb@gmail.com> wrote:
>>> 
>>>> Mon, Oct 27, 2008
>>>> -----------------
>>>> * Dscho: Contributing with Git
>>>>  http://www.youtube.com/watch?v=j45cs5_nY2k
>>> 
>>> It was a good intro, but I was expecting a few more non-GitTogether 
>>> people. We had quite a large room, but there was only about a dozen 
>>> other people who came along. I don't know whether that was the fault 
>>> of the timing, lack of advertising, or a lack of interest.
[...]

>> It would be also nice to have slides for the talk available somewhere, 
>> just like slides for "Git Chronicle".
> 
> Voila:
> 
> http://repo.or.cz/w/git/dscho.git?a=blob_plain;f=all-your-rebase.pdf;hb=f23c1c9868256cae8591bddafd64422f95d67b02

Thanks a lot. Added to http://git.or.cz/gitwiki/GitTogether

P.S. I guess GitTogether wiki page should be perhaps rewritten...
-- 
Jakub Narebski
Poland

Re: How it was at GitTogether'08 ?

[Jakub Narebski]

On Sat, 8 Nov 2008, Jakub Narębski wrote:

> GitTogether 2008, which took place October 27(Mon)-29(Wed), has ended more
> than week ago.  Therefore I'd like to ask you impressions (while it is still
> fresh) from GitTogether, describe talks which are not described below,
> correct wrong information in below, etc.
> 
> Please reply to this email while GitTogether is on front of your mind...

Still missing (neither video, nor slides, nor description, nor email)
are the following talks from GitTogether'08

 
> Talks at GitTogether 2008
> =========================
> 
> Mon, Oct 27, 2008
> -----------------
> * Jeff: Helping new contributors join
> * Shawn: JGit
 
> Tue, Oct 28, 2008
> -----------------
> * Jeff: What needs refactoring?
> * Lighting Round Talks
>   - Jeff: Portability autobuilders
>   - Petr: Quick TopGit introduction, problems, future plans
> * Steven: Life with git-svn

> Wed, Oct 29, 2008
> -----------------
+ What was the difference between "Tim: Large media in Git (Repeat)"
  from Wed, and earlier "Tim: Git as a Media Repository" from Tue?

-- 
Jakub Narebski
Poland

Re: How it was at GitTogether'08 ?

[Steven Grimm]

On Nov 9, 2008, at 3:49 AM, Jakub Narebski wrote:
> Still missing (neither video, nor slides, nor description, nor email)
> are the following talks from GitTogether'08
>> * Steven: Life with git-svn

Here are the slides from David's and my talk:

http://docs.google.com/Presentation?id=dhhs72s2_1wtzbnsnj&invite=v4t8kr

-Steve

Re: How it was at GitTogether'08 ?

[Jakub Narebski]

Steven Grimm wrote:
> On Nov 9, 2008, at 3:49 AM, Jakub Narebski wrote:
>>
>> Still missing (neither video, nor slides, nor description, nor email)
>> are the following talks from GitTogether'08
>>
>>> * Steven: Life with git-svn
> 
> Here are the slides from David's and my talk:
> 
> http://docs.google.com/Presentation?id=dhhs72s2_1wtzbnsnj&invite=v4t8kr

Thanks. Added to http://git.or.cz/gitwiki/GitTogether
(even if I cannot view them in my old web browser).

-- 
Jakub Narebski
Poland

Re: How it was at GitTogether'08 ?

[Robin Rosenberg]

söndag 09 november 2008 17:52:58 skrev Steven Grimm:
> On Nov 9, 2008, at 3:49 AM, Jakub Narebski wrote:
> > Still missing (neither video, nor slides, nor description, nor email)
> > are the following talks from GitTogether'08
> >> * Steven: Life with git-svn
> 
> Here are the slides from David's and my talk:
> 
> http://docs.google.com/Presentation?id=dhhs72s2_1wtzbnsnj&invite=v4t8kr

Requiring a google (or facebook for that matter) account is read public documents
is not nice.

-- robin

Re: How it was at GitTogether'08 ?

[Shawn O. Pearce]

Robin Rosenberg <robin.rosenberg@dewire.com> wrote:
> söndag 09 november 2008 17:52:58 skrev Steven Grimm:
> > On Nov 9, 2008, at 3:49 AM, Jakub Narebski wrote:
> > > Still missing (neither video, nor slides, nor description, nor email)
> > > are the following talks from GitTogether'08
> > >> * Steven: Life with git-svn
> > 
> > Here are the slides from David's and my talk:
> > 
> > http://docs.google.com/Presentation?id=dhhs72s2_1wtzbnsnj&invite=v4t8kr
> 
> Requiring a google (or facebook for that matter) account is read public documents
> is not nice.

No, its not.  The owner of the document can publish the document,
making it world-readable, *without* needing a login.  I think they
just forgot to do that on this particular presentation.

Steven, can you publish that doc, so it doesn't require login
to read?

-- 
Shawn.

Re: How it was at GitTogether'08 ?

[Steven Grimm]

On Nov 9, 2008, at 11:55 AM, Shawn O. Pearce wrote:
> No, its not.  The owner of the document can publish the document,
> making it world-readable, *without* needing a login.  I think they
> just forgot to do that on this particular presentation.


My bad. Here's the published, no-login-required version:

http://docs.google.com/Presentation?id=dhhs72s2_1wtzbnsnj

-Steve

Re: How it was at GitTogether'08 ?

[Jakub Narebski]

Steven Grimm wrote:
> On Nov 9, 2008, at 11:55 AM, Shawn O. Pearce wrote:

> > No, its not.  The owner of the document can publish the document,
> > making it world-readable, *without* needing a login.  I think they
> > just forgot to do that on this particular presentation.
> 
> 
> My bad. Here's the published, no-login-required version:
> 
> http://docs.google.com/Presentation?id=dhhs72s2_1wtzbnsnj

Would it be possible to publish PDF version sowehere, for example
as attachement to http://git.or.cz/gitwiki/GitTogether? My old
web browser doesn't support Google Docs...

-- 
Jakub Narebski
Poland

Re: How it was at GitTogether'08 ?

[Jonas Fonseca]

On Mon, Nov 10, 2008 at 00:52, Jakub Narebski <jnareb@gmail.com> wrote:
> Would it be possible to publish PDF version sowehere, for example
> as attachement to http://git.or.cz/gitwiki/GitTogether? My old
> web browser doesn't support Google Docs...

http://docs.google.com/MiscCommands?command=saveasdoc&docID=dhhs72s2_1wtzbnsnj&exportFormat=pdf

-- 
Jonas Fonseca

Re: How it was at GitTogether'08 ?

[Johannes Schindelin]

Hi,

On Tue, 11 Nov 2008, Jonas Fonseca wrote:

> On Mon, Nov 10, 2008 at 00:52, Jakub Narebski <jnareb@gmail.com> wrote:
> > Would it be possible to publish PDF version sowehere, for example
> > as attachement to http://git.or.cz/gitwiki/GitTogether? My old
> > web browser doesn't support Google Docs...
> 
> http://docs.google.com/MiscCommands?command=saveasdoc&docID=dhhs72s2_1wtzbnsnj&exportFormat=pdf

I also included whatever I could grab (did not take care of mugwumps html 
pages yet) into my presentations branch.

Ciao,
Dscho

Re: How it was at GitTogether'08 ?

[Shawn O. Pearce]

Jakub Narebski <jnareb@gmail.com> wrote:
> 
> Still missing (neither video, nor slides, nor description, nor email)
> are the following talks from GitTogether'08

JGit slides:
http://www.spearce.org/2008/11/JGit.pdf

Pack v4 slides:
http://www.spearce.org/2008/11/Pack_v4.pdf

I also had these as a Google Doc, but its internal on the google.com
domain so I can't easily publish it.  Plus PDF is probably more
portable to older browsers than the Google Docs site is. :-)

-- 
Shawn.

Re: How it was at GitTogether'08 ?

[Jakub Narebski]

Shawn O. Pearce wrote:
> Jakub Narebski <jnareb@gmail.com> wrote:
> > 
> > Still missing (neither video, nor slides, nor description, nor email)
> > are the following talks from GitTogether'08
> 
> JGit slides:
> http://www.spearce.org/2008/11/JGit.pdf
> 
> Pack v4 slides:
> http://www.spearce.org/2008/11/Pack_v4.pdf

Thanks. Added to http://git.or.cz/gitwiki/GitTogether

-- 
Jakub Narebski
Poland

Re: How it was at GitTogether'08 ?

[Tim Ansell]

> > Wed, Oct 29, 2008
> > -----------------
> + What was the difference between "Tim: Large media in Git (Repeat)"
>   from Wed, and earlier "Tim: Git as a Media Repository" from Tue?

The first day I got people feed back on my proposal (whiteboard
discussion) and then presented the proposal on Wednesday.

You can find the slides of the presentation at 
http://www.thousandparsec.net/~tim/media+git.pdf

Sorry about the duplicate messages, I hit send too soon.

Tim 'mithro' Ansell

Re: How it was at GitTogether'08 ?

[Jean-Luc Herren]

Jakub Narebski wrote:
> and the rest is being used to order and distribute some git
> t-shirts.

Do such git shirts exist already and could be ordered somewhere?
I might be interested in buying one.

Cheers,
jlh

Re: How it was at GitTogether'08 ?

[Junio C Hamano]

Jakub Narebski <jnareb@gmail.com> writes:

> * David Brown: Life with Git
>   http://www.davidb.org/git/git-corp.pdf

I really enjoyed this talk, which was in a weird sense very encouraging.

By the way, one thing David talked about that is not in the PDF slides but
I found quite good to stress was the importance of good commit messages.
"Write in present tense, it will read much nicer and you'll appreciate it
after reading hundreds of them".

I'd actually say "imperative mood" instead of "present tense" (but they
look almost always the same in English), but in any case, it really gets
on my nerve to read commit message that talks things in past tense and I
often end up rewriting other people's commit log messages.