Git Community Book

Git Community Book

[Scott Chacon]

So I wanted to develop a really nice, easy to follow book for Git
newcomers to learn git quickly and easily.  One of the issues I
remember having when learning Git is that there is a lot of great
material in the User Guide, Tutorial, Tutorial 2, Everyday Git, etc -
but they're all huge long documents that are sometimes difficult to
come back to and remember where you were, and I didn't know which one
to start with or where to find what I was looking for, etc.

So, what I've started to do is pull material from all of them into a
single book which will be available in online HTML (one page per
chapter) and downloadable PDF form.  I'm trying to give it a very
organized flow that will hopefully be a bit easier to follow and
digest than the current formats, and including a number of diagrams,
illustrations and screencasts to supplement the text.  Where possible,
I am also trying to simplify the explanations a bit to be a tad more
digestible for beginning users, at least in the first couple dozen
chapters. I have put the current html output of this book here:

http://book.git-scm.com

It is not complete - the grey links are chapters that are very short
or completely empty - but it is a start.  Please let me know what you
think, and if anyone is interested in helping with the project, give
me a shout.

Also, for credit, I have generated an Authors page I will be linking
to the site soon that lists everyone that contributed a patch to any
of the Git User Guide, Git Tutorials, etc.  It is in the PDF right
now, but not in the HTML version yet (and the PDF is not yet linked to
the site).

Thanks,
Scott

Re: Git Community Book

[Miklos Vajna]

[-- Attachment #1: Type: text/plain, Size: 436 bytes --]

On Tue, Jul 29, 2008 at 09:20:20AM -0700, Scott Chacon <schacon@gmail.com> wrote:
> It is not complete - the grey links are chapters that are very short
> or completely empty - but it is a start.  Please let me know what you
> think, and if anyone is interested in helping with the project, give
> me a shout.

At http://github.com/schacon/learn-github/wikis/how-to-contribute, there
is a typo: you want 'git checkout origin/book'. ;-)

[-- Attachment #2: Type: application/pgp-signature, Size: 197 bytes --]

Re: Git Community Book

[Petr Baudis]

On Tue, Jul 29, 2008 at 09:20:20AM -0700, Scott Chacon wrote:
> So, what I've started to do is pull material from all of them into a
> single book which will be available in online HTML (one page per
> chapter) and downloadable PDF form.  I'm trying to give it a very
> organized flow that will hopefully be a bit easier to follow and
> digest than the current formats, and including a number of diagrams,
> illustrations and screencasts to supplement the text.  Where possible,
> I am also trying to simplify the explanations a bit to be a tad more
> digestible for beginning users, at least in the first couple dozen
> chapters. I have put the current html output of this book here:
> 
> http://book.git-scm.com

I think what most of the people here would be also interested in is

	http://github.com/schacon/learn-github/wikis/how-to-contribute

There is no license in the source code - what are the copying terms?

It is maybe somewhat unfortunate that this is in a different format that
the standard git choice asciidoc, but the formats do look rather similar
so I assume it should not be hard to even convert from one to another if
needed.

Unfortunately, I probably won't have enough time to review the content
in details anytime soon, so I can only say that that the site looks
pretty. :-) I have skimmed through the Introduction part only, but
frankly, my feelings are somewhat mixed; I think the "direct dive-in"
you take in the Database and Index section is controversial at best, and
I personally much prefer the gentle approach of user manual, which does
not hurl details on git's objects model on the user right away. To me,
it would make sense to move this all somewhere between chapter four and
five. (Incidentally, only after writing this, I have looked at the
actual structure of the User Manual and I think it makes more sense than
your approach.)

So my confusion still is - where does this stand wrt. the user manual?
Why didn't you just start with the manual and work on that? I thought
you were planning to do that, but apparently we misunderstood each other
in the last mails.

Which goals are different between the Git Community Book and the User
Manual? It seems to me that the intent is the same in both cases, and if
the User Manual is not sufficiently digestible and easy to understand
for a newcomer, wouldn't it make more sense to make it so?

The thought of yet another Git resource _in addition_ to the existing
ones just makes me nervous. This isn't only about your time that I feel
is being spent unnecessarily ineffectively by not building upon the
existing text, but also about the _community_ resources - the user
manual has a great benefit that it was actually reviewed by the mailing
list so it will probably have quite smaller error rate than anything
you or me would write on our own, no matter how big Git expert you are.

I'm not saying you don't have good reasons to make the choice you did,
I just don't understand them yet - please help me here.

> So I wanted to develop a really nice, easy to follow book for Git
> newcomers to learn git quickly and easily.  One of the issues I
> remember having when learning Git is that there is a lot of great
> material in the User Guide, Tutorial, Tutorial 2, Everyday Git, etc -
> but they're all huge long documents that are sometimes difficult to
> come back to and remember where you were, and I didn't know which one
> to start with or where to find what I was looking for, etc.

So, one of your arguments is that the current material are huge long
documents that are difficult to come back to and remember where you
were. But if I'd split the User Manaul TOC to the same layout you use
for the Community Book, what is the difference here? It seems to me that
both would appear pretty much the same. Should I do a proof of concept?
;-)

> Also, for credit, I have generated an Authors page I will be linking
> to the site soon that lists everyone that contributed a patch to any
> of the Git User Guide, Git Tutorials, etc.  It is in the PDF right
> now, but not in the HTML version yet (and the PDF is not yet linked to
> the site).

So, right now you are basically taking existing material and rearranging
it? By what rules? What is the underlying idea of your approach, and why
is it better than the current structure of the user manual? Have you
considered how to perform this all so that you can easily get further
updates and corrections to the user manual?

-- 
				Petr "Pasky" Baudis
As in certain cults it is possible to kill a process if you know
its true name.  -- Ken Thompson and Dennis M. Ritchie

Re: Git Community Book

[Junio C Hamano]

"Scott Chacon" <schacon@gmail.com> writes:

> So I wanted to develop a really nice, easy to follow book for Git
> newcomers to learn git quickly and easily.  One of the issues I
> remember having when learning Git is that there is a lot of great
> material in the User Guide, Tutorial, Tutorial 2, Everyday Git, etc -
> but they're all huge long documents that are sometimes difficult to
> come back to and remember where you were, and I didn't know which one
> to start with or where to find what I was looking for, etc.

Interesting.  A few comments, before I get dragged into my day job fully.

[overall]

 - Some people mentioned that the necessity of reading through large
   volume of documentation can be reduced if they were divided by
   developer roles (similar to how Everyday does), e.g. people in
   individual contributor role does not have to learn integrator tools
   such as "am" in their first pass on the documentation.  Has the
   approach considered while developing this book?

 - The order of sections in "Working with Git" chapter somehow does not
   feel quite right, except that I'd agree that "Git on Windows" at the
   beginning is a very good idea (disclaimer. I do not use Windows
   myself). "StGIT" coming next was very understandable, but then
   "Capistrano"????  And no CVS section next to Subversion section?  Ruby
   before Perl or Python (I would have listed Perl, Python and then Ruby
   to avoid language wars.  That's the language age order, and it is even
   alphabetical)???

   Above "Capistrano" and "Ruby" comment shows the bias this TOC has (and
   my bias being different from the TOC's bias).  I'd imagine that
   Ruby-minded folks won't share the same reaction as I had.  What's the
   target audience of this book?  Git users in general, or primarily
   Ruby-minded subset?  If the latter, labeling this as "Community Book"
   may be misleading.

[http://book.git-scm.com/1_the_git_object_database.html]

 - The color of "blob" does not match the blob that is committed to eat
   trees at the top of your site ;-)

 - In a recent thread on the list, quite a lot of people seem to have
   found that teaching the low level details and plumbing first to the new
   people is detrimental.  Do you have response to that thread?

Re: Git Community Book

[Junio C Hamano]

Junio C Hamano <gitster@pobox.com> writes:

> "Scott Chacon" <schacon@gmail.com> writes:
>
>> So I wanted to develop a really nice, easy to follow book for Git
>> newcomers to learn git quickly and easily.  One of the issues I
>> remember having when learning Git is that there is a lot of great
>> material in the User Guide, Tutorial, Tutorial 2, Everyday Git, etc -
>> but they're all huge long documents that are sometimes difficult to
>> come back to and remember where you were, and I didn't know which one
>> to start with or where to find what I was looking for, etc.
>
> Interesting.  A few comments, before I get dragged into my day job fully.

(cont'd)

I was planning to comment on the contents (i.e. text) but it appears that
most of the stuff was borrowed from the User Manual, so I won't.

	Side note: I wonder if this makes the whole "Community Book"
	GPLv2.  What would happen to the part that includes your own
	screencast?  You do not mind it be contaminated by our licence?

But there seems to be some stuff User Manual does not talk about.

[3_basic_branching_and_merging.html]

 - You've talked about low-level individual objects in an earlier section
   but you stopped at showing a single commit pointing at a tree.  People
   would find branching and merging very hard to get, without
   understanding the commit DAG.  On the other hand, you can explain
   commit DAG without going into details down to trees and blobs in the
   earlier section.  The user manual has "understanding reachability"
   section early on for this exact reason.

[5_creating_new_empty_branches.html]

 - As I repeatedly said on the list, I do not think teaching this is
   useful.  Multiple roots may happen as a result of pushing (or pulling)
   from a repository with unrelated root, but it is not something you
   would want to actively aim for.  At least there needs an explanation
   for the reason why making disjoint roots in the same repository is
   (sometimes) a good thing to do, and what its downsides are.

Re: Git Community Book

[Scott Chacon]

>
> There is no license in the source code - what are the copying terms?
>

I copied in the COPYING file from Git - GPL2.

> It is maybe somewhat unfortunate that this is in a different format that
> the standard git choice asciidoc, but the formats do look rather similar
> so I assume it should not be hard to even convert from one to another if
> needed.

I simply didn't want to get asciidoc working locally - it's always
been a bit of a pain to compile (I've heard it referred to more than
once as the only 'nightmare dependancy' in git), and I don't need to
make man pages or anything, so it seemed Markdown would be a better
choice for my output targets.  There are a number of good Markdown
interpreters and they're easy to get running.

>
> Unfortunately, I probably won't have enough time to review the content
> in details anytime soon, so I can only say that that the site looks
> pretty. :-) I have skimmed through the Introduction part only, but
> frankly, my feelings are somewhat mixed; I think the "direct dive-in"
> you take in the Database and Index section is controversial at best, and
> I personally much prefer the gentle approach of user manual, which does
> not hurl details on git's objects model on the user right away. To me,
> it would make sense to move this all somewhere between chapter four and
> five. (Incidentally, only after writing this, I have looked at the
> actual structure of the User Manual and I think it makes more sense than
> your approach.)
>
> So my confusion still is - where does this stand wrt. the user manual?
> Why didn't you just start with the manual and work on that? I thought
> you were planning to do that, but apparently we misunderstood each other
> in the last mails.
>

I was originally planning on doing that, but the problem is the
graphics, diagrams and screencasts.  Unless I am mistaken, there is
not a single outside media reference in any of these guides - the
diagrams that are there are all ascii drawings.  I'm assuming there is
a reason for that. If I wanted to add images and screencast embeds
into the guide, how would that work?

Also, the user guide seems much more technical than I wanted - I
wanted to simplify a lot of the explanations, especially at the
beginning, and I don't want to screw up all the existing text.  I
thought that the best solution would be to have the Community Book as
more of a book format, and the User Guide as more of an advanced
technical guide.  We don't want to put 'Git and Capistrano' or 'Using
Git in Perl' in the User Guide, do we?  I just wanted to copy the
sections that were already well written that need to be in both, so
that I don't have to re-write them.

> Which goals are different between the Git Community Book and the User
> Manual? It seems to me that the intent is the same in both cases, and if
> the User Manual is not sufficiently digestible and easy to understand
> for a newcomer, wouldn't it make more sense to make it so?

I think the goals are a bit different.  I think the User Manual is
helpful for people coming from the Linux/Perl hacker communities that
are more used to guides like that - who like things explained more
technically and possibly even think screencasts are stupid and an
ascii graph is just as understandable as a pretty one with rounded
corners and pastel colors.

I think my goal with the book is to create a book.  The length of a
book, readable one chapter at a time over several days, etc.  Also,
eventually, I want to make it bookmarkable, maybe add some interactive
quizzes at the end of each chapter, maybe add a comments section to
the end of each chapter, add a live search box, etc.  That just seems
so much different than the User Guide and Tutorials that it warrants a
different project, but so much of the content in the Guide is quality
that I didn't want to reinvent the wheel yet again.

> The thought of yet another Git resource _in addition_ to the existing
> ones just makes me nervous. This isn't only about your time that I feel
> is being spent unnecessarily ineffectively by not building upon the
> existing text, but also about the _community_ resources - the user
> manual has a great benefit that it was actually reviewed by the mailing
> list so it will probably have quite smaller error rate than anything
> you or me would write on our own, no matter how big Git expert you are.

Well, that's what the point of this is - to ask everyone to help me
review it, and possibly help me add to it.  The user manual is great,
but even I don't reference it very often because I find it difficult
to find content in it I need quickly.  As Git becomes more and more
popular, more and more resources will continue to come out - I did the
Peepcode mini-book, which sold over a thousand copies already, and
Pragmatic Programmers and O'Reilly both have Git books in the works,
too.  I was planning on a second book with Peepcode, but I thought it
would be better to do this instead.

I would love to develop a book that is totally open and rivals all of
those and is consistently up to date and allows the community to
interact.  I don't think it's really possible to get the User Guide
there very easily except in this way.


> So, one of your arguments is that the current material are huge long
> documents that are difficult to come back to and remember where you
> were. But if I'd split the User Manaul TOC to the same layout you use
> for the Community Book, what is the difference here? It seems to me that
> both would appear pretty much the same. Should I do a proof of concept?
> ;-)

Again, I started to do this, but the image references, screencast
embeds, and general different goal of the book, both in length and
scope, makes me think that is not the best way to go.

> So, right now you are basically taking existing material and rearranging
> it? By what rules? What is the underlying idea of your approach, and why
> is it better than the current structure of the user manual? Have you
> considered how to perform this all so that you can easily get further
> updates and corrections to the user manual?

I have thought about this a lot, and it comes from the talks and
training I've done with Git and the feedback I've gotten from that.
For one, I think it's very helpful to split up the chapters into
sections ('First Time', 'Basic Usage', 'Advanced Usage', etc) so users
of different skill levels can easily see which chapters may have
something for them at a glance.

The specific order I choose is very different from the User Guide and
is likely to bother a number of people, which you mentioned (and I'm
sure Dscho will _hate_) because I introduce the object model at the
beginning.  (I'm still working on that section, trying to simplify it
and add in some other diagrams and a short screencast I have that I
think will be helpful)  This is because I have had a lot of positive
feedback that primary frustration from people comes from them thinking
of Git as a super-better Subversion.  I would venture to say that
_most_ of the users coming to Git now are currently fluent in
Subversion.  Even if they are from Perforce or CVS (the other two ones
I will occasionally run into), their mental model of what an SCM does
is the same - delta storage.  I've found that by ridding them of that
notion off the bat, they have _far_ fewer problems and frustrations
with Git than when I just try to show them the first 10 commands in
sort of a cookbook style.  It's not a complicated model, it doesn't
take long to teach, and in _my personal_ experience (which is not to
say it's necessarily correct), it helps people the most in picking it
up and really loving the tool.

The book is built so that it is just as easy to start in the 'Basic
Usage' section and go back later, but if you're going to sit down and
just start reading, I think it would be better to explain why Git is
different at a fundamental level right off the bat.

Scott

Re: Git Community Book

[Junio C Hamano]

"Scott Chacon" <schacon@gmail.com> writes:

>>
>> There is no license in the source code - what are the copying terms?
>>
>
> I copied in the COPYING file from Git - GPL2.
>
>> It is maybe somewhat unfortunate that this is in a different format that
>> the standard git choice asciidoc, but the formats do look rather similar
>> so I assume it should not be hard to even convert from one to another if
>> needed.
>
> I simply didn't want to get asciidoc working locally - it's always
> been a bit of a pain to compile (I've heard it referred to more than
> once as the only 'nightmare dependancy' in git), and I don't need to
> make man pages or anything, so it seemed Markdown would be a better
> choice for my output targets.  There are a number of good Markdown
> interpreters and they're easy to get running.

I personally like markdown, but doesn't your refusal to work with existing
practices pose a significant problem, unless:

 (0) you do not consider it a goal to keep the documentation shipped with
     git and your book in sync; or

 (1) you have either markdown to asciidoc (or the other way around)
     converter; the book is written in markdown, and its conversion back
     to asciidoc is fed to Documentation as patches (or the other way
     around); or

 (2) somebody tries to find markdown to manpage, and we convert
     Documentation/ to markdown.

Or is this, "fork once and borrow reviewer's time, but never be able to
contribute back to the original text because the result is so different"
approach?

Re: Git Community Book

[Julian Phillips]

On Tue, 29 Jul 2008, Junio C Hamano wrote:

> "Scott Chacon" <schacon@gmail.com> writes:
>
>>>
>>> There is no license in the source code - what are the copying terms?
>>>
>>
>> I copied in the COPYING file from Git - GPL2.
>>
>>> It is maybe somewhat unfortunate that this is in a different format that
>>> the standard git choice asciidoc, but the formats do look rather similar
>>> so I assume it should not be hard to even convert from one to another if
>>> needed.
>>
>> I simply didn't want to get asciidoc working locally - it's always
>> been a bit of a pain to compile (I've heard it referred to more than
>> once as the only 'nightmare dependancy' in git), and I don't need to
>> make man pages or anything, so it seemed Markdown would be a better
>> choice for my output targets.  There are a number of good Markdown
>> interpreters and they're easy to get running.
>
> I personally like markdown, but doesn't your refusal to work with existing
> practices pose a significant problem, unless:
>
> (0) you do not consider it a goal to keep the documentation shipped with
>     git and your book in sync; or
>
> (1) you have either markdown to asciidoc (or the other way around)
>     converter; the book is written in markdown, and its conversion back
>     to asciidoc is fed to Documentation as patches (or the other way
>     around); or
>
> (2) somebody tries to find markdown to manpage, and we convert
>     Documentation/ to markdown.

Haven't used it personally, and without commenting on the "political" side 
of such an approach - there does exist at least one tool that claims to be 
able to convert from markdown to man: http://johnmacfarlane.net/pandoc/

> Or is this, "fork once and borrow reviewer's time, but never be able to
> contribute back to the original text because the result is so different"
> approach?
>
> --
> To unsubscribe from this list: send the line "unsubscribe git" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>

-- 
Julian

  ---
[The French Riviera is] a sunny place for shady people.
 		-- Somerset Maugham

Re: Git Community Book

[Junio C Hamano]

Julian Phillips <julian@quantumfyre.co.uk> writes:

> On Tue, 29 Jul 2008, Junio C Hamano wrote:
>
>> "Scott Chacon" <schacon@gmail.com> writes:
>>
>>> I simply didn't want to get asciidoc working locally - it's always
>>> been a bit of a pain to compile (I've heard it referred to more than
>>> once as the only 'nightmare dependancy' in git), and I don't need to
>>> make man pages or anything, so it seemed Markdown would be a better
>>> choice for my output targets.  There are a number of good Markdown
>>> interpreters and they're easy to get running.
>>
>> I personally like markdown, but doesn't your refusal to work with existing
>> practices pose a significant problem, unless:
>> ...
>> (2) somebody tries to find markdown to manpage, and we convert
>>     Documentation/ to markdown.
> 
> Haven't used it personally, and without commenting on the "political"
> side of such an approach - there does exist at least one tool that
> claims to be able to convert from markdown to man:
> http://johnmacfarlane.net/pandoc/

Oh, there is nothing political about this.  It is not like some of us is
employed by AsciiDoc company and defecting to markdown would cost
somebody's job or life ;-)

It is good to know that an option is availble to make it easier to go
back-and-forth, when/if it becomes necessary.

Re: Git Community Book

[Scott Chacon]

On Tue, Jul 29, 2008 at 10:43 AM, Junio C Hamano <gitster@pobox.com> wrote:
> "Scott Chacon" <schacon@gmail.com> writes:
>
>> So I wanted to develop a really nice, easy to follow book for Git
>> newcomers to learn git quickly and easily.  One of the issues I
>> remember having when learning Git is that there is a lot of great
>> material in the User Guide, Tutorial, Tutorial 2, Everyday Git, etc -
>> but they're all huge long documents that are sometimes difficult to
>> come back to and remember where you were, and I didn't know which one
>> to start with or where to find what I was looking for, etc.
>
> Interesting.  A few comments, before I get dragged into my day job fully.
>
> [overall]
>
>  - Some people mentioned that the necessity of reading through large
>   volume of documentation can be reduced if they were divided by
>   developer roles (similar to how Everyday does), e.g. people in
>   individual contributor role does not have to learn integrator tools
>   such as "am" in their first pass on the documentation.  Has the
>   approach considered while developing this book?
>

Not really - I'm assuming that everyone will have to be one of those
roles at some point - I'm mostly aiming at the smaller developers like
myself, and probably 90% of the Git users, who have 20 git projects
that they work on with 1-5 other people.  I am not aiming at the Linux
or Git developers that have to deal with a project with hundreds of
users - everyone is going to have to be a developer, participant,
integrator and administrator to some degree, so I wanted to introduce
those commands when you need them.  IE, 'gc' and 'fsck' are rarely
_needed_ by most users - you can work just fine for a really long time
without ever needing to run them, but they are first in the Everyday
list.  I'm ordering it roughly in the order that I've seen people need
certain commands.  I could be convinced otherwise on any of them,
though.

>  - The order of sections in "Working with Git" chapter somehow does not
>   feel quite right, except that I'd agree that "Git on Windows" at the
>   beginning is a very good idea (disclaimer. I do not use Windows
>   myself). "StGIT" coming next was very understandable, but then
>   "Capistrano"????  And no CVS section next to Subversion section?  Ruby
>   before Perl or Python (I would have listed Perl, Python and then Ruby
>   to avoid language wars.  That's the language age order, and it is even
>   alphabetical)???
>

This is basically just notes at this point.  I will likely re-arrange
them as they are written.  However, I would argue that there are
likely more Git people using Ruby than there are using Python, though
Perl might rival it.  Nearly every major Ruby project out there is now
using Git, whereas very few Python ones seem to be (possibly because
Mercurial is written in python) - however, in all honesty, I don't
really care what order they are in.

As for the Capistrano section - again it is demand.  I have had tons
and tons of questions about Capistrano and Git, and many thousands of
people use that combination or are beginning to.  Again though, I
don't care where it is - I would be happy to put it at the bottom of
the section.

>   Above "Capistrano" and "Ruby" comment shows the bias this TOC has (and
>   my bias being different from the TOC's bias).  I'd imagine that
>   Ruby-minded folks won't share the same reaction as I had.  What's the
>   target audience of this book?  Git users in general, or primarily
>   Ruby-minded subset?  If the latter, labeling this as "Community Book"
>   may be misleading.

The target audience are users being convinced by their friends to use
Git and I want to impress them with a well thought out and laid out,
comprehensive, easy to use website and book as their first experience,
and show them an easy and smooth path to switch their mind from
thinking in SVN/Perforce to thinking in Git.  The Ruby community is a
very large part of the current surge to Git right now, but I want the
book to be easily accessible and acceptable to all communities that
are doing that.

> [http://book.git-scm.com/1_the_git_object_database.html]
>
>  - The color of "blob" does not match the blob that is committed to eat
>   trees at the top of your site ;-)
>
>  - In a recent thread on the list, quite a lot of people seem to have
>   found that teaching the low level details and plumbing first to the new
>   people is detrimental.  Do you have response to that thread?
>

I think I addressed this in a previous response.  As for the blob
color, a number of diagrams I am planning to introduce initially are
from a talk I gave at RailsConf on Git, and I will likely go back over
them a bit later.

Thanks,
Scott

Re: Git Community Book

[Scott Chacon]

On Tue, Jul 29, 2008 at 11:25 AM, Junio C Hamano <gitster@pobox.com> wrote:
>
> (cont'd)
>
> I was planning to comment on the contents (i.e. text) but it appears that
> most of the stuff was borrowed from the User Manual, so I won't.
>
>        Side note: I wonder if this makes the whole "Community Book"
>        GPLv2.  What would happen to the part that includes your own
>        screencast?  You do not mind it be contaminated by our licence?
>
> But there seems to be some stuff User Manual does not talk about.

Yes, I'm using text from the User Manual and beginning to convert a
bunch of the text to make more sense in the new context, but it is
still under GPL2 and the screencasts are being linked to, not included
and distributed, so they shouldn't be affected.  However, they are
MIT, so you can pretty much do whatever you want with them.

> [3_basic_branching_and_merging.html]
>
>  - You've talked about low-level individual objects in an earlier section
>   but you stopped at showing a single commit pointing at a tree.  People
>   would find branching and merging very hard to get, without
>   understanding the commit DAG.  On the other hand, you can explain
>   commit DAG without going into details down to trees and blobs in the
>   earlier section.  The user manual has "understanding reachability"
>   section early on for this exact reason.
>
> [5_creating_new_empty_branches.html]
>
>  - As I repeatedly said on the list, I do not think teaching this is
>   useful.  Multiple roots may happen as a result of pushing (or pulling)
>   from a repository with unrelated root, but it is not something you
>   would want to actively aim for.  At least there needs an explanation
>   for the reason why making disjoint roots in the same repository is
>   (sometimes) a good thing to do, and what its downsides are.
>

Thank you for your feedback, I'll try to address both of these points
as I revise the book.

Scott

Re: Git Community Book

[Scott Chacon]

On Tue, Jul 29, 2008 at 11:42 AM, Junio C Hamano <gitster@pobox.com> wrote:
> "Scott Chacon" <schacon@gmail.com> writes:
>
>>>
>>> There is no license in the source code - what are the copying terms?
>>>
>>
>> I copied in the COPYING file from Git - GPL2.
>>
>>> It is maybe somewhat unfortunate that this is in a different format that
>>> the standard git choice asciidoc, but the formats do look rather similar
>>> so I assume it should not be hard to even convert from one to another if
>>> needed.
>>
>> I simply didn't want to get asciidoc working locally - it's always
>> been a bit of a pain to compile (I've heard it referred to more than
>> once as the only 'nightmare dependancy' in git), and I don't need to
>> make man pages or anything, so it seemed Markdown would be a better
>> choice for my output targets.  There are a number of good Markdown
>> interpreters and they're easy to get running.
>
> I personally like markdown, but doesn't your refusal to work with existing
> practices pose a significant problem, unless:
>
>  (0) you do not consider it a goal to keep the documentation shipped with
>     git and your book in sync; or
>
>  (1) you have either markdown to asciidoc (or the other way around)
>     converter; the book is written in markdown, and its conversion back
>     to asciidoc is fed to Documentation as patches (or the other way
>     around); or
>
>  (2) somebody tries to find markdown to manpage, and we convert
>     Documentation/ to markdown.
>
> Or is this, "fork once and borrow reviewer's time, but never be able to
> contribute back to the original text because the result is so different"
> approach?
>

The book is basically a fork of all three of the guides I mentioned
(User Manual and both Tutorials), and with the scope and goals I
currently have in mind, will not be kept in sync - it's just not going
to be possible.  I think in the end, the goals of the texts are so
very different that sections it will simply not make sense to try to
keep them in sync in some sort of automated fashion.  That's one of
the reasons why I choose Markdown - I saw no need to use asciidoc, as
the book will not be shipped around with Git or built using the same
processes, and I had no need for the advantages of asciidoc in my
project.  I don't think it makes much sense to have the book be a man
page at all.

However, I will watch the manual and guides and try to incorporate
changes to them as appropriate, and I will likely have some updates to
them myself as I've been more closely scrutinizing them.

Scott

Re: Git Community Book

[Junio C Hamano]

"Scott Chacon" <schacon@gmail.com> writes:

> The book is basically a fork of all three of the guides I mentioned
> (User Manual and both Tutorials), and with the scope and goals I
> currently have in mind, will not be kept in sync - it's just not going
> to be possible.  I think in the end, the goals of the texts are so
> very different that sections it will simply not make sense to try to
> keep them in sync in some sort of automated fashion.  That's one of
> the reasons why I choose Markdown - I saw no need to use asciidoc, as
> the book will not be shipped around with Git or built using the same
> processes, and I had no need for the advantages of asciidoc in my
> project.  I don't think it makes much sense to have the book be a man
> page at all.
>
> However, I will watch the manual and guides and try to incorporate
> changes to them as appropriate, and I will likely have some updates to
> them myself as I've been more closely scrutinizing them.

If that is the approach you decided for your book, I am Ok with that.

Not that you need my blessing to do your own book.  It was unclear what
your goals were, and if one of the goals were to keep the hassle of
maintaining shared materials in both manuals up-to-date, choice of
markdown seemed suboptimal to me, hence my comments.

Thanks.

... /me goes back to work after lunch break ...

Re: Git Community Book

[Daniel Barkalow]

On Tue, 29 Jul 2008, Scott Chacon wrote:

> So I wanted to develop a really nice, easy to follow book for Git
> newcomers to learn git quickly and easily.  One of the issues I
> remember having when learning Git is that there is a lot of great
> material in the User Guide, Tutorial, Tutorial 2, Everyday Git, etc -
> but they're all huge long documents that are sometimes difficult to
> come back to and remember where you were, and I didn't know which one
> to start with or where to find what I was looking for, etc.

It would be good to include stuff from 
http://eagain.net/articles/git-for-computer-scientists/

Maybe only in inspiration, since it doesn't have an obvious license and 
it's stylisticly more technical. But it would be nice to have diagrams of 
"this is what git thinks of as history", possibly even arranging them like 
gitk shows things (older downward, refs pointing in from the side).

In particular, I think it's really useful to show a commit graph with 
branching and merging, and introduce refs as movable pointers to commits 
in the graph, and local branches as refs that you move and tracking refs 
as refs that copy values in other repositories.

I think you can even gloss of details of blobs and trees because they 
pretty much work just like files and directories in a filesystem (except 
that they take up much less storage in large quantities than you'd think). 
The only potentially interesting things are (1) a blob names the inode, 
not the dentry, so it's the file contents, not the name, mode, etc; and 
(2) the permission bits are just 'x', we've got symlinks, there are no 
owner/group or other attributes and "see also Submodules".

But I think that the section:
  http://eagain.net/articles/git-for-computer-scientists/#history
should have an equivalent in any git documentation that can have diagrams, 
and introducing a history diagram style early means that you can do a 
bunch of simple pictures to explain operations like "git checkout -b foo" 
or "git reset --hard HEAD^^" or "git checkout origin/master".

	-Daniel
*This .sig left intentionally blank*

Re: Git Community Book

[Junio C Hamano]

Daniel Barkalow <barkalow@iabervon.org> writes:

> In particular, I think it's really useful to show a commit graph with 
> branching and merging, and introduce refs as movable pointers to commits 
> in the graph, and local branches as refs that you move and tracking refs 
> as refs that copy values in other repositories.

I'd very strongly second this.  If somebody is really into screencasts
(and especially from the Ruby circle, I would guess), this may be worth
a look:

    http://excess.org/article/2008/07/ogre-git-tutorial/

I saw a couple of technical inaccuracies in the presentation (I do not
expect any presentation or screencast to be perfect; I've never seen one
without any technical error anyway, perhaps other than my own at OLS a few
years ago), but otherwise it was very well done.  Espcially the part that
builds the commit ancestry chains I was very happy to see it taught like
so.

Re: Git Community Book

[Bart Trojanowski]

* Junio C Hamano <gitster@pobox.com> [080729 18:48]:
> I'd very strongly second this.  If somebody is really into screencasts
> (and especially from the Ruby circle, I would guess), this may be worth
> a look:
> 
>     http://excess.org/article/2008/07/ogre-git-tutorial/

Thank you very much for the plug, Junio.

> I saw a couple of technical inaccuracies in the presentation (I do not
> expect any presentation or screencast to be perfect; I've never seen one
> without any technical error anyway, perhaps other than my own at OLS a few
> years ago)

Could you let me know what the biggest inaccuracies were?  I would like
to correct my mistakes and update the slides.

Cheers,
-Bart

-- 
				WebSig: http://www.jukie.net/~bart/sig/

Re: Git Community Book

[Bart Trojanowski]

* Junio C Hamano <gitster@pobox.com> [080729 18:48]:
> I'd very strongly second this.  If somebody is really into screencasts
> (and especially from the Ruby circle, I would guess), this may be worth
> a look:
> 
>     http://excess.org/article/2008/07/ogre-git-tutorial/

BTW, if anyone is interested in the SVGs used for the slides...

        git clone git://tachyon.jukie.net/intro-to-git.git/

You will need inkscape and latex-beamer to build the PDF.

Feel free to use them as you wish with attribution.  Although my may
want to wait till I fix the mistakes that Junio mentioned.

-Bart

-- 
				WebSig: http://www.jukie.net/~bart/sig/

Re: Git Community Book

[Junio C Hamano]

Bart Trojanowski <bart@jukie.net> writes:

> Could you let me know what the biggest inaccuracies were?  I would like
> to correct my mistakes and update the slides.

The one I offhand can recall was in your spoken part not on slides ("git
add -u" does not notice new files but does notice removed ones).  Nothing
major, really.

Re: Git Community Book

[J. Bruce Fields]

> > So my confusion still is - where does this stand wrt. the user manual?
> > Why didn't you just start with the manual and work on that? I thought
> > you were planning to do that, but apparently we misunderstood each other
> > in the last mails.
> >
On Tue, Jul 29, 2008 at 11:30:55AM -0700, Scott Chacon wrote:
> 
> I was originally planning on doing that, but the problem is the
> graphics, diagrams and screencasts.  Unless I am mistaken, there is
> not a single outside media reference in any of these guides - the
> diagrams that are there are all ascii drawings.  I'm assuming there is
> a reason for that. If I wanted to add images and screencast embeds
> into the guide, how would that work?
> 

Yeah, some possible obstacles:

	- Size: People probably won't want large binary blobs added to
	  the git repository.
	- Editability: We want to be able to keep the materials up to
	  date and accurate.
	- Source readability: the current documentation can all be read
	  in place without doing a build.
	- Build requirements: I seem to recall complaints about the
	  toolchain required to build the existing documentation.

At least for simple diagrams it might be possible to solve most of those
problems with an appropriate diagram-description-language that could be
compiled into image files.  Screencasts are probably totally out,
though.

In cases where you do find you're working with the same material, any
improvements you could contribute back to the in-tree documentation
would of course be appreciated.

> Well, that's what the point of this is - to ask everyone to help me
> review it, and possibly help me add to it.  The user manual is great,
> but even I don't reference it very often because I find it difficult
> to find content in it I need quickly.

If you had notes on any particular examples (I looked for X in place Y,
then place Z, and finally found it where I least expected it in place
Q...), they'd be appreciated.

> The specific order I choose is very different from the User Guide and
> is likely to bother a number of people, which you mentioned (and I'm
> sure Dscho will _hate_) because I introduce the object model at the
> beginning.  (I'm still working on that section, trying to simplify it
> and add in some other diagrams and a short screencast I have that I
> think will be helpful)  This is because I have had a lot of positive
> feedback that primary frustration from people comes from them thinking
> of Git as a super-better Subversion.
>
> I would venture to say that
> _most_ of the users coming to Git now are currently fluent in
> Subversion.  Even if they are from Perforce or CVS (the other two ones
> I will occasionally run into), their mental model of what an SCM does
> is the same - delta storage.  I've found that by ridding them of that
> notion off the bat, they have _far_ fewer problems and frustrations
> with Git than when I just try to show them the first 10 commands in
> sort of a cookbook style.  It's not a complicated model, it doesn't
> take long to teach, and in _my personal_ experience (which is not to
> say it's necessarily correct), it helps people the most in picking it
> up and really loving the tool.

I've considered doing the same for the user manual, actually, for some
of the same reasons--my main concern would be that it be done very
quickly, so as not to make people feel like it was a big obstacle on
their way to actually doing what they need to do.

So, anyway, that's to say that suggestions for reorganization of the
in-tree documentation (as opposed to just smaller-scale fixes) would
also be welcomed....

--b.

> 
> The book is built so that it is just as easy to start in the 'Basic
> Usage' section and go back later, but if you're going to sit down and
> just start reading, I think it would be better to explain why Git is
> different at a fundamental level right off the bat.
> 
> Scott
> --
> To unsubscribe from this list: send the line "unsubscribe git" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Git Community Book

[Scott Chacon]

Hey all,

I just wanted to let those of you who are interested know that I've
been making a lot of progress on the Git Community Book
(http://book.git-scm.com)  I was wondering if anyone was interested in
helping me with a few parts.  For one, there are some sections that I
personally have very little experience with, and was looking for some
notes/blog posts/personal experiences on, namely Advanced History
Modification (filter-branch, advanced rebasing, etc), Corruption
Recovery, Branch Tracking, Subversion Integration, Git with
Perl/Python/PHP, and Using Git with Editors (especially
NetBeans/Eclipse).

Also, the last section of the book is on some of the plumbing - mostly
stuff I've found difficult to pick up with the existing documentation
while re-implementing stuff in Ruby.  I would really appreciate it if
someone could proofread some of these chapters for errors:

http://book.git-scm.com/7_the_packfile.html
http://book.git-scm.com/7_raw_git.html
http://book.git-scm.com/7_transfer_protocols.html

Some of the next things I'm interested in producing is a cookbook
style guide and some searching tools for all the online documentation,
just to keep everyone up to date on where I'm going with the project.
Also, there is now a simple PDF downloadable version of the book
available and being kept up to date with the html version.

Thanks,
Scott

Re: Git Community Book

[Thomas Adam]

2008/9/5 Scott Chacon <schacon@gmail.com>:
> Hey all,
>
> I just wanted to let those of you who are interested know that I've
> been making a lot of progress on the Git Community Book
> (http://book.git-scm.com)  I was wondering if anyone was interested in

I'm going to bite and ask the obvious questions:

1.  How does what you're producing differ from the current Git Users' Manual?
2.  Is this project of yours aiming to obsolete the Git Users' Manual
with "official" sanctioning from people involved with Git?
3.  Assuming 2 is a "no", patches to the Users' Guide would be nice.  :)

-- Thomas Adam

Re: Git Community Book

[Junio C Hamano]

"Scott Chacon" <schacon@gmail.com> writes:

> Also, the last section of the book is on some of the plumbing - mostly
> stuff I've found difficult to pick up with the existing documentation
> while re-implementing stuff in Ruby.  I would really appreciate it if
> someone could proofread some of these chapters for errors:
>
> http://book.git-scm.com/7_the_packfile.html

Nice pictures.  You might also want to know that code for reading pack idx
version 2 was backported to 1.4.4.5 for people who are stuck on 1.4.4
series for whatever reason.

What is the target audience of this section?  If it is written for a mere
curious type, or if it is written to give "here is the general idea, for
more details read the source", the level of detail here would be Ok.

If you are writing for people who want to (re)implement something that
produces these files, you might want to at least say that offset/sha1[]
table is sorted by sha1[] values (this is to allow binary search of this
table), and fanout[] table points at the offset/sha1[] table in a specific
way (so that part of the latter table that covers all hashes that start
with a given byte can be found to avoid 8 iterations of the binary
search).

<data> part is just zlib stream for non-delta object types; for the two
delta object representations, the <data> portion contains something that
identifies which base object this delta representation depends on, and the
delta to apply on the base object to resurrect this object.  ref-delta
uses 20-byte hash of the base object at the beginning of <data>, while
ofs-delta stores an offset within the same packfile to identify the base
object.  In either case, two important constraints a reimplementor must
adhere to are:

 * delta representation must be based on some other object within the same
   packfile;

 * the base object must be of the same underlying type (blob, tree, commit
   or tag);

> http://book.git-scm.com/7_raw_git.html

I am guessing this is for Porcelain writers who use plumbing.  Please
don't teach echoing into .git/refs/...  but DO teach using update-ref with
the -m option.  We do not want people's random Porcelains flipping the tip
of branches without leaving trail in reflog for users to use to recover
from mistakes.

Re: Git Community Book

[Linus Torvalds]

On Fri, 5 Sep 2008, Scott Chacon wrote:
> 
> http://book.git-scm.com/7_the_packfile.html 

The checksums in the index file "trailers" are all claiming to be 4 bytes, 
and that's wrong - they're full SHA1 sums at 20 bytes each.

The v2 pack-file _also_ has per-object CRC's, and those are indeed just 4 
bytes each, and are correctly listed as such.

The pack-file itself also has a few more things there, it's not just the 
"PACK" string and then the objects. It has two more 32-bit words: a pack 
file version number and the number of entries in the pack-file (all 
network byte order). It also has its own checksum at the end (20-byte SHA1 
again).

But looks good otherwise from a quick look.

		Linus

Re: Git Community Book

[Scott Chacon]

On Fri, Sep 5, 2008 at 12:15 PM, Thomas Adam <thomas.adam22@gmail.com> wrote:
> 2008/9/5 Scott Chacon <schacon@gmail.com>:
>> Hey all,
>>
>> I just wanted to let those of you who are interested know that I've
>> been making a lot of progress on the Git Community Book
>> (http://book.git-scm.com)  I was wondering if anyone was interested in
>
> I'm going to bite and ask the obvious questions:

Just for reference, a lot of this was discussed here a while back:

http://thread.gmane.org/gmane.comp.version-control.git/90653

however, I would be happy to answer these for you.

>
> 1.  How does what you're producing differ from the current Git Users' Manual?

I'm going for a different audience with this project.  I'd like for it
to be a lot more user-friendly, easily digestible, and to include
images, diagrams and screencasts.

> 2.  Is this project of yours aiming to obsolete the Git Users' Manual
> with "official" sanctioning from people involved with Git?

I think there will be people who prefer the Users Manual format, who
think screencasts are wussy :)
Also, I'm not sure an "official" sanctioning would do much of anything
- because of the images and screencasts, this will never be included
in the git source like the UM is, but it's also open source so if
people want to take content from it to improve the UM, that's cool.

> 3.  Assuming 2 is a "no", patches to the Users' Guide would be nice.  :)

I would love to do this, but I don't know what exactly the community
thinks is missing/lacking.  My ideas about what is helpful is rarely
the same as the git lists :)  However, if someone pointed to one of
the chapters I wrote and said "that would be great in the UM", I would
happily convert it.

Scott

Re: Git Community Book

[Scott Chacon]

On Fri, Sep 5, 2008 at 12:41 PM, Junio C Hamano <gitster@pobox.com> wrote:
> "Scott Chacon" <schacon@gmail.com> writes:
>
>> Also, the last section of the book is on some of the plumbing - mostly
>> stuff I've found difficult to pick up with the existing documentation
>> while re-implementing stuff in Ruby.  I would really appreciate it if
>> someone could proofread some of these chapters for errors:
>>
>> http://book.git-scm.com/7_the_packfile.html
>
> Nice pictures.  You might also want to know that code for reading pack idx
> version 2 was backported to 1.4.4.5 for people who are stuck on 1.4.4
> series for whatever reason.
>
> What is the target audience of this section?  If it is written for a mere
> curious type, or if it is written to give "here is the general idea, for
> more details read the source", the level of detail here would be Ok.
>
> If you are writing for people who want to (re)implement something that
> produces these files, you might want to at least say that offset/sha1[]
> table is sorted by sha1[] values (this is to allow binary search of this
> table), and fanout[] table points at the offset/sha1[] table in a specific
> way (so that part of the latter table that covers all hashes that start
> with a given byte can be found to avoid 8 iterations of the binary
> search).
>
> <data> part is just zlib stream for non-delta object types; for the two
> delta object representations, the <data> portion contains something that
> identifies which base object this delta representation depends on, and the
> delta to apply on the base object to resurrect this object.  ref-delta
> uses 20-byte hash of the base object at the beginning of <data>, while
> ofs-delta stores an offset within the same packfile to identify the base
> object.  In either case, two important constraints a reimplementor must
> adhere to are:
>
>  * delta representation must be based on some other object within the same
>   packfile;
>
>  * the base object must be of the same underlying type (blob, tree, commit
>   or tag);
>
>> http://book.git-scm.com/7_raw_git.html
>
> I am guessing this is for Porcelain writers who use plumbing.  Please
> don't teach echoing into .git/refs/...  but DO teach using update-ref with
> the -m option.  We do not want people's random Porcelains flipping the tip
> of branches without leaving trail in reflog for users to use to recover
> from mistakes.
>

I've implemented all of these and Linus's fixes and suggestions.
Thanks for the feedback.

To answer your earlier question, these docs are basically for people
working on bindings/re-implementations in other languages, since there
is no real linked library available yet, as a primer before they dig
into the source, or possibly so they don't have to.

I'm not fantastic at C, so it took me a while in some cases - figuring
out that the size listed in the object header was not the actual size
of the data, but the size of it when expanded, for example, was not
very easy to do.  I've been doing a lot of work on re-implementations
in Ruby and ObjC because I can't easily make real bindings, so I
thought I would add things that I could not easily find in the docs
for others that are trying in other languages.

If you want, I could create a patch for any of this stuff to
Documentation/ (that goes for the whole book), but someone will have
to tell me which parts might be useful to add.

Thanks again for taking the time!
Scott

Re: Git Community Book

[Felipe Contreras]

On Sat, Sep 6, 2008 at 12:34 AM, Scott Chacon <schacon@gmail.com> wrote:
> On Fri, Sep 5, 2008 at 12:41 PM, Junio C Hamano <gitster@pobox.com> wrote:
>> "Scott Chacon" <schacon@gmail.com> writes:
>>
>>> Also, the last section of the book is on some of the plumbing - mostly
>>> stuff I've found difficult to pick up with the existing documentation
>>> while re-implementing stuff in Ruby.  I would really appreciate it if
>>> someone could proofread some of these chapters for errors:
>>>
>>> http://book.git-scm.com/7_the_packfile.html
>>
>> Nice pictures.  You might also want to know that code for reading pack idx
>> version 2 was backported to 1.4.4.5 for people who are stuck on 1.4.4
>> series for whatever reason.
>>
>> What is the target audience of this section?  If it is written for a mere
>> curious type, or if it is written to give "here is the general idea, for
>> more details read the source", the level of detail here would be Ok.
>>
>> If you are writing for people who want to (re)implement something that
>> produces these files, you might want to at least say that offset/sha1[]
>> table is sorted by sha1[] values (this is to allow binary search of this
>> table), and fanout[] table points at the offset/sha1[] table in a specific
>> way (so that part of the latter table that covers all hashes that start
>> with a given byte can be found to avoid 8 iterations of the binary
>> search).
>>
>> <data> part is just zlib stream for non-delta object types; for the two
>> delta object representations, the <data> portion contains something that
>> identifies which base object this delta representation depends on, and the
>> delta to apply on the base object to resurrect this object.  ref-delta
>> uses 20-byte hash of the base object at the beginning of <data>, while
>> ofs-delta stores an offset within the same packfile to identify the base
>> object.  In either case, two important constraints a reimplementor must
>> adhere to are:
>>
>>  * delta representation must be based on some other object within the same
>>   packfile;
>>
>>  * the base object must be of the same underlying type (blob, tree, commit
>>   or tag);
>>
>>> http://book.git-scm.com/7_raw_git.html
>>
>> I am guessing this is for Porcelain writers who use plumbing.  Please
>> don't teach echoing into .git/refs/...  but DO teach using update-ref with
>> the -m option.  We do not want people's random Porcelains flipping the tip
>> of branches without leaving trail in reflog for users to use to recover
>> from mistakes.
>>
>
> I've implemented all of these and Linus's fixes and suggestions.
> Thanks for the feedback.
>
> To answer your earlier question, these docs are basically for people
> working on bindings/re-implementations in other languages, since there
> is no real linked library available yet, as a primer before they dig
> into the source, or possibly so they don't have to.
>
> I'm not fantastic at C, so it took me a while in some cases - figuring
> out that the size listed in the object header was not the actual size
> of the data, but the size of it when expanded, for example, was not
> very easy to do.  I've been doing a lot of work on re-implementations
> in Ruby and ObjC because I can't easily make real bindings, so I
> thought I would add things that I could not easily find in the docs
> for others that are trying in other languages.

I have experience mixing C and Ruby code if you are interested, it's
actually quite easy.

I also think a shared library would make sense.

Keep up the good work ;)

-- 
Felipe Contreras

Re: Git Community Book

[Stephan Beyer]

Hi,

Scott Chacon wrote:
> I just wanted to let those of you who are interested know that I've
> been making a lot of progress on the Git Community Book
> (http://book.git-scm.com)  I was wondering if anyone was interested in
> helping me with a few parts.

I just had a very quick look over the PDF, meaning only looking at
pictures and headlines.

Just nitpicking about one thing:
I was wondering if "Stash Queue" is the right headline, because I
usually use

	git stash save	# oh, an interrupt, have to do something else now

and after this is done:

	git stash pop	# back to the real work

And if you are interrupted in an interrupt, you want the last stash
being the first one to pop, which is a stack-like (last in, first out)
behavior.

Of course, there may be cases where you want the queuing behavior that
you advertise in the book.
I use it rather seldomly. But perhaps it is just me :-)

Regards,
  Stephan

-- 
Stephan Beyer <s-beyer@gmx.net>, PGP 0x6EDDD207FCC5040F

Re: Git Community Book

[Shawn O. Pearce]

Scott Chacon <schacon@gmail.com> wrote:
> On Fri, Sep 5, 2008 at 12:41 PM, Junio C Hamano <gitster@pobox.com> wrote:
> > "Scott Chacon" <schacon@gmail.com> writes:
> >
> >> Also, the last section of the book is on some of the plumbing - mostly
> >> stuff I've found difficult to pick up with the existing documentation
> >> while re-implementing stuff in Ruby.  I would really appreciate it if
> >> someone could proofread some of these chapters for errors:
> >>
> >> http://book.git-scm.com/7_the_packfile.html

OK, time for me to throw in comments.  ;-)

I do like this book, its organized and concise.  Thanks for doing it.


http://book.git-scm.com/7_how_git_stores_objects.html:

The loose object formatting of

 header = "#{type} #{size}#body"
 store = header + content

I can't read Ruby so I'm not sure what the header value computes
out to here.  #body should be a \0.  I'm also not sure that the
prior line setting size = content.length.to_s is very clear for
the non-Ruby people to understand how a size is formatted.

If the code shown here is the Ruby implementation I'm a little
concerned about it writing directly into the loose object.  If the
write is partial then you have a partial object which is at the
right name, but is unusable.  That can give you corruption that
is difficult to track down and fix.  C Git and JGit both write
to temporary files then atomically move the temporary file into
position under its proper name only after it has been fully written.

If an implementor is implementing they should be offered this advice,
and probably do so right here in this section of the book.

"When objects are written to disk, it is often in the loose format,
since that format is less expensive to access."

I'm not sure that statement is true.  Access from packs tends
to scream compared to access from loose objects.  The overheads
of opening and closing the file descriptors, even on Linux, is
what kills performance for data access.  However Git writes to
loose objects first and packs later for _safety_ not efficiency.
Although it is a lot more efficient to write a 2 KB loose object
and avoid rewriting a 50 MB pack, but its also less likely to fail
and make you lose your work.


http://book.git-scm.com/7_the_git_index.html:

I wouldn't say that the index stores permissions.  More like it
stores the "class" or "type" of the thing located at that path.
There are 4 major classes:

	- regular file
	- executable file
	- symbolic link
	- git submodule

The 5th class is the subtree, but only appears in trees and not
in the index since the index file is actually flat.


http://book.git-scm.com/7_the_packfile.html:

You should probably point out that the .idx file uses network byte
order for the numeric fields like the version number and the file
offsets.

I'd also point out that the offsets in index v1 are unsigned and
from the start of the pack file.  The offsets in index v2 are
also unsigned, but the 1<<31 is tested in the 32 bit offset to
see if a 64 bit offset is used.  The algorithm there is:

	if offset32 & 1<<31:
		offset = ofs64_table[offset32 & ~(1<<31)]
	else
		offset = offset32

Its also rather unclear how the fan out table can be used to limit
the binary search.  What you are missing is describing that fanout[X]
holds the number of objects whose first byte of their SHA-1 is <= X.
Hence fanout[0] has the number of objects whose SHA-1 starts with
"00" and fanout[0x15] has the number of objects whose SHA-1 starts
with "15", "14", "13", ..., "00".  Thus fanout[0xff] has the total
number of objects in the pack.

In the pack file section I'd also point out the version and entry
count are unsigned network byte order.  This is not clear from the
Ruby code, although one can guess at it if one knows the git.git
code very very well (like I do).

"After that, you get a series of packed objects, in order of thier SHAs"

Aside from s/thier/their/ this is not a correct statement _AT ALL_.

The ordering of objects in the packfile is very carefully planned
by the packer to maximize data locality from most recent -> least
recent information, making the most recent revisions of a project
the fastest to access.  This has _NOTHING_ to do with their SHA-1
names.

Technically a pack may store objects in any random order.  Heck,
you can wire up an RNG to the packer to always produce a different
ordering each time you pack.  Practically an implementation shouldn't
be that stupid and should instead try to order objects by recency,
like git.git and JGit both do.

"At the end of the packfile is a 20-byte SHA1 sum of all the shas
(in sorted order) in that packfile."

Also incorrect.  The 20-byte checksum at the end of the pack file
is a checksum of all bytes preceeding the checksum itself.  We use
it as an end-to-end data integrity check, especially on the network
transport to verify that every bit sent by the one side is received
correctly on the other side.

BTW, can I just say, I love the graphics in this book.  They are
quite well done.  Very worthwhile.


http://book.git-scm.com/7_transfer_protocols.html:

You might as well explain that the stream returned by upload-pack
uses the same 4 byte line length framing to form "packets", with
the 5th byte (really first byte of the payload) indicating the
"stream":

	- stream 1 ('\001') is the PACK data
	- stream 2 ('\002') is progress data/information
	- stream 3 ('\003') is the OH S**T we are aborting, died, dead

You may also want to explain that the way you know the end of the
pack is to read the header, get the entry count, and then read that
many objects from the stream, and then verify the pack checksum.

-- 
Shawn.

Re: Git Community Book

[Scott Chacon]

Thanks a ton for this, I'll incorporate all of this.

On Fri, Sep 5, 2008 at 11:33 PM, Shawn O. Pearce <spearce@spearce.org> wrote:
> Scott Chacon <schacon@gmail.com> wrote:
>> On Fri, Sep 5, 2008 at 12:41 PM, Junio C Hamano <gitster@pobox.com> wrote:
>> > "Scott Chacon" <schacon@gmail.com> writes:
>> >
>> >> Also, the last section of the book is on some of the plumbing - mostly
>> >> stuff I've found difficult to pick up with the existing documentation
>> >> while re-implementing stuff in Ruby.  I would really appreciate it if
>> >> someone could proofread some of these chapters for errors:
>> >>
>> >> http://book.git-scm.com/7_the_packfile.html
>
> OK, time for me to throw in comments.  ;-)
>
> I do like this book, its organized and concise.  Thanks for doing it.
>
>
> http://book.git-scm.com/7_how_git_stores_objects.html:
>
> The loose object formatting of
>
>  header = "#{type} #{size}#body"
>  store = header + content
>
> I can't read Ruby so I'm not sure what the header value computes
> out to here.  #body should be a \0.  I'm also not sure that the
> prior line setting size = content.length.to_s is very clear for
> the non-Ruby people to understand how a size is formatted.
>

Sorry, the markdown thingy is translating all the '\0's to '#body' for
some freaking reason unless I write it as '\\0'.  I'll fix this - it's
difficult for me to find these sometimes.  As for the rest of the ruby
stuff, I think I'll add some comments.

> If the code shown here is the Ruby implementation I'm a little
> concerned about it writing directly into the loose object.  If the
> write is partial then you have a partial object which is at the
> right name, but is unusable.  That can give you corruption that
> is difficult to track down and fix.  C Git and JGit both write
> to temporary files then atomically move the temporary file into
> position under its proper name only after it has been fully written.

That is a good idea - I don't do it that way and I certainly will
change the implementation to do so and modify these docs to reflect
that advice.

> "When objects are written to disk, it is often in the loose format,
> since that format is less expensive to access."
>
> I'm not sure that statement is true.  Access from packs tends
> to scream compared to access from loose objects.  The overheads
> of opening and closing the file descriptors, even on Linux, is
> what kills performance for data access.  However Git writes to
> loose objects first and packs later for _safety_ not efficiency.
> Although it is a lot more efficient to write a 2 KB loose object
> and avoid rewriting a 50 MB pack, but its also less likely to fail
> and make you lose your work.

Thanks for the clarification.  I write to loose objects first largely
because it's so much easier to do.  But also because I don't mmap
objects, so packfile access is not faster for implementations that
can't do that very well.  Also, I had originally meant "less expensive
to write", but I can see that is not clear.


> http://book.git-scm.com/7_the_git_index.html:
>
> I wouldn't say that the index stores permissions.  More like it
> stores the "class" or "type" of the thing located at that path.
> There are 4 major classes:
>
>        - regular file
>        - executable file
>        - symbolic link
>        - git submodule
>
> The 5th class is the subtree, but only appears in trees and not
> in the index since the index file is actually flat.

Interesting.  This documentation is actually from the User Manual -
I'll update this chapter first and if it looks better, I'll submit a
patch to the UM, too.

> http://book.git-scm.com/7_the_packfile.html:
>
> You should probably point out that the .idx file uses network byte
> order for the numeric fields like the version number and the file
> offsets.

Will do.

>
> I'd also point out that the offsets in index v1 are unsigned and
> from the start of the pack file.  The offsets in index v2 are
> also unsigned, but the 1<<31 is tested in the 32 bit offset to
> see if a 64 bit offset is used.  The algorithm there is:
>
>        if offset32 & 1<<31:
>                offset = ofs64_table[offset32 & ~(1<<31)]
>        else
>                offset = offset32
>
> Its also rather unclear how the fan out table can be used to limit
> the binary search.  What you are missing is describing that fanout[X]
> holds the number of objects whose first byte of their SHA-1 is <= X.
> Hence fanout[0] has the number of objects whose SHA-1 starts with
> "00" and fanout[0x15] has the number of objects whose SHA-1 starts
> with "15", "14", "13", ..., "00".  Thus fanout[0xff] has the total
> number of objects in the pack.
>
> In the pack file section I'd also point out the version and entry
> count are unsigned network byte order.  This is not clear from the
> Ruby code, although one can guess at it if one knows the git.git
> code very very well (like I do).
>
> "After that, you get a series of packed objects, in order of thier SHAs"
>
> Aside from s/thier/their/ this is not a correct statement _AT ALL_.
>
> The ordering of objects in the packfile is very carefully planned
> by the packer to maximize data locality from most recent -> least
> recent information, making the most recent revisions of a project
> the fastest to access.  This has _NOTHING_ to do with their SHA-1
> names.
>
> Technically a pack may store objects in any random order.  Heck,
> you can wire up an RNG to the packer to always produce a different
> ordering each time you pack.  Practically an implementation shouldn't
> be that stupid and should instead try to order objects by recency,
> like git.git and JGit both do.
>
> "At the end of the packfile is a 20-byte SHA1 sum of all the shas
> (in sorted order) in that packfile."
>
> Also incorrect.  The 20-byte checksum at the end of the pack file
> is a checksum of all bytes preceeding the checksum itself.  We use
> it as an end-to-end data integrity check, especially on the network
> transport to verify that every bit sent by the one side is received
> correctly on the other side.
>

I'm an idiot.  I say this because I actually implemented a bunch of
this stuff (in Ruby) and ran into most of these issues when trying to
implement it.  So I knew these things not 3 weeks ago, but I still
wrote it this way.  Dur.  Thanks for the corrections, I'll update
everything accordingly.

> BTW, can I just say, I love the graphics in this book.  They are
> quite well done.  Very worthwhile.

Thanks.

>
>
> http://book.git-scm.com/7_transfer_protocols.html:
>
> You might as well explain that the stream returned by upload-pack
> uses the same 4 byte line length framing to form "packets", with
> the 5th byte (really first byte of the payload) indicating the
> "stream":
>
>        - stream 1 ('\001') is the PACK data
>        - stream 2 ('\002') is progress data/information
>        - stream 3 ('\003') is the OH S**T we are aborting, died, dead
>
> You may also want to explain that the way you know the end of the
> pack is to read the header, get the entry count, and then read that
> many objects from the stream, and then verify the pack checksum.
>
> --
> Shawn.
>

Thanks again for all the time it must have taken to review all of this
- I'll make sure it gets into the book, and where appropriate, back
into the UM or other internal git docs.

Scott

Re: Git Community Book

[Christos Τrochalakis]

On Fri, Sep 5, 2008 at 10:08 PM, Scott Chacon <schacon@gmail.com> wrote:
> Hey all,
>
> I just wanted to let those of you who are interested know that I've
> been making a lot of progress on the Git Community Book
> (http://book.git-scm.com)
> ...

Hello Scott!

Nice book, I just started reading it and I have a recommendation to
make, at "Chapter 4: Git Treeishes" you write

---------
http://book.git-scm.com/4_git_treeishes.html
Range

Finally, you can specify a range of commits with the range spec. This
will give you all the commits between 7b593b5 and 51bea1 (where 51bea1
is most recent), excluding 7b593b5 but including 51bea1:

7b593b5..51bea1

This will include every commit since 7b593b:

7b593b..
---------

This in not quite correct. "commits between A and B" cannot really
apply here. I believe that "commits reachable from B and not from A"
is more precise. Actually you are already using the "reachability"
explanation at the start of "Chapter 3: Basic usage".

This issue is also described at the rev-parse man page.

Apart from that, you could also include "a...b" syntax for completeness.

-christos