git.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
* fast-import's notemodify doesn't work the same as git notes
@ 2014-12-23  0:06 Mike Hommey
  2015-01-05  3:37 ` Johan Herland
  0 siblings, 1 reply; 6+ messages in thread
From: Mike Hommey @ 2014-12-23  0:06 UTC (permalink / raw)
  To: git; +Cc: Johan Herland

Hi,

There are two major differences between adding notes with fast-import
and git notes, one of which is a serious problem:

- fast-import doesn't want to add notes for non commits, while git notes
  does.

- fast-import and git notes have different, conflicting fanouts:
  - take e.g. the git repo (there needs to be a lot of commits to start
    to see the problem)
  - run the following to create notes for every commit:
      (echo 'blob';
       echo 'mark :1';
       echo 'data 0';
       echo 'commit refs/notes/foo';
       echo 'committer <foo> 0 +0000';
       echo 'data 0';
       git rev-list --all | awk '{print "N :1", $1}';
       echo) | git fast-import
  - pick a random commit. I'll pick
    bbcefffcea9789e4a1a2023a1c778e2c07db77a7 as it is current master as
    of writing. Take the first two characters of that sha1, and look at
    the ls-tree:
      git ls-tree refs/notes/foo bb/
    You'll see a number of blobs.
  - Now, remove the note for that commit with git notes:
      git notes --ref foo remove bbcefffcea9789e4a1a2023a1c778e2c07db77a7
  - ls-tree again, you'll now see a number of trees instead of blobs,
    because git notes will have done a fanout. -> git notes does fanouts
    for much less items than fast-import does.
  - Re-add a note for that commit with fast-import:
      git fast-import <<EOF
      blob
      mark :1
      data 0
      commit refs/notes/foo
      committer <foo> 0 +0000
      data 0
      from refs/notes/foo^0
      N :1 bbcefffcea9789e4a1a2023a1c778e2c07db77a7

      EOF
  - ls-tree again, and you'll see a number of trees and *one* blob, for
    bb/cefffcea9789e4a1a2023a1c778e2c07db77a7
  - See the thread starting with 20141126004242.GA13915@glandium.org,
    this type of notes branch make things very slow.
  - Now, if you take an even bigger repository (as long as there are more
    than 65536 commits, that's good ; I guess the linux kernel
    qualifies, I've been checking with a mozilla-central clone), and
    create exactly 65535 notes with git fast-import, you'll end up with
    a 1-level tree (2/38). Add one more note, and the entire tree turns
    into a 2-level tree (2/2/36). git notes would only add a level to
    the tree containing the added note. git notes's behavior scales
    better, because think about what happens on the next fanout with
    fast-import... adding one note would need to create millions of trees.

Cheers,

Mike

^ permalink raw reply	[flat|nested] 6+ messages in thread
end of thread, other threads:[~2015-01-05  4:01 UTC | newest]

Thread overview: 6+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2014-12-23  0:06 fast-import's notemodify doesn't work the same as git notes Mike Hommey
2015-01-05  3:37 ` Johan Herland
2015-01-05  3:39   ` [RFC/PATCHv0 1/4] fast-import.c:do_change_note_fanout(): Remove unneeded local var 't' Johan Herland
2015-01-05  3:39     ` [RFC/PATCHv0 2/4] fast-import.c:do_change_note_fanout(): Also apply load_tree() to initial root Johan Herland
2015-01-05  3:39     ` [RFC/PATCHv0 3/4] fast-import: Support adding notes to non-commits Johan Herland
2015-01-05  3:39     ` [RFC/PATCHv0 4/4] fast-import.c:note_change_n(): Rename commit_* to target_* Johan Herland

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).