Re: fast-import's gfi_unpack_entry causes too many munmap/mmap cycles

[Mike Hommey <mh@glandium.org>]

On Sun, Apr 17, 2016 at 09:54:43AM +0900, Mike Hommey wrote:
> On Sat, Apr 16, 2016 at 08:04:03PM +0900, Mike Hommey wrote:
> > So I think I got myself a workaround...
> > 
> > > A --- B
> > >  \
> > >   \-- C
> > > 
> > > I have:
> > > - diff between null-tree and A
> > > - diff between A and B
> > > - diff between B and C
> > 
> > I should be able to do:
> > 
> > - start the commit command for A
> > - before finishing it, `ls ""`
> > - then apply the diff for B and `ls ""`
> > - then apply the diff for C and `ls ""`
> > - then `deleteall`
> > - then `M 040000 sha1_from_first_ls ` and finally finish A
> > - create the commit for B with `from
> >   0000000000000000000000000000000000000000\nmerge :mark` and `M 040000
> >   sha1_from_second_ls`
> > - likewise for C
> > 
> > ... and avoid gfi_unpack_entry.
> 
> And it works... as an avoidance of gfi_unpack_entry... but it has its
> own problem: somehow the store_tree() that happens for each of those
> `ls ""` commands is storing *all* trees. Even the ones that haven't
> changed. In terms of a minimalistic fast-import script:
> 
> With:
>   commit refs/FOO
>   committer <foo@foo> 0 +0
>   data 0
> 
>   M 644 inline a/a
>   data 1
>   a
> 
>   commit refs/FOO
>   committer <foo@foo> 0 +0
>   data 0
> 
>   M 644 inline b/b
>   data 1
>   b
> 
> store_tree is called for:
> - b39954843ff6e09ec3aa2b942938c30c6bd1629e
> - 2c3b59f77afa6fea6c1a380eeb0cb1eb292515b5
> - 51e58bf6ce558dd384bbf9d493f9a376f3bcb089
> - a97dda9f3a819113b3b239b9a62edece27136080
> 
> With:
>   commit refs/FOO
>   committer <foo@foo> 0 +0
>   data 0
> 
>   M 644 inline a/a
>   data 1
>   a
>   ls ""
>   M 644 inline b/b
>   data 1
>   b
> 
> store_tree is called for:
> - b39954843ff6e09ec3aa2b942938c30c6bd1629e
> - 2c3b59f77afa6fea6c1a380eeb0cb1eb292515b5
> - b39954843ff6e09ec3aa2b942938c30c6bd1629e
> - 51e58bf6ce558dd384bbf9d493f9a376f3bcb089
> - a97dda9f3a819113b3b239b9a62edece27136080
> 
> Note how b39954843ff6e09ec3aa2b942938c30c6bd1629e is being stored twice
> (it's the tree for a/).

And that happens because tree_content_get returns a duplicate of the
tree, so when parse_ls does a store_tree, it does it on a duplicate, not
on the tree itself. So all the work done by store_tree through parse_ls
is thrown away.

Mike