From: Junio C Hamano <gitster@pobox.com>
To: Michael Haggerty <mhagger@alum.mit.edu>
Cc: "Nguyễn Thái Ngọc Duy" <pclouds@gmail.com>,
"Stefan Beller" <sbeller@google.com>, "Jeff King" <peff@peff.net>,
"Ævar Arnfjörð Bjarmason" <avarab@gmail.com>,
"Brandon Williams" <bmwill@google.com>,
git@vger.kernel.org
Subject: Re: [PATCH 00/20] Read `packed-refs` using mmap()
Date: Thu, 14 Sep 2017 06:35:13 +0900 [thread overview]
Message-ID: <xmqqbmme1dv2.fsf@gitster.mtv.corp.google.com> (raw)
In-Reply-To: <cover.1505319366.git.mhagger@alum.mit.edu> (Michael Haggerty's message of "Wed, 13 Sep 2017 19:15:54 +0200")
Michael Haggerty <mhagger@alum.mit.edu> writes:
> Following lots of work to extract the `packed_ref_store` into a
> separate module and decouple it from the `files_ref_store`, it is now
> possible to fundamentally change how the `packed-refs` file is read.
>
> * `mmap()` the whole file rather than `read()`ing it.
>
> * Instead of parsing the complete file at once into a `ref_cache`,
> parse the references out of the file contents on demand.
>
> * Use a binary search to find, very quickly, the reference or group of
> references that needs to be read. Parse *only* those references out
> of the file contents, without creating in-memory data structures at
> all.
Oooh, juicy and very exciting.
> ...
> This branch applies on top of mh/packed-ref-transactions. It can also
> be obtained from my git fork [1] as branch `mmap-packed-refs`.
>
> Michael
>
> [1] https://github.com/mhagger/git
Thanks for working on this. I expect that it will take more than a
few hours for me to pick it up, but I am looking forward to it.
next prev parent reply other threads:[~2017-09-13 21:37 UTC|newest]
Thread overview: 27+ messages / expand[flat|nested] mbox.gz Atom feed top
2017-09-13 17:15 [PATCH 00/20] Read `packed-refs` using mmap() Michael Haggerty
2017-09-13 17:15 ` [PATCH 01/20] ref_iterator: keep track of whether the iterator output is ordered Michael Haggerty
2017-09-13 17:15 ` [PATCH 02/20] prefix_ref_iterator: break when we leave the prefix Michael Haggerty
2017-09-13 17:15 ` [PATCH 03/20] packed_ref_cache: add a backlink to the associated `packed_ref_store` Michael Haggerty
2017-09-13 17:15 ` [PATCH 04/20] die_unterminated_line(), die_invalid_line(): new functions Michael Haggerty
2017-09-13 17:15 ` [PATCH 05/20] read_packed_refs(): use mmap to read the `packed-refs` file Michael Haggerty
2017-09-13 17:16 ` [PATCH 06/20] read_packed_refs(): only check for a header at the top of the file Michael Haggerty
2017-09-13 17:16 ` [PATCH 07/20] read_packed_refs(): make parsing of the header line more robust Michael Haggerty
2017-09-13 17:16 ` [PATCH 08/20] read_packed_refs(): read references with minimal copying Michael Haggerty
2017-09-13 17:16 ` [PATCH 09/20] packed_ref_cache: remember the file-wide peeling state Michael Haggerty
2017-09-13 17:16 ` [PATCH 10/20] mmapped_ref_iterator: add iterator over a packed-refs file Michael Haggerty
2017-09-13 17:16 ` [PATCH 11/20] mmapped_ref_iterator_advance(): no peeled value for broken refs Michael Haggerty
2017-09-13 17:16 ` [PATCH 12/20] packed_ref_cache: keep the `packed-refs` file open and mmapped Michael Haggerty
2017-09-13 17:16 ` [PATCH 13/20] read_packed_refs(): ensure that references are ordered when read Michael Haggerty
2017-09-13 17:16 ` [PATCH 14/20] packed_ref_iterator_begin(): iterate using `mmapped_ref_iterator` Michael Haggerty
2017-09-13 17:16 ` [PATCH 15/20] packed_read_raw_ref(): read the reference from the mmapped buffer Michael Haggerty
2017-09-13 17:16 ` [PATCH 16/20] ref_store: implement `refs_peel_ref()` generically Michael Haggerty
2017-09-13 17:16 ` [PATCH 17/20] packed_ref_store: get rid of the `ref_cache` entirely Michael Haggerty
2017-09-13 17:16 ` [PATCH 18/20] ref_cache: remove support for storing peeled values Michael Haggerty
2017-09-13 17:16 ` [PATCH 19/20] mmapped_ref_iterator: inline into `packed_ref_iterator` Michael Haggerty
2017-09-13 17:16 ` [PATCH 20/20] packed-backend.c: rename a bunch of things and update comments Michael Haggerty
2017-09-13 23:00 ` Stefan Beller
2017-09-13 21:35 ` Junio C Hamano [this message]
2017-09-14 16:12 ` [PATCH 00/20] Read `packed-refs` using mmap() Michael Haggerty
2017-09-14 22:16 ` Johannes Schindelin
2017-09-14 20:23 ` Johannes Schindelin
2017-09-15 4:21 ` Michael Haggerty
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=xmqqbmme1dv2.fsf@gitster.mtv.corp.google.com \
--to=gitster@pobox.com \
--cc=avarab@gmail.com \
--cc=bmwill@google.com \
--cc=git@vger.kernel.org \
--cc=mhagger@alum.mit.edu \
--cc=pclouds@gmail.com \
--cc=peff@peff.net \
--cc=sbeller@google.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.