git.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
From: "Nguyễn Thái Ngọc Duy" <pclouds@gmail.com>
To: git@vger.kernel.org
Cc: "Christian Couder" <christian.couder@gmail.com>,
	"Nguyễn Thái Ngọc Duy" <pclouds@gmail.com>
Subject: [PATCH v4 0/9] Reduce index load time
Date: Sun,  1 Nov 2015 14:42:40 +0100	[thread overview]
Message-ID: <1446385369-8669-1-git-send-email-pclouds@gmail.com> (raw)

This is the rebased version since last time [1] with
s/free_index_shm/release_index_shm/ as suggested by David Turner. It
introduces a daemon that can cache index data in memory so that
subsequent git processes can avoid reading (and more importantly,
verifying) the index from disk. Together with split-index it should
keep index I/O cost down to minimum. The series can also be found at
[2].

One of the factors that affected my design was Windows support. We
now have Dscho back, he can evaluate my approach for Windows.

This daemon is the foundation for watchman support later to reduce
refresh time. To be posted shortly after this.

[0] http://mid.gmane.org/1406548995-28549-1-git-send-email-pclouds@gmail.com
[2] http://github.com/pclouds/git/commits/index-helper

Nguyễn Thái Ngọc Duy (9):
  trace.c: add GIT_TRACE_PACK_STATS for pack usage statistics
  read-cache.c: fix constness of verify_hdr()
  read-cache: allow to keep mmap'd memory after reading
  index-helper: new daemon for caching index and related stuff
  trace.c: add GIT_TRACE_INDEX_STATS for index  statistics
  index-helper: add --strict
  daemonize(): set a flag before exiting the main process
  index-helper: add --detach
  index-helper: add Windows support

 .gitignore                               |   1 +
 Documentation/git-index-helper.txt (new) |  56 +++++++
 Documentation/git.txt                    |   4 +
 Makefile                                 |   9 ++
 builtin/gc.c                             |   2 +-
 cache.h                                  |  12 +-
 config.mak.uname                         |   3 +
 daemon.c                                 |   2 +-
 git-compat-util.h                        |   1 +
 git.c                                    |   1 +
 index-helper.c (new)                     | 264 +++++++++++++++++++++++++++++++
 read-cache.c                             | 147 ++++++++++++++++-
 setup.c                                  |   4 +-
 sha1_file.c                              |  24 +++
 shm.c (new)                              | 163 +++++++++++++++++++
 shm.h (new)                              |  23 +++
 trace.c                                  |  16 ++
 trace.h                                  |   1 +
 18 files changed, 721 insertions(+), 12 deletions(-)
 create mode 100644 Documentation/git-index-helper.txt
 create mode 100644 index-helper.c
 create mode 100644 shm.c
 create mode 100644 shm.h

-- 
2.2.0.513.g477eb31

             reply	other threads:[~2015-11-01 13:49 UTC|newest]

Thread overview: 12+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2015-11-01 13:42 Nguyễn Thái Ngọc Duy [this message]
2015-11-01 13:42 ` [PATCH v4 1/9] trace.c: add GIT_TRACE_PACK_STATS for pack usage statistics Nguyễn Thái Ngọc Duy
2015-11-01 13:42 ` [PATCH v4 2/9] read-cache.c: fix constness of verify_hdr() Nguyễn Thái Ngọc Duy
2015-11-01 13:42 ` [PATCH v4 3/9] read-cache: allow to keep mmap'd memory after reading Nguyễn Thái Ngọc Duy
2015-11-01 13:42 ` [PATCH v4 4/9] index-helper: new daemon for caching index and related stuff Nguyễn Thái Ngọc Duy
2015-11-02 22:14   ` David Turner
2015-11-01 13:42 ` [PATCH v4 5/9] trace.c: add GIT_TRACE_INDEX_STATS for index statistics Nguyễn Thái Ngọc Duy
2015-11-01 13:42 ` [PATCH v4 6/9] index-helper: add --strict Nguyễn Thái Ngọc Duy
2015-11-01 13:42 ` [PATCH v4 7/9] daemonize(): set a flag before exiting the main process Nguyễn Thái Ngọc Duy
2015-11-01 13:42 ` [PATCH v4 8/9] index-helper: add --detach Nguyễn Thái Ngọc Duy
2015-11-01 13:42 ` [PATCH v4 9/9] index-helper: add Windows support Nguyễn Thái Ngọc Duy
2015-11-16 21:51 ` [PATCH v4 0/9] Reduce index load time Johannes Schindelin

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=1446385369-8669-1-git-send-email-pclouds@gmail.com \
    --to=pclouds@gmail.com \
    --cc=christian.couder@gmail.com \
    --cc=git@vger.kernel.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).