git.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
From: Patrick Steinhardt <ps@pks.im>
To: git@vger.kernel.org
Cc: Han-Wen Nienhuys <hanwenn@gmail.com>
Subject: [PATCH 0/4] reftable: optimize I/O patterns
Date: Mon, 8 Jan 2024 13:18:27 +0100	[thread overview]
Message-ID: <cover.1704714575.git.ps@pks.im> (raw)

[-- Attachment #1: Type: text/plain, Size: 890 bytes --]

Hi,

this patch series optimizes some I/O patterns that the reftable library
uses:

  - Establish a stat(3P)-based caching mechanism to avoid re-reading
    "tables.list" all the time.

  - Convert the block source to use mmap(3P) instead of read(3P) to read
    data.

Combined these lead to a ~2.7x speedup when writing many refs.

Patrick

Patrick Steinhardt (4):
  reftable/stack: refactor stack reloading to have common exit path
  reftable/stack: refactor reloading to use file descriptor
  reftable/stack: use stat info to avoid re-reading stack list
  reftable/blocksource: use mmap to read tables

 reftable/blocksource.c |  48 +++++++++++++-----
 reftable/stack.c       | 111 ++++++++++++++++++++++++-----------------
 reftable/stack.h       |   1 +
 reftable/system.h      |   1 +
 4 files changed, 103 insertions(+), 58 deletions(-)

-- 
2.43.GIT


[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

             reply	other threads:[~2024-01-08 12:18 UTC|newest]

Thread overview: 22+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2024-01-08 12:18 Patrick Steinhardt [this message]
2024-01-08 12:18 ` [PATCH 1/4] reftable/stack: refactor stack reloading to have common exit path Patrick Steinhardt
2024-01-10 17:30   ` Junio C Hamano
2024-01-11  7:33     ` Patrick Steinhardt
2024-01-08 12:18 ` [PATCH 2/4] reftable/stack: refactor reloading to use file descriptor Patrick Steinhardt
2024-01-10 19:57   ` Junio C Hamano
2024-01-08 12:18 ` [PATCH 3/4] reftable/stack: use stat info to avoid re-reading stack list Patrick Steinhardt
2024-01-10 20:07   ` Junio C Hamano
2024-01-11  7:41     ` Patrick Steinhardt
2024-01-08 12:18 ` [PATCH 4/4] reftable/blocksource: use mmap to read tables Patrick Steinhardt
2024-01-10 21:24   ` Junio C Hamano
2024-01-11  9:21     ` Patrick Steinhardt
2024-01-11 10:06 ` [PATCH v2 0/5] reftable: optimize I/O patterns Patrick Steinhardt
2024-01-11 10:06   ` [PATCH v2 1/5] reftable/stack: refactor stack reloading to have common exit path Patrick Steinhardt
2024-01-11 10:06   ` [PATCH v2 2/5] reftable/stack: refactor reloading to use file descriptor Patrick Steinhardt
2024-01-14 10:14     ` Jeff King
2024-01-15 10:03       ` Patrick Steinhardt
2024-01-16 15:14         ` Jeff King
2024-01-16 16:44           ` Junio C Hamano
2024-01-11 10:06   ` [PATCH v2 3/5] reftable/stack: use stat info to avoid re-reading stack list Patrick Steinhardt
2024-01-11 10:06   ` [PATCH v2 4/5] reftable/blocksource: refactor code to match our coding style Patrick Steinhardt
2024-01-11 10:06   ` [PATCH v2 5/5] reftable/blocksource: use mmap to read tables Patrick Steinhardt

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=cover.1704714575.git.ps@pks.im \
    --to=ps@pks.im \
    --cc=git@vger.kernel.org \
    --cc=hanwenn@gmail.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).