All of lore.kernel.org
 help / color / mirror / Atom feed
From: Patrick Steinhardt <ps@pks.im>
To: git@vger.kernel.org
Cc: Han-Wen Nienhuys <hanwenn@gmail.com>
Subject: [PATCH 0/4] reftable: optimize I/O patterns
Date: Mon, 8 Jan 2024 13:18:27 +0100	[thread overview]
Message-ID: <cover.1704714575.git.ps@pks.im> (raw)

[-- Attachment #1: Type: text/plain, Size: 890 bytes --]

Hi,

this patch series optimizes some I/O patterns that the reftable library
uses:

  - Establish a stat(3P)-based caching mechanism to avoid re-reading
    "tables.list" all the time.

  - Convert the block source to use mmap(3P) instead of read(3P) to read
    data.

Combined these lead to a ~2.7x speedup when writing many refs.

Patrick

Patrick Steinhardt (4):
  reftable/stack: refactor stack reloading to have common exit path
  reftable/stack: refactor reloading to use file descriptor
  reftable/stack: use stat info to avoid re-reading stack list
  reftable/blocksource: use mmap to read tables

 reftable/blocksource.c |  48 +++++++++++++-----
 reftable/stack.c       | 111 ++++++++++++++++++++++++-----------------
 reftable/stack.h       |   1 +
 reftable/system.h      |   1 +
 4 files changed, 103 insertions(+), 58 deletions(-)

-- 
2.43.GIT


[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

             reply	other threads:[~2024-01-08 12:18 UTC|newest]

Thread overview: 22+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2024-01-08 12:18 Patrick Steinhardt [this message]
2024-01-08 12:18 ` [PATCH 1/4] reftable/stack: refactor stack reloading to have common exit path Patrick Steinhardt
2024-01-10 17:30   ` Junio C Hamano
2024-01-11  7:33     ` Patrick Steinhardt
2024-01-08 12:18 ` [PATCH 2/4] reftable/stack: refactor reloading to use file descriptor Patrick Steinhardt
2024-01-10 19:57   ` Junio C Hamano
2024-01-08 12:18 ` [PATCH 3/4] reftable/stack: use stat info to avoid re-reading stack list Patrick Steinhardt
2024-01-10 20:07   ` Junio C Hamano
2024-01-11  7:41     ` Patrick Steinhardt
2024-01-08 12:18 ` [PATCH 4/4] reftable/blocksource: use mmap to read tables Patrick Steinhardt
2024-01-10 21:24   ` Junio C Hamano
2024-01-11  9:21     ` Patrick Steinhardt
2024-01-11 10:06 ` [PATCH v2 0/5] reftable: optimize I/O patterns Patrick Steinhardt
2024-01-11 10:06   ` [PATCH v2 1/5] reftable/stack: refactor stack reloading to have common exit path Patrick Steinhardt
2024-01-11 10:06   ` [PATCH v2 2/5] reftable/stack: refactor reloading to use file descriptor Patrick Steinhardt
2024-01-14 10:14     ` Jeff King
2024-01-15 10:03       ` Patrick Steinhardt
2024-01-16 15:14         ` Jeff King
2024-01-16 16:44           ` Junio C Hamano
2024-01-11 10:06   ` [PATCH v2 3/5] reftable/stack: use stat info to avoid re-reading stack list Patrick Steinhardt
2024-01-11 10:06   ` [PATCH v2 4/5] reftable/blocksource: refactor code to match our coding style Patrick Steinhardt
2024-01-11 10:06   ` [PATCH v2 5/5] reftable/blocksource: use mmap to read tables Patrick Steinhardt

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=cover.1704714575.git.ps@pks.im \
    --to=ps@pks.im \
    --cc=git@vger.kernel.org \
    --cc=hanwenn@gmail.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.