Git development
 help / color / mirror / Atom feed
From: Justin Tobler <jltobler@gmail.com>
To: Patrick Steinhardt <ps@pks.im>
Cc: git@vger.kernel.org
Subject: Re: [PATCH 2/4] object-file: only flush batched fsync writes when enabled
Date: Tue, 17 Nov 2026 17:35:44 -0600	[thread overview]
Message-ID: <avzghl3FBW3TTWhV@denethor> (raw)
In-Reply-To: <avRqlHdN83O67HB1@pks.im>

On 26/11/11 02:16PM, Patrick Steinhardt wrote:
> On Mon, Nov 09, 2026 at 06:22:49PM -0600, Justin Tobler wrote:
> > When committing an ODB transaction for the "files" backend, a full
> > hardware flush is issued against a temporary file before the staged
> > objects are migrated into the main object database. This flush acts as a
> > barrier for objects that have been written with batched fsync, where
> > `odb_transaction_files_fsync()` only issues a writeout request for each
> > object instead of fully syncing it.
> 
> Hmm, okay. The difference here is bascially that we don't issue a
> hardware flush. So we know that the data is not in the page cache
> anymore, but it may still only be cached by the disk.

Performing the hardware sync here is really only intended for batch fsync 

Right, that's what writeout-only gives us for each object with batched
fsync, and why the single hardware flush at commit time is needed as a
barrier.

That state only arises with batched fsync though. Otherwise, objects are
either fully synced individually, in which case they are already durable
and the extra flush is redundant, or not synced at all, in which case
there is nothing to make durable. So the flush is only useful if there
were writeout-only syncs.

> > The flush is issued whenever the transaction has a temporary object
> > directory though, regardless of whether batched fsync is actually
> > enabled. For transactions configured for git-receive-pack(1), the
> > temporary object directory is always created, so the flush is also
> > performed when objects are synced individually or not synced at all. In
> > those cases there are no pending writeout requests and the flush is
> > unnecessary. Only issue the flush when batched fsync is enabled
> > accordingly.
> 
> So this is essentially a performance optimization?

Yes and I also think its not really the correct behavior. Even is fsync
is disabled, the fsync here happens. It doesn't really harm anything,
but I think it makes sense to address as we begin to roll out
transactions more widely.

[snip]
> I assume that we only support batched fsyncs for loose objects in the
> first place? In other words, if a transaction were to write a bunch of
> packfiles, it would not be possible to batch-fsync those right now?

Correct, this mechanism was intended to batch fsync loose objects.

> If so this new condition correct to me. But I think this should be
> documented as part of the commit message, because it's an implicit
> assumption.

Makes sense. I'll update the commit message in the next version.

Thanks,
-Justin

  reply	other threads:[~2026-11-17 23:35 UTC|newest]

Thread overview: 19+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2026-11-10  0:22 [PATCH 0/4] object-file: eagerly create ODB transaction temporary directory Justin Tobler
2026-11-10  0:22 ` [PATCH 1/4] object-file: rename default transaction prefix Justin Tobler
2026-11-11 13:15   ` Patrick Steinhardt
2026-11-11 14:36   ` Karthik Nayak
2026-11-10  0:22 ` [PATCH 2/4] object-file: only flush batched fsync writes when enabled Justin Tobler
2026-11-11 13:16   ` Patrick Steinhardt
2026-11-17 23:35     ` Justin Tobler [this message]
2026-11-11 14:48   ` Karthik Nayak
2026-11-17 23:42     ` Justin Tobler
2026-11-10  0:22 ` [PATCH 3/4] tmp-objdir: preserve errno when directory creation fails Justin Tobler
2026-11-11 13:16   ` Patrick Steinhardt
2026-11-11 16:44   ` Karthik Nayak
2026-11-17 23:46     ` Justin Tobler
2026-11-10  0:22 ` [PATCH 4/4] object-file: create ODB transaction temporary directory eagerly Justin Tobler
2026-11-11 13:15   ` Patrick Steinhardt
2026-11-17 23:48     ` Justin Tobler
2026-11-11 16:48   ` Karthik Nayak
2026-11-17 23:57     ` Justin Tobler
2026-11-11 14:33 ` [PATCH 0/4] object-file: eagerly create ODB transaction temporary directory Karthik Nayak

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=avzghl3FBW3TTWhV@denethor \
    --to=jltobler@gmail.com \
    --cc=git@vger.kernel.org \
    --cc=ps@pks.im \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox