All of lore.kernel.org
 help / color / mirror / Atom feed
From: Justin Tobler <jltobler@gmail.com>
To: Patrick Steinhardt <ps@pks.im>
Cc: git@vger.kernel.org, gitster@pobox.com
Subject: Re: [PATCH v2 06/11] odb/transaction: propagate begin errors
Date: Wed, 8 Jul 2026 11:56:43 -0500	[thread overview]
Message-ID: <ak57VEF56HkRKygQ@denethor> (raw)
In-Reply-To: <ak3xJFDqTSN1Naoy@pks.im>

On 26/07/08 08:41AM, Patrick Steinhardt wrote:
> On Tue, Jul 07, 2026 at 11:14:07PM -0500, Justin Tobler wrote:
> > @@ -490,10 +491,12 @@ int cache_tree_update(struct index_state *istate, int flags)
> >  
> >  	trace_performance_enter();
> >  	trace2_region_enter("cache_tree", "update", istate->repo);
> > -	transaction = odb_transaction_begin(the_repository->objects);
> > +	if (!inflight)
> > +		odb_transaction_begin_or_die(the_repository->objects, &transaction);
> >  	i = update_one(istate->cache_tree, istate->cache, istate->cache_nr,
> >  		       "", 0, &skip, flags);
> > -	odb_transaction_commit(transaction);
> > +	if (!inflight)
> > +		odb_transaction_commit(transaction);
> >  	trace2_region_leave("cache_tree", "update", istate->repo);
> >  	trace_performance_leave("cache_tree_update");
> >  	if (i < 0)
> 
> Callsites like this really make me wonder why we even care to create
> a transaction in the first place if we basically just commit it
> immediately anyway. And while it's a bit sad that we have so many sites
> where we don't really know whether we even have a transaction, I think
> it's a good change that we have now annotated them clearly. A subsequent
> patch series may then eventually refactor those sites so that we stop
> depending on `odb->transaction` and inject the transaction via a
> parameter.

Call sites like the one mentioned above are using ODB transactions as an
optimization to batch the full fsyncs in bulk. In cases where the is not
already a transaction, they start one to take advantage of it.

I fully agree though that an ODB transaction should ideally be started
at a higher layer and wired down to these call sites. I have a couple of
patches in my tree that start to tackle this which I plan to send in
another series. :)

> > diff --git a/odb/transaction.h b/odb/transaction.h
> > index d52f0533ce..36032a5365 100644
> > --- a/odb/transaction.h
> > +++ b/odb/transaction.h
> > @@ -1,6 +1,8 @@
> >  #ifndef ODB_TRANSACTION_H
> >  #define ODB_TRANSACTION_H
> >  
> > +#include "git-compat-util.h"
> 
> We typically don't include "git-compat-util.h" in header files.

Will remove. Thanks

> > @@ -36,11 +38,21 @@ struct odb_transaction {
> >  };
> >  
> >  /*
> > - * Starts an ODB transaction. Subsequent objects are written to the transaction
> > - * and not committed until odb_transaction_commit() is invoked on the
> > - * transaction. If the ODB already has a pending transaction, NULL is returned.
> > + * Starts an ODB transaction and returns it via `out`. Subsequent objects are
> > + * written to the transaction and not committed until odb_transaction_commit()
> > + * is invoked on the transaction. Returns 0 on success and a negative value on
> > + * error. Note that it is considered an error to start a new transaction if the
> > + * ODB already has an inflight transaction pending.
> >   */
> > -struct odb_transaction *odb_transaction_begin(struct object_database *odb);
> > +int odb_transaction_begin(struct object_database *odb,
> > +			  struct odb_transaction **out);
> > +
> > +static inline void odb_transaction_begin_or_die(struct object_database *odb,
> > +						struct odb_transaction **out)
> > +{
> > +	if (odb_transaction_begin(odb, out))
> > +		die(_("failed to start ODB transaction"));
> > +}
> 
> We could make it a bit simpler to use this function by continuing to
> return the transaction directly. But on the other hand this results in a
> more consistent interface.

Ya, I was a bit back and forth about this myself. I ultimately landed on
keeping a more consistent interface though. Happy to change if others
feel differently though.

-Justin

  reply	other threads:[~2026-07-08 16:56 UTC|newest]

Thread overview: 90+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2026-06-24  4:19 [PATCH 0/6] receive-pack: use ODB transactions to stage object writes Justin Tobler
2026-06-24  4:19 ` [PATCH 1/6] object-file: rename files transaction prepare function Justin Tobler
2026-06-24 18:26   ` Junio C Hamano
2026-06-29 18:11     ` Justin Tobler
2026-06-24  4:19 ` [PATCH 2/6] object-file: propagate files transaction errors Justin Tobler
2026-06-24 11:26   ` Patrick Steinhardt
2026-06-29 18:58     ` Justin Tobler
2026-06-29 19:04       ` Justin Tobler
2026-06-30  8:45         ` Patrick Steinhardt
2026-06-30 14:14           ` Justin Tobler
2026-06-30  8:45       ` Patrick Steinhardt
2026-06-24 18:35   ` Junio C Hamano
2026-06-29 19:10     ` Justin Tobler
2026-06-24  4:19 ` [PATCH 3/6] odb/transaction: propagate begin errors Justin Tobler
2026-06-24 11:26   ` Patrick Steinhardt
2026-06-29 19:15     ` Justin Tobler
2026-06-24  4:19 ` [PATCH 4/6] odb/transaction: propagate commit errors Justin Tobler
2026-06-24 11:26   ` Patrick Steinhardt
2026-06-29 19:16     ` Justin Tobler
2026-06-24  4:19 ` [PATCH 5/6] odb/transaction: add transaction env interface Justin Tobler
2026-06-24 11:26   ` Patrick Steinhardt
2026-06-29 19:20     ` Justin Tobler
2026-06-24  4:19 ` [PATCH 6/6] builtin/receive-pack: stage incoming objects via ODB transactions Justin Tobler
2026-06-24 11:26   ` Patrick Steinhardt
2026-06-29 20:25     ` Justin Tobler
2026-06-30  8:45       ` Patrick Steinhardt
2026-06-24 11:27 ` [PATCH 0/6] receive-pack: use ODB transactions to stage object writes Patrick Steinhardt
2026-06-24 20:09 ` Junio C Hamano
2026-07-08  4:14 ` [PATCH v2 00/11] " Justin Tobler
2026-07-08  4:14   ` [PATCH v2 01/11] object-file: rename files transaction prepare function Justin Tobler
2026-07-08  4:14   ` [PATCH v2 02/11] object-file: rename files transaction fsync function Justin Tobler
2026-07-08  6:41     ` Patrick Steinhardt
2026-07-08  4:14   ` [PATCH v2 03/11] object-file: embed transaction flush logic in commit function Justin Tobler
2026-07-08  6:41     ` Patrick Steinhardt
2026-07-08 16:08       ` Justin Tobler
2026-07-08  4:14   ` [PATCH v2 04/11] object-file: drop check for inflight transactions Justin Tobler
2026-07-08  6:41     ` Patrick Steinhardt
2026-07-08  4:14   ` [PATCH v2 05/11] object-file: propagate files transaction errors Justin Tobler
2026-07-08  6:41     ` Patrick Steinhardt
2026-07-08 16:21       ` Justin Tobler
2026-07-08  4:14   ` [PATCH v2 06/11] odb/transaction: propagate begin errors Justin Tobler
2026-07-08  6:41     ` Patrick Steinhardt
2026-07-08 16:56       ` Justin Tobler [this message]
2026-07-09  9:39         ` Patrick Steinhardt
2026-07-08  4:14   ` [PATCH v2 07/11] odb/transaction: propagate commit errors Justin Tobler
2026-07-08  6:41     ` Patrick Steinhardt
2026-07-08 17:24       ` Justin Tobler
2026-07-08  4:14   ` [PATCH v2 08/11] odb/transaction: add transaction env interface Justin Tobler
2026-07-08  4:14   ` [PATCH v2 09/11] odb/transaction: introduce ODB transaction flags Justin Tobler
2026-07-08  6:41     ` Patrick Steinhardt
2026-07-08 17:34       ` Justin Tobler
2026-07-08  4:14   ` [PATCH v2 10/11] builtin/receive-pack: drop redundant tmpdir env Justin Tobler
2026-07-08  6:41     ` Patrick Steinhardt
2026-07-08  4:14   ` [PATCH v2 11/11] builtin/receive-pack: stage incoming objects via ODB transactions Justin Tobler
2026-07-08  6:42   ` [PATCH v2 00/11] receive-pack: use ODB transactions to stage object writes Patrick Steinhardt
2026-07-08 17:36     ` Justin Tobler
2026-07-08 23:59   ` [PATCH v3 " Justin Tobler
2026-07-08 23:59     ` [PATCH v3 01/11] object-file: rename files transaction prepare function Justin Tobler
2026-07-08 23:59     ` [PATCH v3 02/11] object-file: rename files transaction fsync function Justin Tobler
2026-07-08 23:59     ` [PATCH v3 03/11] object-file: embed transaction flush logic in commit function Justin Tobler
2026-07-09  9:38       ` Patrick Steinhardt
2026-07-08 23:59     ` [PATCH v3 04/11] object-file: drop check for inflight transactions Justin Tobler
2026-07-08 23:59     ` [PATCH v3 05/11] object-file: propagate files transaction errors Justin Tobler
2026-07-08 23:59     ` [PATCH v3 06/11] odb/transaction: propagate begin errors Justin Tobler
2026-07-09  3:32       ` Junio C Hamano
2026-07-09 14:03         ` Justin Tobler
2026-07-08 23:59     ` [PATCH v3 07/11] odb/transaction: propagate commit errors Justin Tobler
2026-07-08 23:59     ` [PATCH v3 08/11] odb/transaction: add transaction env interface Justin Tobler
2026-07-09  3:36       ` Junio C Hamano
2026-07-09 15:02         ` Justin Tobler
2026-07-08 23:59     ` [PATCH v3 09/11] odb/transaction: introduce ODB transaction flags Justin Tobler
2026-07-08 23:59     ` [PATCH v3 10/11] builtin/receive-pack: drop redundant tmpdir env Justin Tobler
2026-07-08 23:59     ` [PATCH v3 11/11] builtin/receive-pack: stage incoming objects via ODB transactions Justin Tobler
2026-07-09  3:49       ` Junio C Hamano
2026-07-10 14:37         ` Justin Tobler
2026-07-10 16:52           ` Junio C Hamano
2026-07-09  9:39     ` [PATCH v3 00/11] receive-pack: use ODB transactions to stage object writes Patrick Steinhardt
2026-07-10 16:37     ` [PATCH v4 " Justin Tobler
2026-07-10 16:37       ` [PATCH v4 01/11] object-file: rename files transaction prepare function Justin Tobler
2026-07-10 16:37       ` [PATCH v4 02/11] object-file: rename files transaction fsync function Justin Tobler
2026-07-10 16:37       ` [PATCH v4 03/11] object-file: embed transaction flush logic in commit function Justin Tobler
2026-07-10 16:37       ` [PATCH v4 04/11] object-file: drop check for inflight transactions Justin Tobler
2026-07-10 16:37       ` [PATCH v4 05/11] object-file: propagate files transaction errors Justin Tobler
2026-07-10 16:37       ` [PATCH v4 06/11] odb/transaction: propagate begin errors Justin Tobler
2026-07-10 16:37       ` [PATCH v4 07/11] odb/transaction: propagate commit errors Justin Tobler
2026-07-10 16:37       ` [PATCH v4 08/11] odb/transaction: add transaction env interface Justin Tobler
2026-07-10 16:37       ` [PATCH v4 09/11] odb/transaction: introduce ODB transaction flags Justin Tobler
2026-07-10 16:37       ` [PATCH v4 10/11] builtin/receive-pack: drop redundant tmpdir env Justin Tobler
2026-07-10 16:37       ` [PATCH v4 11/11] builtin/receive-pack: stage incoming objects via ODB transactions Justin Tobler
2026-07-13  5:18       ` [PATCH v4 00/11] receive-pack: use ODB transactions to stage object writes Patrick Steinhardt

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=ak57VEF56HkRKygQ@denethor \
    --to=jltobler@gmail.com \
    --cc=git@vger.kernel.org \
    --cc=gitster@pobox.com \
    --cc=ps@pks.im \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.