git only writing 4k at a time

git only writing 4k at a time

[Haylin Moore]

Hiya list,

I've been investigating some performance issues around git clones over
network mounts. We have noticed that git is only writing 4k at a time.
These small serial writes are making it such that even though each
write is only a 3ms operation, the total time balloons. Looking around
the source code I found that reftable_writer is initialized by default
(though I cannot find the block_size argument being supplied in my
cursory look) always to DEFAULT_BLOCK_SIZE (4096). Is there some way
to increase/configure this block size such that larger writes happen?
In git/Documentation/config/reftable.adoc this block size is mentioned
in a manner that almost feels configurable, but I'm not sure if this
is just internal for development.

Thanks,
~ Haylin Moore

Re: git only writing 4k at a time

[brian m. carlson]

[-- Attachment #1: Type: text/plain, Size: 1531 bytes --]

On 2025-06-18 at 20:58:52, Haylin Moore wrote:
> Hiya list,
> 
> I've been investigating some performance issues around git clones over
> network mounts. We have noticed that git is only writing 4k at a time.
> These small serial writes are making it such that even though each
> write is only a 3ms operation, the total time balloons. Looking around
> the source code I found that reftable_writer is initialized by default
> (though I cannot find the block_size argument being supplied in my
> cursory look) always to DEFAULT_BLOCK_SIZE (4096). Is there some way
> to increase/configure this block size such that larger writes happen?
> In git/Documentation/config/reftable.adoc this block size is mentioned
> in a manner that almost feels configurable, but I'm not sure if this
> is just internal for development.

It's fine to adjust reftable.blockSize upwards if you'd like, which
controls the block size for reftable writes (which is what you're seeing
if the writes are from the reftable code).  I think at least some
versions of JGit use 64 KiB for various reasons.

As the documentation describes, there may be some performance penalties
during reads since more refs will have to be read, so reading a single
ref will likely be more expensive.  However, you may find that
acceptable and you can adjust the values such that they provide the
right balance in your environment.  I would definitely recommend a
power-of-two block size, though.
-- 
brian m. carlson (they/them)
Toronto, Ontario, CA

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 262 bytes --]

Re: git only writing 4k at a time

[Patrick Steinhardt]

On Wed, Jun 18, 2025 at 10:27:30PM +0000, brian m. carlson wrote:
> On 2025-06-18 at 20:58:52, Haylin Moore wrote:
> > Hiya list,
> > 
> > I've been investigating some performance issues around git clones over
> > network mounts. We have noticed that git is only writing 4k at a time.
> > These small serial writes are making it such that even though each
> > write is only a 3ms operation, the total time balloons. Looking around
> > the source code I found that reftable_writer is initialized by default
> > (though I cannot find the block_size argument being supplied in my
> > cursory look) always to DEFAULT_BLOCK_SIZE (4096). Is there some way
> > to increase/configure this block size such that larger writes happen?
> > In git/Documentation/config/reftable.adoc this block size is mentioned
> > in a manner that almost feels configurable, but I'm not sure if this
> > is just internal for development.
> 
> It's fine to adjust reftable.blockSize upwards if you'd like, which
> controls the block size for reftable writes (which is what you're seeing
> if the writes are from the reftable code).  I think at least some
> versions of JGit use 64 KiB for various reasons.

Yup. The default block size of 4kB was picked because most filesystems
use it. Google uses 64kB because to the best of my knowledge they use
Spanner to store the tables? At least that's what I recall from past
conversations.

> As the documentation describes, there may be some performance penalties
> during reads since more refs will have to be read, so reading a single
> ref will likely be more expensive.  However, you may find that
> acceptable and you can adjust the values such that they provide the
> right balance in your environment.  I would definitely recommend a
> power-of-two block size, though.

So this kind of depends on the filesystem's block size. If yours uses
bigger blocks it's definitely recommended to adjust as needed. The block
size is ultimately a tradeoff, and the best value heavily depends on
both your system and on your use case.

I'm curious though -- are you sure that this is actually the bottleneck?
Reftables are only used if you explicitly opted into them, and I would
be very surprised if a clone is really slowed down significantly by a
clone.

Patrick