git.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
From: Patrick Steinhardt <ps@pks.im>
To: git@vger.kernel.org
Cc: "Edward Thomson" <ethomson@edwardthomson.com>,
	"Junio C Hamano" <gitster@pobox.com>,
	"René Scharfe" <l.s.r@web.de>
Subject: [PATCH v3 01/22] reftable/error: introduce out-of-memory error code
Date: Mon, 30 Sep 2024 10:08:11 +0200	[thread overview]
Message-ID: <8c99ecc3255b1b10cdec047da01804dc23a7d287.1727680272.git.ps@pks.im> (raw)
In-Reply-To: <cover.1727680272.git.ps@pks.im>

The reftable library does not use the same memory allocation functions
as the rest of the Git codebase. Instead, as the reftable library is
supposed to be usable as a standalone library without Git, it provides a
set of pluggable memory allocators.

Compared to `xmalloc()` and friends these allocators are _not_ expected
to die when an allocation fails. This design choice is concious, as a
library should leave it to its caller to handle any kind of error. While
it is very likely that the caller cannot really do much in the case of
an out-of-memory situation anyway, we are not the ones to make that
decision.

Curiously though, we never handle allocation errors even though memory
allocation functions are allowed to fail. And as we do not plug in Git's
memory allocator via `reftable_set_alloc()` either the consequence is
that we'd instead segfault as soon as we run out of memory.

While the easy fix would be to wire up `xmalloc()` and friends, it
would only fix the usage of the reftable library in Git itself. Other
users like libgit2, which is about to revive its efforts to land a
backend for reftables, wouldn't be able to benefit from this solution.

Instead, we are about to do it the hard way: adapt all allocation sites
to perform error checking. Introduce a new error code for out-of-memory
errors that we will wire up in subsequent steps.

This commit also serves as the motivator for all the remaining steps in
this series such that we do not have to repeat the same arguments in
every single subsequent commit.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 reftable/error.c          | 2 ++
 reftable/reftable-error.h | 3 +++
 2 files changed, 5 insertions(+)

diff --git a/reftable/error.c b/reftable/error.c
index a25f28a43e..660d029617 100644
--- a/reftable/error.c
+++ b/reftable/error.c
@@ -35,6 +35,8 @@ const char *reftable_error_str(int err)
 		return "entry too large";
 	case REFTABLE_OUTDATED_ERROR:
 		return "data concurrently modified";
+	case REFTABLE_OUT_OF_MEMORY_ERROR:
+		return "out of memory";
 	case -1:
 		return "general error";
 	default:
diff --git a/reftable/reftable-error.h b/reftable/reftable-error.h
index 6368cd9ed9..f404826562 100644
--- a/reftable/reftable-error.h
+++ b/reftable/reftable-error.h
@@ -57,6 +57,9 @@ enum reftable_error {
 
 	/* Trying to write out-of-date data. */
 	REFTABLE_OUTDATED_ERROR = -12,
+
+	/* An allocation has failed due to an out-of-memory situation. */
+	REFTABLE_OUT_OF_MEMORY_ERROR = -13,
 };
 
 /* convert the numeric error code to a string. The string should not be
-- 
2.46.2.852.g229c0bf0e5.dirty


  reply	other threads:[~2024-09-30  8:08 UTC|newest]

Thread overview: 151+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2024-09-16 12:28 [PATCH 00/22] reftable: handle allocation errors Patrick Steinhardt
2024-09-16 12:28 ` [PATCH 01/22] reftable/error: introduce out-of-memory error code Patrick Steinhardt
2024-09-16 12:28 ` [PATCH 02/22] reftable/basics: merge "publicbasics" into "basics" Patrick Steinhardt
2024-09-16 12:28 ` [PATCH 03/22] reftable: introduce `reftable_strdup()` Patrick Steinhardt
2024-09-16 12:28 ` [PATCH 04/22] reftable/basics: handle allocation failures in `reftable_calloc()` Patrick Steinhardt
2024-09-21 19:37   ` Junio C Hamano
2024-09-24  5:48     ` Patrick Steinhardt
2024-09-24  6:02       ` Patrick Steinhardt
2024-09-24 16:39       ` Junio C Hamano
2024-09-16 12:28 ` [PATCH 05/22] reftable/basics: handle allocation failures in `parse_names()` Patrick Steinhardt
2024-09-16 12:28 ` [PATCH 06/22] reftable/record: handle allocation failures on copy Patrick Steinhardt
2024-09-16 12:28 ` [PATCH 07/22] reftable/record: handle allocation failures when decoding records Patrick Steinhardt
2024-09-16 12:28 ` [PATCH 08/22] reftable/writer: handle allocation failures in `writer_index_hash()` Patrick Steinhardt
2024-09-16 12:28 ` [PATCH 09/22] reftable/writer: handle allocation failures in `reftable_new_writer()` Patrick Steinhardt
2024-09-16 12:28 ` [PATCH 10/22] reftable/merged: handle allocation failures in `merged_table_init_iter()` Patrick Steinhardt
2024-09-16 12:28 ` [PATCH 11/22] reftable/reader: handle allocation failures for unindexed reader Patrick Steinhardt
2024-09-16 12:28 ` [PATCH 12/22] reftable/reader: handle allocation failures in `reader_init_iter()` Patrick Steinhardt
2024-09-16 12:28 ` [PATCH 13/22] reftable/stack: handle allocation failures on reload Patrick Steinhardt
2024-09-16 12:28 ` [PATCH 14/22] reftable/stack: handle allocation failures in `reftable_new_stack()` Patrick Steinhardt
2024-09-16 12:29 ` [PATCH 15/22] reftable/stack: handle allocation failures in `stack_compact_range()` Patrick Steinhardt
2024-09-16 12:29 ` [PATCH 16/22] reftable/stack: handle allocation failures in auto compaction Patrick Steinhardt
2024-09-16 12:29 ` [PATCH 17/22] reftable/iter: handle allocation failures when creating indexed table iter Patrick Steinhardt
2024-09-22  6:26   ` Junio C Hamano
2024-09-24  5:49     ` Patrick Steinhardt
2024-09-16 12:29 ` [PATCH 18/22] reftable/blocksource: handle allocation failures Patrick Steinhardt
2024-09-16 12:29 ` [PATCH 19/22] reftable/block: " Patrick Steinhardt
2024-09-16 12:29 ` [PATCH 20/22] reftable/pq: handle allocation failures when adding entries Patrick Steinhardt
2024-09-16 12:29 ` [PATCH 21/22] reftable/tree: handle allocation failures Patrick Steinhardt
2024-09-16 12:29 ` [PATCH 22/22] reftable: handle trivial " Patrick Steinhardt
2024-09-24  6:31 ` [PATCH v2 00/22] reftable: handle allocation errors Patrick Steinhardt
2024-09-24  6:31   ` [PATCH v2 01/22] reftable/error: introduce out-of-memory error code Patrick Steinhardt
2024-09-24  6:31   ` [PATCH v2 02/22] reftable/basics: merge "publicbasics" into "basics" Patrick Steinhardt
2024-09-24  6:31   ` [PATCH v2 03/22] reftable: introduce `reftable_strdup()` Patrick Steinhardt
2024-09-24  6:32   ` [PATCH v2 04/22] reftable/basics: handle allocation failures in `reftable_calloc()` Patrick Steinhardt
2024-09-24 16:59     ` Junio C Hamano
2024-09-26 12:11       ` Patrick Steinhardt
2024-09-26 16:13         ` Junio C Hamano
2024-09-27  5:28           ` Patrick Steinhardt
2024-09-27 12:21             ` Han-Wen Nienhuys
2024-09-27 15:21               ` Junio C Hamano
2024-09-24  6:32   ` [PATCH v2 05/22] reftable/basics: handle allocation failures in `parse_names()` Patrick Steinhardt
2024-09-24 22:19     ` René Scharfe
2024-09-26 12:09       ` Patrick Steinhardt
2024-09-24  6:32   ` [PATCH v2 06/22] reftable/record: handle allocation failures on copy Patrick Steinhardt
2024-09-24  6:32   ` [PATCH v2 07/22] reftable/record: handle allocation failures when decoding records Patrick Steinhardt
2024-09-24  6:32   ` [PATCH v2 08/22] reftable/writer: handle allocation failures in `writer_index_hash()` Patrick Steinhardt
2024-09-24  6:32   ` [PATCH v2 09/22] reftable/writer: handle allocation failures in `reftable_new_writer()` Patrick Steinhardt
2024-09-24  6:32   ` [PATCH v2 10/22] reftable/merged: handle allocation failures in `merged_table_init_iter()` Patrick Steinhardt
2024-09-24  6:32   ` [PATCH v2 11/22] reftable/reader: handle allocation failures for unindexed reader Patrick Steinhardt
2024-09-24  6:32   ` [PATCH v2 12/22] reftable/reader: handle allocation failures in `reader_init_iter()` Patrick Steinhardt
2024-09-24  6:32   ` [PATCH v2 13/22] reftable/stack: handle allocation failures on reload Patrick Steinhardt
2024-09-24  6:32   ` [PATCH v2 14/22] reftable/stack: handle allocation failures in `reftable_new_stack()` Patrick Steinhardt
2024-09-24  6:32   ` [PATCH v2 15/22] reftable/stack: handle allocation failures in `stack_compact_range()` Patrick Steinhardt
2024-09-24  6:32   ` [PATCH v2 16/22] reftable/stack: handle allocation failures in auto compaction Patrick Steinhardt
2024-09-24  6:32   ` [PATCH v2 17/22] reftable/iter: handle allocation failures when creating indexed table iter Patrick Steinhardt
2024-09-24  6:32   ` [PATCH v2 18/22] reftable/blocksource: handle allocation failures Patrick Steinhardt
2024-09-24  6:32   ` [PATCH v2 19/22] reftable/block: " Patrick Steinhardt
2024-09-24  6:32   ` [PATCH v2 20/22] reftable/pq: handle allocation failures when adding entries Patrick Steinhardt
2024-09-24  6:33   ` [PATCH v2 21/22] reftable/tree: handle allocation failures Patrick Steinhardt
2024-09-24  6:33   ` [PATCH v2 22/22] reftable: handle trivial " Patrick Steinhardt
2024-09-30  8:08 ` [PATCH v3 00/22] refatble: handle allocation errors Patrick Steinhardt
2024-09-30  8:08   ` Patrick Steinhardt [this message]
2024-09-30  8:08   ` [PATCH v3 02/22] reftable/basics: merge "publicbasics" into "basics" Patrick Steinhardt
2024-09-30  8:08   ` [PATCH v3 03/22] reftable: introduce `reftable_strdup()` Patrick Steinhardt
2024-09-30  8:08   ` [PATCH v3 04/22] reftable/basics: handle allocation failures in `reftable_calloc()` Patrick Steinhardt
2024-09-30  8:08   ` [PATCH v3 05/22] reftable/basics: handle allocation failures in `parse_names()` Patrick Steinhardt
2024-09-30 17:40     ` René Scharfe
2024-09-30  8:08   ` [PATCH v3 06/22] reftable/record: handle allocation failures on copy Patrick Steinhardt
2024-09-30  8:08   ` [PATCH v3 07/22] reftable/record: handle allocation failures when decoding records Patrick Steinhardt
2024-09-30  8:08   ` [PATCH v3 08/22] reftable/writer: handle allocation failures in `writer_index_hash()` Patrick Steinhardt
2024-09-30  8:08   ` [PATCH v3 09/22] reftable/writer: handle allocation failures in `reftable_new_writer()` Patrick Steinhardt
2024-09-30 17:40     ` René Scharfe
2024-09-30 18:22       ` Patrick Steinhardt
2024-09-30 19:11         ` Junio C Hamano
2024-09-30  8:08   ` [PATCH v3 10/22] reftable/merged: handle allocation failures in `merged_table_init_iter()` Patrick Steinhardt
2024-09-30  8:08   ` [PATCH v3 11/22] reftable/reader: handle allocation failures for unindexed reader Patrick Steinhardt
2024-09-30  8:08   ` [PATCH v3 12/22] reftable/reader: handle allocation failures in `reader_init_iter()` Patrick Steinhardt
2024-09-30  8:08   ` [PATCH v3 13/22] reftable/stack: handle allocation failures on reload Patrick Steinhardt
2024-09-30  8:08   ` [PATCH v3 14/22] reftable/stack: handle allocation failures in `reftable_new_stack()` Patrick Steinhardt
2024-09-30  8:08   ` [PATCH v3 15/22] reftable/stack: handle allocation failures in `stack_compact_range()` Patrick Steinhardt
2024-09-30  8:08   ` [PATCH v3 16/22] reftable/stack: handle allocation failures in auto compaction Patrick Steinhardt
2024-09-30  8:09   ` [PATCH v3 17/22] reftable/iter: handle allocation failures when creating indexed table iter Patrick Steinhardt
2024-09-30  8:09   ` [PATCH v3 18/22] reftable/blocksource: handle allocation failures Patrick Steinhardt
2024-09-30  8:09   ` [PATCH v3 19/22] reftable/block: " Patrick Steinhardt
2024-09-30  8:09   ` [PATCH v3 20/22] reftable/pq: handle allocation failures when adding entries Patrick Steinhardt
2024-09-30  8:09   ` [PATCH v3 21/22] reftable/tree: handle allocation failures Patrick Steinhardt
2024-09-30  8:09   ` [PATCH v3 22/22] reftable: handle trivial " Patrick Steinhardt
2024-09-30 18:18   ` [PATCH v3 00/22] refatble: handle allocation errors Junio C Hamano
2024-10-01  9:41 ` [PATCH v4 00/25] reftable: " Patrick Steinhardt
2024-10-01  9:41   ` [PATCH v4 01/25] reftable/error: introduce out-of-memory error code Patrick Steinhardt
2024-10-01  9:41   ` [PATCH v4 02/25] reftable/basics: merge "publicbasics" into "basics" Patrick Steinhardt
2024-10-01  9:41   ` [PATCH v4 03/25] reftable: introduce `reftable_strdup()` Patrick Steinhardt
2024-10-01  9:41   ` [PATCH v4 04/25] reftable/basics: handle allocation failures in `reftable_calloc()` Patrick Steinhardt
2024-10-01  9:41   ` [PATCH v4 05/25] reftable/basics: handle allocation failures in `parse_names()` Patrick Steinhardt
2024-10-01  9:42   ` [PATCH v4 06/25] reftable/record: handle allocation failures on copy Patrick Steinhardt
2024-10-01  9:42   ` [PATCH v4 07/25] reftable/record: handle allocation failures when decoding records Patrick Steinhardt
2024-10-01  9:42   ` [PATCH v4 08/25] reftable/writer: handle allocation failures in `writer_index_hash()` Patrick Steinhardt
2024-10-01  9:42   ` [PATCH v4 09/25] reftable/writer: handle allocation failures in `reftable_new_writer()` Patrick Steinhardt
2024-10-01  9:42   ` [PATCH v4 10/25] reftable/merged: handle allocation failures in `merged_table_init_iter()` Patrick Steinhardt
2024-10-01  9:42   ` [PATCH v4 11/25] reftable/reader: handle allocation failures for unindexed reader Patrick Steinhardt
2024-10-01  9:42   ` [PATCH v4 12/25] reftable/reader: handle allocation failures in `reader_init_iter()` Patrick Steinhardt
2024-10-01  9:42   ` [PATCH v4 13/25] reftable/stack: handle allocation failures on reload Patrick Steinhardt
2024-10-01  9:42   ` [PATCH v4 14/25] reftable/stack: handle allocation failures in `reftable_new_stack()` Patrick Steinhardt
2024-10-01  9:42   ` [PATCH v4 15/25] reftable/stack: handle allocation failures in `stack_compact_range()` Patrick Steinhardt
2024-10-01  9:42   ` [PATCH v4 16/25] reftable/stack: handle allocation failures in auto compaction Patrick Steinhardt
2024-10-01  9:42   ` [PATCH v4 17/25] reftable/iter: handle allocation failures when creating indexed table iter Patrick Steinhardt
2024-10-01  9:42   ` [PATCH v4 18/25] reftable/blocksource: handle allocation failures Patrick Steinhardt
2024-10-01  9:42   ` [PATCH v4 19/25] reftable/block: " Patrick Steinhardt
2024-10-01  9:42   ` [PATCH v4 20/25] reftable/pq: handle allocation failures when adding entries Patrick Steinhardt
2024-10-01  9:42   ` [PATCH v4 21/25] reftable/tree: handle allocation failures Patrick Steinhardt
2024-10-01  9:42   ` [PATCH v4 22/25] reftable: handle trivial " Patrick Steinhardt
2024-10-01  9:42   ` [PATCH v4 23/25] reftable: fix calls to free(3P) Patrick Steinhardt
2024-10-01  9:43   ` [PATCH v4 24/25] reftable: introduce `REFTABLE_FREE_AND_NULL()` Patrick Steinhardt
2024-10-01  9:43   ` [PATCH v4 25/25] reftable/basics: ban standard allocator functions Patrick Steinhardt
2024-10-01 22:50     ` Junio C Hamano
2024-10-02  4:30       ` Patrick Steinhardt
2024-10-01 17:52   ` [PATCH v4 00/25] reftable: handle allocation errors Junio C Hamano
2024-10-01 18:30     ` René Scharfe
2024-10-01 19:25       ` Junio C Hamano
2024-10-02  4:29         ` Patrick Steinhardt
2024-10-02 18:04           ` Junio C Hamano
2024-10-02 10:55 ` [PATCH v5 " Patrick Steinhardt
2024-10-02 10:55   ` [PATCH v5 01/25] reftable/error: introduce out-of-memory error code Patrick Steinhardt
2024-10-02 10:55   ` [PATCH v5 02/25] reftable/basics: merge "publicbasics" into "basics" Patrick Steinhardt
2024-10-02 10:55   ` [PATCH v5 03/25] reftable: introduce `reftable_strdup()` Patrick Steinhardt
2024-10-02 10:55   ` [PATCH v5 04/25] reftable/basics: handle allocation failures in `reftable_calloc()` Patrick Steinhardt
2024-10-02 10:55   ` [PATCH v5 05/25] reftable/basics: handle allocation failures in `parse_names()` Patrick Steinhardt
2024-10-02 22:07     ` Eric Sunshine
2024-10-04  4:58       ` Patrick Steinhardt
2024-10-04  5:43         ` Eric Sunshine
2024-10-02 10:55   ` [PATCH v5 06/25] reftable/record: handle allocation failures on copy Patrick Steinhardt
2024-10-02 10:55   ` [PATCH v5 07/25] reftable/record: handle allocation failures when decoding records Patrick Steinhardt
2024-10-02 10:55   ` [PATCH v5 08/25] reftable/writer: handle allocation failures in `writer_index_hash()` Patrick Steinhardt
2024-10-02 10:55   ` [PATCH v5 09/25] reftable/writer: handle allocation failures in `reftable_new_writer()` Patrick Steinhardt
2024-10-02 10:55   ` [PATCH v5 10/25] reftable/merged: handle allocation failures in `merged_table_init_iter()` Patrick Steinhardt
2024-10-02 10:55   ` [PATCH v5 11/25] reftable/reader: handle allocation failures for unindexed reader Patrick Steinhardt
2024-10-02 10:55   ` [PATCH v5 12/25] reftable/reader: handle allocation failures in `reader_init_iter()` Patrick Steinhardt
2024-10-02 10:56   ` [PATCH v5 13/25] reftable/stack: handle allocation failures on reload Patrick Steinhardt
2024-10-02 10:56   ` [PATCH v5 14/25] reftable/stack: handle allocation failures in `reftable_new_stack()` Patrick Steinhardt
2024-10-02 10:56   ` [PATCH v5 15/25] reftable/stack: handle allocation failures in `stack_compact_range()` Patrick Steinhardt
2024-10-02 10:56   ` [PATCH v5 16/25] reftable/stack: handle allocation failures in auto compaction Patrick Steinhardt
2024-10-02 10:56   ` [PATCH v5 17/25] reftable/iter: handle allocation failures when creating indexed table iter Patrick Steinhardt
2024-10-02 10:56   ` [PATCH v5 18/25] reftable/blocksource: handle allocation failures Patrick Steinhardt
2024-10-02 10:56   ` [PATCH v5 19/25] reftable/block: " Patrick Steinhardt
2024-10-02 10:56   ` [PATCH v5 20/25] reftable/pq: handle allocation failures when adding entries Patrick Steinhardt
2024-10-02 10:56   ` [PATCH v5 21/25] reftable/tree: handle allocation failures Patrick Steinhardt
2024-10-02 10:56   ` [PATCH v5 22/25] reftable: handle trivial " Patrick Steinhardt
2024-10-02 10:56   ` [PATCH v5 23/25] reftable: fix calls to free(3P) Patrick Steinhardt
2024-10-02 10:56   ` [PATCH v5 24/25] reftable: introduce `REFTABLE_FREE_AND_NULL()` Patrick Steinhardt
2024-10-02 10:56   ` [PATCH v5 25/25] reftable/basics: ban standard allocator functions Patrick Steinhardt
2024-10-02 19:32   ` [PATCH v5 00/25] reftable: handle allocation errors Junio C Hamano

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=8c99ecc3255b1b10cdec047da01804dc23a7d287.1727680272.git.ps@pks.im \
    --to=ps@pks.im \
    --cc=ethomson@edwardthomson.com \
    --cc=git@vger.kernel.org \
    --cc=gitster@pobox.com \
    --cc=l.s.r@web.de \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).