[PATCH] use xstrdup, not strdup in ll-merge.c

[PATCH] use xstrdup, not strdup in ll-merge.c

[Jim Meyering]

Otherwise, a fluky allocation failure would cause merge
configuration settings to be silently ignored.

Signed-off-by: Jim Meyering <meyering@redhat.com>
---
 ll-merge.c |    8 ++++----
 1 files changed, 4 insertions(+), 4 deletions(-)

diff --git a/ll-merge.c b/ll-merge.c
index 31d6f0a..9168958 100644
--- a/ll-merge.c
+++ b/ll-merge.c
@@ -231,7 +231,7 @@ static int read_merge_config(const char *var, const char *value, void *cb)

 	if (!strcmp(var, "merge.default")) {
 		if (value)
-			default_ll_merge = strdup(value);
+			default_ll_merge = xstrdup(value);
 		return 0;
 	}

@@ -265,7 +265,7 @@ static int read_merge_config(const char *var, const char *value, void *cb)
 	if (!strcmp("name", ep)) {
 		if (!value)
 			return error("%s: lacks value", var);
-		fn->description = strdup(value);
+		fn->description = xstrdup(value);
 		return 0;
 	}

@@ -288,14 +288,14 @@ static int read_merge_config(const char *var, const char *value, void *cb)
 		 * file named by %A, and signal that it has done with zero exit
 		 * status.
 		 */
-		fn->cmdline = strdup(value);
+		fn->cmdline = xstrdup(value);
 		return 0;
 	}

 	if (!strcmp("recursive", ep)) {
 		if (!value)
 			return error("%s: lacks value", var);
-		fn->recursive = strdup(value);
+		fn->recursive = xstrdup(value);
 		return 0;
 	}

--
1.6.3.2.406.gd6a466

Re: [PATCH] use xstrdup, not strdup in ll-merge.c

[Alex Riesen]

2009/6/14 Jim Meyering <jim@meyering.net>:
> @@ -231,7 +231,7 @@ static int read_merge_config(const char *var, const char *value, void *cb)
>
>        if (!strcmp(var, "merge.default")) {
>                if (value)
> -                       default_ll_merge = strdup(value);
> +                       default_ll_merge = xstrdup(value);

read_merge_config has a failure mode (where it returns -1), why not use it?

Re: [PATCH] use xstrdup, not strdup in ll-merge.c

[Jim Meyering]

Alex Riesen wrote:
> 2009/6/14 Jim Meyering <jim@meyering.net>:
>> @@ -231,7 +231,7 @@ static int read_merge_config(const char *var, const char *value, void *cb)
>>
>>        if (!strcmp(var, "merge.default")) {
>>                if (value)
>> -                       default_ll_merge = strdup(value);
>> +                       default_ll_merge = xstrdup(value);
>
> read_merge_config has a failure mode (where it returns -1), why not use it?

I didn't even consider it, because it would be inconsistent with
the other heap-allocation functions used there (xcalloc, xmemdupz).

However, now that I do, it looks like that would mean adding four times
the same code (including conditionals and code to generate a diagnostic via
a call to error -- or a goto). Why bother, when all of that is already
encapsulated in xmalloc?
Maybe because you want to be able to continue after an allocation failure?
If a small strdup allocation fails, odds are good that the code won't
be able to do anything useful, so when not in library code, cleanest is
simply to exit.

In addition, if you insist on using strdup, you'll probably want to
be consistent and use calloc and memdupz, too.  Adding all of the code
required to recover from those failures and to avoid leaks would be messy.

Re: [PATCH] use xstrdup, not strdup in ll-merge.c

[Alex Riesen]

2009/6/15 Jim Meyering <jim@meyering.net>:
> Alex Riesen wrote:
>> 2009/6/14 Jim Meyering <jim@meyering.net>:
>>> @@ -231,7 +231,7 @@ static int read_merge_config(const char *var, const char *value, void *cb)
>>>
>>>        if (!strcmp(var, "merge.default")) {
>>>                if (value)
>>> -                       default_ll_merge = strdup(value);
>>> +                       default_ll_merge = xstrdup(value);
>>
>> read_merge_config has a failure mode (where it returns -1), why not use it?
>
> I didn't even consider it, because it would be inconsistent with
> the other heap-allocation functions used there (xcalloc, xmemdupz).
>
> However, now that I do, it looks like that would mean adding four times
> the same code (including conditionals and code to generate a diagnostic via
> a call to error -- or a goto). Why bother, when all of that is already
> encapsulated in xmalloc?

So that a useful error message can be given in the _caller_ (it knows
more about context)?

Otherwise the error message ("Out of memory, strdup failed") does not
have anything about the place nor situation in it. As the situations
when a modern system really runs out of memory are very rare,
mostly such reports just point at some inconsistency elsewhere
(like bloody stupid memory management in system support libraries
on an OS-not-to-be-named-again or the usual corruption of heap
control structures).

Besides, xstrdup does more then just allocation: it tries to free global
list of cached pack chunks. This does not play very well with the efforts
to make a library out of the modern Git code.

> Maybe because you want to be able to continue after an allocation failure?

No.

> If a small strdup allocation fails, odds are good that the code won't
> be able to do anything useful, so when not in library code, cleanest is
> simply to exit.

Doubt it (because you better describe _why_ you "simply" interrupted
users workflow, so the said user can do something about it).

> In addition, if you insist on using strdup, you'll probably want to
> be consistent and use calloc and memdupz, too.  Adding all of the code
> required to recover from those failures and to avoid leaks would be messy.

I don't insist on that. I should have said in the first message about
more elaborate explanation of the error to user. Sorry.

Re: [PATCH] use xstrdup, not strdup in ll-merge.c

[Jim Meyering]

Alex Riesen wrote:
> 2009/6/15 Jim Meyering <jim@meyering.net>:
>> Alex Riesen wrote:
>>> 2009/6/14 Jim Meyering <jim@meyering.net>:
>>>> @@ -231,7 +231,7 @@ static int read_merge_config(const char *var, const char *value, void *cb)
>>>>
>>>>        if (!strcmp(var, "merge.default")) {
>>>>                if (value)
>>>> -                       default_ll_merge = strdup(value);
>>>> +                       default_ll_merge = xstrdup(value);
>>>
>>> read_merge_config has a failure mode (where it returns -1), why not use it?
>>
>> I didn't even consider it, because it would be inconsistent with
>> the other heap-allocation functions used there (xcalloc, xmemdupz).
>>
>> However, now that I do, it looks like that would mean adding four times
>> the same code (including conditionals and code to generate a diagnostic via
>> a call to error -- or a goto). Why bother, when all of that is already
>> encapsulated in xmalloc?
>
> So that a useful error message can be given in the _caller_ (it knows
> more about context)?

So you want to tell the user that we failed
to strdup the "merge.default" value?
Or the "driver" value?

Given the apparently-high cost/benefit ratio, I would not bother.  I.e.,
this failure is so unlikely to trigger, and when it does, knowing for which
config value strdup failed is even less likely to be useful,
that minimal diagnostics should be fine.

Of more general interest, when xstrdup fails, it might be useful to
include in the diagnostic how long the would-be-dup'd string was.  I.e.,
rather than saying

    die("Out of memory, strdup failed");
say
    die("Out of memory, failed to strdup a %lu-byte string",
        (unsigned long int) strlen(str));

Then, you have at least a clue as to whether the failure
is due to some ridiculously-long string value, or to some
unrelated-to-config, systemic problem.

> Otherwise the error message ("Out of memory, strdup failed") does not
> have anything about the place nor situation in it. As the situations
> when a modern system really runs out of memory are very rare,
> mostly such reports just point at some inconsistency elsewhere

Exactly.  This is why I think it's not worthwhile to invest in
a more precise diagnostic, here.

> (like bloody stupid memory management in system support libraries
> on an OS-not-to-be-named-again or the usual corruption of heap
> control structures).
>
> Besides, xstrdup does more then just allocation: it tries to free global
> list of cached pack chunks. This does not play very well with the efforts
> to make a library out of the modern Git code.

Ahh... librarification.  This is a slightly different topic.
I see existing uses of xcalloc and xmemdupz, not to
mention "error" calls, and conclude that this function is
not library caliber code, so there's no need to invest.

If you want a version of this function that is more library-friendly,
then that will be more work.  However, I think librarification should
be addressed separately from this simple patch to avoid a potential NULL
dereference (and *no* diagnostic).

Re: [PATCH] use xstrdup, not strdup in ll-merge.c

[Alex Riesen]

2009/6/15 Jim Meyering <jim@meyering.net>:
> Alex Riesen wrote:
>> 2009/6/15 Jim Meyering <jim@meyering.net>:
>>> Alex Riesen wrote:
>>>> 2009/6/14 Jim Meyering <jim@meyering.net>:
>>>>> @@ -231,7 +231,7 @@ static int read_merge_config(const char *var, const char *value, void *cb)
>>>>>
>>>>>        if (!strcmp(var, "merge.default")) {
>>>>>                if (value)
>>>>> -                       default_ll_merge = strdup(value);
>>>>> +                       default_ll_merge = xstrdup(value);
>>>>
>>>> read_merge_config has a failure mode (where it returns -1), why not use it?
>>>
>>> I didn't even consider it, because it would be inconsistent with
>>> the other heap-allocation functions used there (xcalloc, xmemdupz).
>>>
>>> However, now that I do, it looks like that would mean adding four times
>>> the same code (including conditionals and code to generate a diagnostic via
>>> a call to error -- or a goto). Why bother, when all of that is already
>>> encapsulated in xmalloc?
>>
>> So that a useful error message can be given in the _caller_ (it knows
>> more about context)?
>
> So you want to tell the user that we failed
> to strdup the "merge.default" value?
> Or the "driver" value?

"merge: recursive: error loading configuration (last seen:
merge.default): Out of memory\n"

> Of more general interest, when xstrdup fails, it might be useful to
> include in the diagnostic how long the would-be-dup'd string was.  I.e.,
> rather than saying
>
>    die("Out of memory, strdup failed");
> say
>    die("Out of memory, failed to strdup a %lu-byte string",
>        (unsigned long int) strlen(str));

Yes. Still lacks higher level information, though.

>> Otherwise the error message ("Out of memory, strdup failed") does not
>> have anything about the place nor situation in it. As the situations
>> when a modern system really runs out of memory are very rare,
>> mostly such reports just point at some inconsistency elsewhere
>
> Exactly.  This is why I think it's not worthwhile to invest in
> a more precise diagnostic, here.

I disagree. It is already hard to find starting point for debugging if
the failed code is just a layer: the config of ll-merge is called not only
from the merge drivers, but also indirectly from the programs which
call the merge itself. Now, go figure where has it failed...

>> (like bloody stupid memory management in system support libraries
>> on an OS-not-to-be-named-again or the usual corruption of heap
>> control structures).
>>
>> Besides, xstrdup does more then just allocation: it tries to free global
>> list of cached pack chunks. This does not play very well with the efforts
>> to make a library out of the modern Git code.
>
> Ahh... librarification.  This is a slightly different topic.
> I see existing uses of xcalloc and xmemdupz, not to
> mention "error" calls, and conclude that this function is
> not library caliber code, so there's no need to invest.

Well, error() does not finish the programs, and the rest (in just my opinion)
suffer the same problems except where it is used in cmd_-functions
(IOW, in top-level caller).

> If you want a version of this function that is more library-friendly,
> then that will be more work.  However, I think librarification should
> be addressed separately from this simple patch to avoid a potential NULL
> dereference (and *no* diagnostic).

I do not explicitely _want_ librarification in this particular context.
I'm just pointing out that you just made another (maybe small) obstacle for it.

Re: [PATCH] use xstrdup, not strdup in ll-merge.c

[Jim Meyering]

Alex Riesen wrote:

> 2009/6/15 Jim Meyering <jim@meyering.net>:
>> Alex Riesen wrote:
>>> 2009/6/15 Jim Meyering <jim@meyering.net>:
>>>> Alex Riesen wrote:
>>>>> 2009/6/14 Jim Meyering <jim@meyering.net>:
>>>>>> @@ -231,7 +231,7 @@ static int read_merge_config(const char *var, const char *value, void *cb)
>>>>>>
>>>>>>        if (!strcmp(var, "merge.default")) {
>>>>>>                if (value)
>>>>>> -                       default_ll_merge = strdup(value);
>>>>>> +                       default_ll_merge = xstrdup(value);
>>>>>
>>>>> read_merge_config has a failure mode (where it returns -1), why not use it?
>>>>
>>>> I didn't even consider it, because it would be inconsistent with
>>>> the other heap-allocation functions used there (xcalloc, xmemdupz).
>>>>
>>>> However, now that I do, it looks like that would mean adding four times
>>>> the same code (including conditionals and code to generate a diagnostic via
>>>> a call to error -- or a goto). Why bother, when all of that is already
>>>> encapsulated in xmalloc?
>>>
>>> So that a useful error message can be given in the _caller_ (it knows
>>> more about context)?
>>
>> So you want to tell the user that we failed
>> to strdup the "merge.default" value?
>> Or the "driver" value?
>
> "merge: recursive: error loading configuration (last seen:
> merge.default): Out of memory\n"
>
>> Of more general interest, when xstrdup fails, it might be useful to
>> include in the diagnostic how long the would-be-dup'd string was.  I.e.,
>> rather than saying
>>
>>    die("Out of memory, strdup failed");
>> say
>>    die("Out of memory, failed to strdup a %lu-byte string",
>>        (unsigned long int) strlen(str));
>
> Yes. Still lacks higher level information, though.
>
>>> Otherwise the error message ("Out of memory, strdup failed") does not
>>> have anything about the place nor situation in it. As the situations
>>> when a modern system really runs out of memory are very rare,
>>> mostly such reports just point at some inconsistency elsewhere
>>
>> Exactly.  This is why I think it's not worthwhile to invest in
>> a more precise diagnostic, here.
>
> I disagree. It is already hard to find starting point for debugging if
> the failed code is just a layer: the config of ll-merge is called not only
> from the merge drivers, but also indirectly from the programs which
> call the merge itself. Now, go figure where has it failed...

If you're convinced of the value of such a change, go for it.
Though it sounds like you're saying you'd prefer a stack trace.

Re: [PATCH] use xstrdup, not strdup in ll-merge.c

[Alex Riesen]

2009/6/15 Jim Meyering <jim@meyering.net>:
> Alex Riesen wrote:
>> 2009/6/15 Jim Meyering <jim@meyering.net>:
>>>
>>> Exactly.  This is why I think it's not worthwhile to invest in
>>> a more precise diagnostic, here.
>>
>> I disagree. It is already hard to find starting point for debugging if
>> the failed code is just a layer: the config of ll-merge is called not only
>> from the merge drivers, but also indirectly from the programs which
>> call the merge itself. Now, go figure where has it failed...
>
> If you're convinced of the value of such a change, go for it.

As much as I'd like to know as much as possible about why something
failed, I can't make a failure handling automatically simple (given the tools).
I do suggest using goto to handle this particular OOM (below), but it looks
almost too ugly. Maybe I am just paranoid, and am overdoing this particular
case.

> Though it sounds like you're saying you'd prefer a stack trace.

I haven't though of that, but yes, that would be perfect (except for
a small problem where it is impossible to do in a portable way,
and there is know way to get information about).

The "goto oom" patch:

diff --git a/ll-merge.c b/ll-merge.c
index 31d6f0a..4977f20 100644
--- a/ll-merge.c
+++ b/ll-merge.c
@@ -230,8 +230,11 @@ static int read_merge_config(const char *var,
const char *value, void *cb)
 	int namelen;

 	if (!strcmp(var, "merge.default")) {
-		if (value)
+		if (value) {
 			default_ll_merge = strdup(value);
+			if (!default_ll_merge)
+				goto oom;
+		}
 		return 0;
 	}

@@ -266,6 +269,8 @@ static int read_merge_config(const char *var,
const char *value, void *cb)
 		if (!value)
 			return error("%s: lacks value", var);
 		fn->description = strdup(value);
+		if (!fn->description)
+			goto oom;
 		return 0;
 	}

@@ -289,6 +294,8 @@ static int read_merge_config(const char *var,
const char *value, void *cb)
 		 * status.
 		 */
 		fn->cmdline = strdup(value);
+		if (!fn->cmdline)
+			goto oom;
 		return 0;
 	}

@@ -296,10 +303,15 @@ static int read_merge_config(const char *var,
const char *value, void *cb)
 		if (!value)
 			return error("%s: lacks value", var);
 		fn->recursive = strdup(value);
+		if (!fn->recursive)
+			goto oom;
 		return 0;
 	}

 	return 0;
+oom:
+	return error("line merge: %s%s%s: out of memory", var,
+		     value ? " = ": "", value ? value: "");
 }

 static void initialize_ll_merge(void)
@@ -307,7 +319,8 @@ static void initialize_ll_merge(void)
 	if (ll_user_merge_tail)
 		return;
 	ll_user_merge_tail = &ll_user_merge;
-	git_config(read_merge_config, NULL);
+	if (git_config(read_merge_config, NULL) < 0)
+		exit(1);
 }

 static const struct ll_merge_driver *find_ll_merge_driver(const char
*merge_attr)

Re: [PATCH] use xstrdup, not strdup in ll-merge.c

[Shawn O. Pearce]

Alex Riesen <raa.lkml@gmail.com> wrote:
> 2009/6/15 Jim Meyering <jim@meyering.net>:
> > Alex Riesen wrote:
> >> 2009/6/15 Jim Meyering <jim@meyering.net>:
> >>>
> >>> Exactly. ??This is why I think it's not worthwhile to invest in
> >>> a more precise diagnostic, here.

Sorry Alex, but I have to agree with Jim here.  Use xstrdup()
and its friends and move on with life.

> As much as I'd like to know as much as possible about why something
> failed, I can't make a failure handling automatically simple (given the tools).
> I do suggest using goto to handle this particular OOM (below), but it looks
> almost too ugly. Maybe I am just paranoid, and am overdoing this particular
> case.

Yes, this is horrible.  Its a huge amount of code for checking a
strdup failure that is unlikely to occur out of a configuration file.

Really, how big is that config file?  It can't be so large that a
strdup() is likely to run out of memory pulling a value from it.

And even if it does run out of memory, unless we can reclaim space
by shedding pack windows, we're doomed, we can't continue executing.
The official error is "out of memory", most likely due to too low
of a rlimit on our heap, and the only resolution is to boost the
rlimit higher, not something we can do in our process if we're
already hitting the hard limit.
 
> The "goto oom" patch:

NAK.  We don't do this anywhere else in git.  Please don't start now.
 
-- 
Shawn.

Re: [PATCH] use xstrdup, not strdup in ll-merge.c

[Alex Riesen]

2009/6/15 Shawn O. Pearce <spearce@spearce.org>:
> Alex Riesen <raa.lkml@gmail.com> wrote:
>> As much as I'd like to know as much as possible about why something
>> failed, I can't make a failure handling automatically simple (given the tools).
>> I do suggest using goto to handle this particular OOM (below), but it looks
>> almost too ugly. Maybe I am just paranoid, and am overdoing this particular
>> case.
>
> Yes, this is horrible.  Its a huge amount of code for checking a
> strdup failure that is unlikely to occur out of a configuration file.
>
> Really, how big is that config file?  It can't be so large that a
> strdup() is likely to run out of memory pulling a value from it.

Oh, I don't worry about config files being too big (it is noticeable
without this checks). I'm worried about our bugs triggering the
conditions where allocation routines return NULL, like heap corruption
which is hard to notice without valgrind.

>> The "goto oom" patch:
>
> NAK.  We don't do this anywhere else in git.  Please don't start now.

We'll have to start paying more attention to memory management sometime.
Although, this particular case may be not the best case for it.