Re: [PATCH] kexec: return -ENOEXEC from image probe functions on mismatch

[Mukesh Pilaniya <mpilaniy@redhat.com>]

Hi Pratyush,

On 18/08/26 3:14 pm, Pratyush Yadav wrote:
> On Fri, Aug 14 2026, Mukesh Pilaniya wrote:
> 
>> Hi Pratyush,
>> On 14/08/26 7:43 pm, Pratyush Yadav wrote:
>>> On Thu, Aug 13 2026, Mukesh Pilaniya wrote:
>>>
>>>> Several kexec_file_load() image probe functions return -EINVAL when
>>>> they do not recognize the image format.  A probe function that rejects
>>>> an image should return -ENOEXEC to indicate that the image is not a
>>>> recognized executable format.  -EINVAL implies a problem with the
>>>> syscall parameters, not with image recognition.
>>>>
>>>> kexec_image_probe_default() iterates through registered loaders and
>>>> returns the last probe's error code to the caller.  That error
>>>> propagates as the kexec_file_load() return value to userspace.
>>>> Returning -EINVAL from a probe when no loader matches is semantically
>>>> incorrect and misleads userspace about the nature of the failure.
>>>>
>>>> Return -ENOEXEC from all probe functions and their helpers when the
>>>> image format is not recognized.
>>>
>>> Sounds fine in principle but can you please also share what the real
>>> problem you face is and how changing these return codes helps? These
>>> error codes are uAPI and while we _can_ change them as long as we don't
>>> break something, there should be a clear motivation for doing so.
>>>
>>> [...]
>>>
>> While debugging a misleading error on s390x where kexec -s reported
>> "syscall kexec_file_load not available" instead of the actual EINVAL
>> from a kernel command line that exceeded the architecture limit, we
>> traced the problem to the kexec-tools userspace utility treating
>> EINVAL the same as ENOSYS and ENOEXEC -- as a signal to silently fall
>> back to kexec_load().
>>
>> kexec-tools supports two syscalls: kexec_file_load() and the older
>> kexec_load(). With -a (the default), it tries kexec_file_load()
>> first and falls back to kexec_load() when the syscall is not
>> implemented (ENOSYS) or the kernel does not have a loader for the
>> image format. With -s, it uses kexec_file_load() only with no
>> fallback.
>>
>> When the kernel returns -EINVAL it means something went wrong while
>> loading the image, not that the syscall is missing or the image
>> format is unrecognized. kexec-tools should not fall back to the
>> older syscall in that case. However, some kernel probe functions
>> currently return -EINVAL when the image header does not match,
>> instead of returning -ENOEXEC. Keeping EINVAL in the fallback set
>> to accommodate these probes has the side effect of also hiding
>> genuine loading errors like an oversized command line.
>>
>> kexec-tools should only fall back when kexec_file_load() is not
>> implemented or does not have a matching loader -- not when something
>> goes wrong during load.
>>
>> The fix on the kexec-tools side is to remove EINVAL from the fallback
>> set, but that requires the kernel to be clean first -- probe functions
>> must return -ENOEXEC when they do not recognize an image format, not -EINVAL.
> 
> Okay, the idea makes sense. Though I am curious if in practice there are
> images that kexec_file_load() can't load but kexec-tools can.
> 
> But then I went and looked at the code. The only caller of the probe
> functions I can see is kexec_image_probe_default(). Looking at its code:
> 
>   int kexec_image_probe_default(struct kimage *image, void *buf,
>   			      unsigned long buf_len)
>   {
>   	const struct kexec_file_ops * const *fops;
>   	int ret = -ENOEXEC;
>   
>   	for (fops = &kexec_file_loaders[0]; *fops && (*fops)->probe; ++fops) {
>   		ret = (*fops)->probe(buf, buf_len);
>   		if (!ret) {
>   			image->fops = *fops;
>   			return ret;
>   		}
>   	}
>   
>   	return ret;
>   }
> 
> It defaults ret to -ENOEXEC, which makes sense, but then lets it be
> over-written by each loader. So in practice, it returns what the _last_
> loader returned. _This_ makes no sense. Why should the last loader be
> any special?
Right, the last loader overwriting ret, the commit message describes
this as well. It makes no sense for the return value to depend on
whichever loader happens to be last in the array.

> 
> From reading the code, I think what the author of this wanted to do is
> to try all loaders, and return -ENOEXEC if none succeeded. But the code
> of course strays from that and ends up returning the last loader's
> value.IMHO the function should only try the next loader when a probe returns
-ENOEXEC, not for any other error like -ENOMEM, which is a real failure
and should be propagated immediately, not silently discarded.

> 
> So how about the below diff instead? (** only compile tested **)
> 
> --- 8< ---
> diff --git a/kernel/kexec_file.c b/kernel/kexec_file.c
> index 2bfbb2d144e6..cfb2b8cd5679 100644
> --- a/kernel/kexec_file.c
> +++ b/kernel/kexec_file.c
> @@ -67,17 +67,16 @@ int kexec_image_probe_default(struct kimage *image, void *buf,
>  			      unsigned long buf_len)
>  {
>  	const struct kexec_file_ops * const *fops;
> -	int ret = -ENOEXEC;
>  
>  	for (fops = &kexec_file_loaders[0]; *fops && (*fops)->probe; ++fops) {
> -		ret = (*fops)->probe(buf, buf_len);
> -		if (!ret) {
> +		if (!(*fops)->probe(buf, buf_len)) {
>  			image->fops = *fops;
> -			return ret;
> +			return 0;
>  		}
>  	}
>  
> -	return ret;
> +	/* No loader found. */
> +	return -ENOEXEC;
>  }
Nice catch, Pratyush but this discards all non-zero return values from
probe functions, which means real errors get swallowed. For example,
kexec_elf_probe() can return -ENOMEM when kzalloc() fails in
elf_read_phdrs(). With this diff, that -ENOMEM becomes -ENOEXEC, which
tells userspace "no loader found" when the actual problem was a memory
allocation failure.

>  
>  static void *kexec_image_load_default(struct kimage *image)
> 
How about the following instead? It keeps the probe return value and
distinguishes -ENOEXEC (format not recognized, try next loader) from any
other error (real failure, propagate immediately):

diff --git a/kernel/kexec_file.c b/kernel/kexec_file.c
index 59fb9d71e9d8..ceb5c97cedac 100644
--- a/kernel/kexec_file.c
+++ b/kernel/kexec_file.c
@@ -68,17 +68,19 @@ int kexec_image_probe_default(struct kimage *image,
void *buf,
 			      unsigned long buf_len)
 {
 	const struct kexec_file_ops * const *fops;
-	int ret = -ENOEXEC;
+	int ret;

 	for (fops = &kexec_file_loaders[0]; *fops && (*fops)->probe; ++fops) {
 		ret = (*fops)->probe(buf, buf_len);
-		if (!ret) {
+		if (ret == 0) {
 			image->fops = *fops;
-			return ret;
+			return 0;
 		}
+		if (ret != -ENOEXEC)
+			return ret;
 	}

-	return ret;
+	return -ENOEXEC;
 }

 static void *kexec_image_load_default(struct kimage *image)


This requires probe functions to return -ENOEXEC from fixes in patch 1.
I will add this as a separate patch on top in v2.

Does this looks good to you ?

--
Regards,
Mukesh Pilaniya