get_relocated_path: the configured paths are not looked for?

get_relocated_path: the configured paths are not looked for?

[Michael Tokarev]

Hello!

Today I discovered an interesting issue here: I copied a system-installed
binary into another directory, in order to debug an unrelated issue. Just
to discover it does not work, being unable to find any modules or data
files.

Here's how the strace of typical qemu-system-i386 run looks like (the
relevant parts only):

access("/tmp/qemu-bundle", R_OK) = -1 ENOENT (No such file or directory)
access("/tmp/b/../lib/x86_64-linux-gnu/qemu/accel-tcg-i386.so", F_OK) = -1 ENOENT (No such file or directory)
access("/var/run/qemu/Debian_1_8.0~rc4+dfsg-3/accel-tcg-i386.so", F_OK) = -1 ENOENT (No such file or directory)

(the executable in this case is in /tmp, obviously).  And it fails with
error "fatal: could not load module for type 'tcg-accel-ops'".

This is despite the fact that qemu has been configured with proper --libdir
and other --foodir to point to actual dirs such as /usr/lib /usr/share etc.

Looking at the code I see this, in cutil.c:get_relocated_path() (simplified):

char *get_relocated_path(const char *dir)
{
     const char *bindir = CONFIG_BINDIR;
     const char *exec_dir = qemu_get_exec_dir();

     result = concat(exec_dir, "/qemu-bundle");
     if (access(result, R_OK) == 0) {  <== should be X_OK and should be dir, too!
         g_string_append(result, dir);
     } else if (!starts_with_prefix(dir) || !starts_with_prefix(bindir)) {
         g_string_assign(result, dir);
     } else {
         g_string_assign(result, exec_dir);
         .. search in  execdir and its parents ..
     }

This seems to be questionable.  And btw, even when running qemu from
the installed /usr/bin/ location, it does something weird:

access("/usr/bin/qemu-bundle", R_OK)    = -1 ENOENT (No such file or directory)
access("/usr/bin/../lib/x86_64-linux-gnu/qemu/accel-tcg-x86_64.so", F_OK) = 0
openat(AT_FDCWD, "/usr/bin/../lib/x86_64-linux-gnu/qemu/accel-tcg-x86_64.so", O_RDONLY|O_CLOEXEC) = 14

so it is not obvious why it skips the full path in this case (the 2nd
choice) and falls into the 3rd variant (search in execdir and parents).

Why it *ever* wants to search in parent dirs relative to executable
path? And why these !starts_with_prefix()?

BTW, the second starts_with_prefix(bindir) condition is just stupid, -
bindir is CONFIG_BINDIR (static), and start_with_prefix() checks for
CONFIG_PREFIX which is also static.

The whole this logic smells.. strange at best :)

How about this: if qemu-bundle exists, use qemu-bundle/dir, instead
use dir directly?  Why do we need all other conditions? Why do we
do different things depending on the place the executable is located?

In the past, qemu tried to look in pc-bios/ and other similar places, -
when run from the build dir.  But since it does not do this anymore,
we can just skip whole thing, no?

/mjt

Re: get_relocated_path: the configured paths are not looked for?

[Akihiko Odaki]

On 2023/04/19 16:32, Michael Tokarev wrote:
> Hello!
> 
> Today I discovered an interesting issue here: I copied a system-installed
> binary into another directory, in order to debug an unrelated issue. Just
> to discover it does not work, being unable to find any modules or data
> files.
> 
> Here's how the strace of typical qemu-system-i386 run looks like (the
> relevant parts only):
> 
> access("/tmp/qemu-bundle", R_OK) = -1 ENOENT (No such file or directory)
> access("/tmp/b/../lib/x86_64-linux-gnu/qemu/accel-tcg-i386.so", F_OK) = 
> -1 ENOENT (No such file or directory)
> access("/var/run/qemu/Debian_1_8.0~rc4+dfsg-3/accel-tcg-i386.so", F_OK) 
> = -1 ENOENT (No such file or directory)
> 
> (the executable in this case is in /tmp, obviously).  And it fails with
> error "fatal: could not load module for type 'tcg-accel-ops'".
> 
> This is despite the fact that qemu has been configured with proper --libdir
> and other --foodir to point to actual dirs such as /usr/lib /usr/share etc.

Some files in QEMU installation is closely coupled with the binary so it 
does not make much sense to copy only the executable to another 
directory. You need to copy all of the files QEMU owns to relocate a 
QEMU installation. QEMU uses relative paths to find such relocated files.

That said, it is indeed confusing that QEMU uses relative paths even if 
you specify an absolute path for --libdir. I prefer to require to 
specify relative paths for --libdir and other options to make a QEMU 
installation relocatable, but I didn't dare making such a breaking change.

Regards,
Akihiko Odaki

> 
> Looking at the code I see this, in cutil.c:get_relocated_path() 
> (simplified):
> 
> char *get_relocated_path(const char *dir)
> {
>      const char *bindir = CONFIG_BINDIR;
>      const char *exec_dir = qemu_get_exec_dir();
> 
>      result = concat(exec_dir, "/qemu-bundle");
>      if (access(result, R_OK) == 0) {  <== should be X_OK and should be 
> dir, too!
>          g_string_append(result, dir);
>      } else if (!starts_with_prefix(dir) || !starts_with_prefix(bindir)) {
>          g_string_assign(result, dir);
>      } else {
>          g_string_assign(result, exec_dir);
>          .. search in  execdir and its parents ..
>      }
> 
> This seems to be questionable.  And btw, even when running qemu from
> the installed /usr/bin/ location, it does something weird:
> 
> access("/usr/bin/qemu-bundle", R_OK)    = -1 ENOENT (No such file or 
> directory)
> access("/usr/bin/../lib/x86_64-linux-gnu/qemu/accel-tcg-x86_64.so", 
> F_OK) = 0
> openat(AT_FDCWD, 
> "/usr/bin/../lib/x86_64-linux-gnu/qemu/accel-tcg-x86_64.so", 
> O_RDONLY|O_CLOEXEC) = 14
> 
> so it is not obvious why it skips the full path in this case (the 2nd
> choice) and falls into the 3rd variant (search in execdir and parents).
> 
> Why it *ever* wants to search in parent dirs relative to executable
> path? And why these !starts_with_prefix()?
> 
> BTW, the second starts_with_prefix(bindir) condition is just stupid, -
> bindir is CONFIG_BINDIR (static), and start_with_prefix() checks for
> CONFIG_PREFIX which is also static.
> 
> The whole this logic smells.. strange at best :)
> 
> How about this: if qemu-bundle exists, use qemu-bundle/dir, instead
> use dir directly?  Why do we need all other conditions? Why do we
> do different things depending on the place the executable is located?
> 
> In the past, qemu tried to look in pc-bios/ and other similar places, -
> when run from the build dir.  But since it does not do this anymore,
> we can just skip whole thing, no?
> 
> /mjt

Re: get_relocated_path: the configured paths are not looked for?

[Michael Tokarev]

20.04.2023 08:29, Akihiko Odaki wrote:
> On 2023/04/19 16:32, Michael Tokarev wrote:
>> Hello!
>>
>> Today I discovered an interesting issue here: I copied a system-installed
>> binary into another directory, in order to debug an unrelated issue. Just
>> to discover it does not work, being unable to find any modules or data
>> files.
>>
>> Here's how the strace of typical qemu-system-i386 run looks like (the
>> relevant parts only):
>>
>> access("/tmp/qemu-bundle", R_OK) = -1 ENOENT (No such file or directory)
>> access("/tmp/b/../lib/x86_64-linux-gnu/qemu/accel-tcg-i386.so", F_OK) = -1 ENOENT (No such file or directory)
>> access("/var/run/qemu/Debian_1_8.0~rc4+dfsg-3/accel-tcg-i386.so", F_OK) = -1 ENOENT (No such file or directory)
>>
>> (the executable in this case is in /tmp, obviously).  And it fails with
>> error "fatal: could not load module for type 'tcg-accel-ops'".
>>
>> This is despite the fact that qemu has been configured with proper --libdir
>> and other --foodir to point to actual dirs such as /usr/lib /usr/share etc.
> 
> Some files in QEMU installation is closely coupled with the binary so it does not make much sense to copy only the executable to another directory. 
> You need to copy all of the files QEMU owns to relocate a QEMU installation. QEMU uses relative paths to find such relocated files.
> 
> That said, it is indeed confusing that QEMU uses relative paths even if you specify an absolute path for --libdir. I prefer to require to specify 
> relative paths for --libdir and other options to make a QEMU installation relocatable, but I didn't dare making such a breaking change.

Well, quite often it makes little sense still.

For example, in debian we had to (temporarily) move qemu-system-i386 from
/usr/bin/ to /usr/libexec/, replacing the /usr/bin/ one with a shell wrapper
(to decouple xen hvm build out of main qemu build). It was just by a chance
the directory nesting is the same still, - I wanted to move it to /usr/lib/qemu/
instead.  But at least this one still works.  Ditto for the actual xen version,
the binary is in /usr/libexec/qemu-system-i386-xen now, I was about to move it
to /usr/libexec/xen/qemu-system-i386 instead.

I see absolutely no reason for the binary to look into /usr/libexec/share/qemu/bios.bin
if it is told to get all data files from /usr/share/qemu/.

Quite often I debug issues and have to compare "old" qemu with current one
(exactly like I did in this case), extracting the old binary into /tmp/.
And wonder why it breaks in other ways.

And I still don't see how this turning absolute paths into relative (behind the
scenes) can be useful. If we put qemu into /usr/local/bin for example, it
can be configured to look for data files in /usr/local/share just fine,
there's no need for this "magic". On the other hand, if we really want
to use a different data dir, we can pass a right -L option.

I think I'll just remove this (very questionable) behavior at least from the
Debian package.  I already patched out the previous version of this, when
it looked at ${exe_path}/../pc-bios - this at least made sense when we
needed to find firmware files for just-built qemu, in the source dir. But
it makes no sense (in my opinion) to do that for the installed version.

Thanks,

/mjt

Re: get_relocated_path: the configured paths are not looked for?

[Akihiko Odaki]

On 2023/04/23 19:28, Michael Tokarev wrote:
> 20.04.2023 08:29, Akihiko Odaki wrote:
>> On 2023/04/19 16:32, Michael Tokarev wrote:
>>> Hello!
>>>
>>> Today I discovered an interesting issue here: I copied a 
>>> system-installed
>>> binary into another directory, in order to debug an unrelated issue. 
>>> Just
>>> to discover it does not work, being unable to find any modules or data
>>> files.
>>>
>>> Here's how the strace of typical qemu-system-i386 run looks like (the
>>> relevant parts only):
>>>
>>> access("/tmp/qemu-bundle", R_OK) = -1 ENOENT (No such file or directory)
>>> access("/tmp/b/../lib/x86_64-linux-gnu/qemu/accel-tcg-i386.so", F_OK) 
>>> = -1 ENOENT (No such file or directory)
>>> access("/var/run/qemu/Debian_1_8.0~rc4+dfsg-3/accel-tcg-i386.so", 
>>> F_OK) = -1 ENOENT (No such file or directory)
>>>
>>> (the executable in this case is in /tmp, obviously).  And it fails with
>>> error "fatal: could not load module for type 'tcg-accel-ops'".
>>>
>>> This is despite the fact that qemu has been configured with proper 
>>> --libdir
>>> and other --foodir to point to actual dirs such as /usr/lib 
>>> /usr/share etc.
>>
>> Some files in QEMU installation is closely coupled with the binary so 
>> it does not make much sense to copy only the executable to another 
>> directory. You need to copy all of the files QEMU owns to relocate a 
>> QEMU installation. QEMU uses relative paths to find such relocated files.
>>
>> That said, it is indeed confusing that QEMU uses relative paths even 
>> if you specify an absolute path for --libdir. I prefer to require to 
>> specify relative paths for --libdir and other options to make a QEMU 
>> installation relocatable, but I didn't dare making such a breaking 
>> change.
> 
> Well, quite often it makes little sense still.
> 
> For example, in debian we had to (temporarily) move qemu-system-i386 from
> /usr/bin/ to /usr/libexec/, replacing the /usr/bin/ one with a shell 
> wrapper
> (to decouple xen hvm build out of main qemu build). It was just by a chance
> the directory nesting is the same still, - I wanted to move it to 
> /usr/lib/qemu/
> instead.  But at least this one still works.  Ditto for the actual xen 
> version,
> the binary is in /usr/libexec/qemu-system-i386-xen now, I was about to 
> move it
> to /usr/libexec/xen/qemu-system-i386 instead.
> 
> I see absolutely no reason for the binary to look into 
> /usr/libexec/share/qemu/bios.bin
> if it is told to get all data files from /usr/share/qemu/.

What about specifying --bindir=/usr/libexec/xen?

> 
> Quite often I debug issues and have to compare "old" qemu with current one
> (exactly like I did in this case), extracting the old binary into /tmp/.
> And wonder why it breaks in other ways.
> 
> And I still don't see how this turning absolute paths into relative 
> (behind the
> scenes) can be useful. If we put qemu into /usr/local/bin for example, it
> can be configured to look for data files in /usr/local/share just fine,
> there's no need for this "magic". On the other hand, if we really want
> to use a different data dir, we can pass a right -L option.

As you say, data files are not tightly coupled with QEMU version and it 
is quite possible to run QEMU with data files from another QEMU version.

However it's not true for all files QEMU installs. Especially module 
files like accel-tcg-i386.so are tightly coupled with the executable and 
QEMU is not expected to work if the exact modules which came with the 
executable are not provided.

> 
> I think I'll just remove this (very questionable) behavior at least from 
> the
> Debian package.  I already patched out the previous version of this, when
> it looked at ${exe_path}/../pc-bios - this at least made sense when we
> needed to find firmware files for just-built qemu, in the source dir. But
> it makes no sense (in my opinion) to do that for the installed version.

It uses qemu-bundle directory for just-built QEMU so it does not matter 
for the scenario.

The scenario where this conversion for relative path makes sense is the 
case you need to compare the behavior of two packaged versions of QEMU, 
for example. In such case, you can extract each package into a different 
directory and run them. Each QEMU executable will use relative paths to 
find modules tied with it.

If you are removing this behavior from Debian package you may consult 
other package maintainers, especially GCC package maintainers. If I 
remember correctly, GCC also has similar logic to find executables like 
cc1 and shared objects like LTO plugins.

Regards,
Akihiko Odaki

> 
> Thanks,
> 
> /mjt

Re: get_relocated_path: the configured paths are not looked for?

[Michael Tokarev]

23.04.2023 14:22, Akihiko Odaki пишет:
> On 2023/04/23 19:28, Michael Tokarev wrote:
>> 20.04.2023 08:29, Akihiko Odaki wrote:
>>> On 2023/04/19 16:32, Michael Tokarev wrote:
>>>> Hello!
>>>>
>>>> Today I discovered an interesting issue here: I copied a system-installed
>>>> binary into another directory, in order to debug an unrelated issue. Just
>>>> to discover it does not work, being unable to find any modules or data
>>>> files.
>>>>
>>>> Here's how the strace of typical qemu-system-i386 run looks like (the
>>>> relevant parts only):
>>>>
>>>> access("/tmp/qemu-bundle", R_OK) = -1 ENOENT (No such file or directory)
>>>> access("/tmp/b/../lib/x86_64-linux-gnu/qemu/accel-tcg-i386.so", F_OK) = -1 ENOENT (No such file or directory)
>>>> access("/var/run/qemu/Debian_1_8.0~rc4+dfsg-3/accel-tcg-i386.so", F_OK) = -1 ENOENT (No such file or directory)
>>>>
>>>> (the executable in this case is in /tmp, obviously).  And it fails with
>>>> error "fatal: could not load module for type 'tcg-accel-ops'".
>>>>
>>>> This is despite the fact that qemu has been configured with proper --libdir
>>>> and other --foodir to point to actual dirs such as /usr/lib /usr/share etc.
>>>
>>> Some files in QEMU installation is closely coupled with the binary so it does not make much sense to copy only the executable to another directory. 
>>> You need to copy all of the files QEMU owns to relocate a QEMU installation. QEMU uses relative paths to find such relocated files.
>>>
>>> That said, it is indeed confusing that QEMU uses relative paths even if you specify an absolute path for --libdir. I prefer to require to specify 
>>> relative paths for --libdir and other options to make a QEMU installation relocatable, but I didn't dare making such a breaking change.
>>
>> Well, quite often it makes little sense still.
>>
>> For example, in debian we had to (temporarily) move qemu-system-i386 from
>> /usr/bin/ to /usr/libexec/, replacing the /usr/bin/ one with a shell wrapper
>> (to decouple xen hvm build out of main qemu build). It was just by a chance
>> the directory nesting is the same still, - I wanted to move it to /usr/lib/qemu/
>> instead.  But at least this one still works.  Ditto for the actual xen version,
>> the binary is in /usr/libexec/qemu-system-i386-xen now, I was about to move it
>> to /usr/libexec/xen/qemu-system-i386 instead.
>>
>> I see absolutely no reason for the binary to look into /usr/libexec/share/qemu/bios.bin
>> if it is told to get all data files from /usr/share/qemu/.
> 
> What about specifying --bindir=/usr/libexec/xen?

It will try to find other binaries in /usr/libexec/xen/, which are actually in
/usr/bin.

> As you say, data files are not tightly coupled with QEMU version and it is quite possible to run QEMU with data files from another QEMU version.

Nope and nope: first, I never said it (it was you who said that), and second,
it is quite okay to mix-n-match data files with qemu binaries. We're way past
the times when each minor qemu release required its own build of vgabios.

If we're to require qemu-version-specific data dir, it would be something
like /usr/share/qemu/$qemu_version/bios.bin.

> However it's not true for all files QEMU installs. Especially module files like accel-tcg-i386.so are tightly coupled with the executable and QEMU is 
> not expected to work if the exact modules which came with the executable are not provided.

The .so files are different. There are also other modules (block-iscsi.so).
*Those* are already in version-specific subdir of a lbdir.

>> I think I'll just remove this (very questionable) behavior at least from the
>> Debian package.  I already patched out the previous version of this, when
>> it looked at ${exe_path}/../pc-bios - this at least made sense when we
>> needed to find firmware files for just-built qemu, in the source dir. But
>> it makes no sense (in my opinion) to do that for the installed version.
> 
> It uses qemu-bundle directory for just-built QEMU so it does not matter for the scenario.

Yes. Still I want to be able to run tests, - this is exactly what the older code
looking for ../pc-bios/ did, and what current qemu-bundle code does.

> The scenario where this conversion for relative path makes sense is the case you need to compare the behavior of two packaged versions of QEMU, for 
> example. In such case, you can extract each package into a different directory and run them. Each QEMU executable will use relative paths to find 
> modules tied with it.

This is exactly what I do, - comparing qemu binaries from different packaged
builds. And this is exactly where it fails, - because the "other" binary is
not finding the binary files even if it were built with --qemu-datadir=/usr/share/qemu/
and all the binaries is there.

It should be the other way around: if you want this qemu to use a non-standard
path for firmware, - say, when you extracted "another" package in a temp dir
and need that one to use its own data dir - you run it wiht -L argument.

Thanks,

/mjt

Re: get_relocated_path: the configured paths are not looked for?

[Akihiko Odaki]

On 2023/04/23 20:31, Michael Tokarev wrote:
> 23.04.2023 14:22, Akihiko Odaki пишет:
>> On 2023/04/23 19:28, Michael Tokarev wrote:
>>> 20.04.2023 08:29, Akihiko Odaki wrote:
>>>> On 2023/04/19 16:32, Michael Tokarev wrote:
>>>>> Hello!
>>>>>
>>>>> Today I discovered an interesting issue here: I copied a 
>>>>> system-installed
>>>>> binary into another directory, in order to debug an unrelated 
>>>>> issue. Just
>>>>> to discover it does not work, being unable to find any modules or data
>>>>> files.
>>>>>
>>>>> Here's how the strace of typical qemu-system-i386 run looks like (the
>>>>> relevant parts only):
>>>>>
>>>>> access("/tmp/qemu-bundle", R_OK) = -1 ENOENT (No such file or 
>>>>> directory)
>>>>> access("/tmp/b/../lib/x86_64-linux-gnu/qemu/accel-tcg-i386.so", 
>>>>> F_OK) = -1 ENOENT (No such file or directory)
>>>>> access("/var/run/qemu/Debian_1_8.0~rc4+dfsg-3/accel-tcg-i386.so", 
>>>>> F_OK) = -1 ENOENT (No such file or directory)
>>>>>
>>>>> (the executable in this case is in /tmp, obviously).  And it fails 
>>>>> with
>>>>> error "fatal: could not load module for type 'tcg-accel-ops'".
>>>>>
>>>>> This is despite the fact that qemu has been configured with proper 
>>>>> --libdir
>>>>> and other --foodir to point to actual dirs such as /usr/lib 
>>>>> /usr/share etc.
>>>>
>>>> Some files in QEMU installation is closely coupled with the binary 
>>>> so it does not make much sense to copy only the executable to 
>>>> another directory. You need to copy all of the files QEMU owns to 
>>>> relocate a QEMU installation. QEMU uses relative paths to find such 
>>>> relocated files.
>>>>
>>>> That said, it is indeed confusing that QEMU uses relative paths even 
>>>> if you specify an absolute path for --libdir. I prefer to require to 
>>>> specify relative paths for --libdir and other options to make a QEMU 
>>>> installation relocatable, but I didn't dare making such a breaking 
>>>> change.
>>>
>>> Well, quite often it makes little sense still.
>>>
>>> For example, in debian we had to (temporarily) move qemu-system-i386 
>>> from
>>> /usr/bin/ to /usr/libexec/, replacing the /usr/bin/ one with a shell 
>>> wrapper
>>> (to decouple xen hvm build out of main qemu build). It was just by a 
>>> chance
>>> the directory nesting is the same still, - I wanted to move it to 
>>> /usr/lib/qemu/
>>> instead.  But at least this one still works.  Ditto for the actual 
>>> xen version,
>>> the binary is in /usr/libexec/qemu-system-i386-xen now, I was about 
>>> to move it
>>> to /usr/libexec/xen/qemu-system-i386 instead.
>>>
>>> I see absolutely no reason for the binary to look into 
>>> /usr/libexec/share/qemu/bios.bin
>>> if it is told to get all data files from /usr/share/qemu/.
>>
>> What about specifying --bindir=/usr/libexec/xen?
> 
> It will try to find other binaries in /usr/libexec/xen/, which are 
> actually in
> /usr/bin.

So you want to place only qemu-system-i386 (and perhaps 
qemu-system-x86_64?) into /usr/libexec/xen but others in /usr/bin? That 
scenario is certainly not covered with the current QEMU and it will need 
to be patched for it.

> 
>> As you say, data files are not tightly coupled with QEMU version and 
>> it is quite possible to run QEMU with data files from another QEMU 
>> version.
> 
> Nope and nope: first, I never said it (it was you who said that), and 
> second,
> it is quite okay to mix-n-match data files with qemu binaries. We're way 
> past
> the times when each minor qemu release required its own build of vgabios.
> 
> If we're to require qemu-version-specific data dir, it would be something
> like /usr/share/qemu/$qemu_version/bios.bin.

I think we don't have any disagreement here. Data files are *not* 
specific to a QEMU version, and you *can* just use the same data files 
for different QEMU versions. No need for qemu-version-specific data dir.

> 
>> However it's not true for all files QEMU installs. Especially module 
>> files like accel-tcg-i386.so are tightly coupled with the executable 
>> and QEMU is not expected to work if the exact modules which came with 
>> the executable are not provided.
> 
> The .so files are different. There are also other modules (block-iscsi.so).
> *Those* are already in version-specific subdir of a lbdir.

So that is how QEMU is configured for Debian? That's perfectly 
legitimate, but not all distributions do that.

Also, that works only if the version changed. For example, consider the 
case where you patched a module downstream. To compare the behaviors of 
the patched and unpatched ones, you'll need to copy the modules 
somewhere else.

> 
>>> I think I'll just remove this (very questionable) behavior at least 
>>> from the
>>> Debian package.  I already patched out the previous version of this, 
>>> when
>>> it looked at ${exe_path}/../pc-bios - this at least made sense when we
>>> needed to find firmware files for just-built qemu, in the source dir. 
>>> But
>>> it makes no sense (in my opinion) to do that for the installed version.
>>
>> It uses qemu-bundle directory for just-built QEMU so it does not 
>> matter for the scenario.
> 
> Yes. Still I want to be able to run tests, - this is exactly what the 
> older code
> looking for ../pc-bios/ did, and what current qemu-bundle code does.
> 
>> The scenario where this conversion for relative path makes sense is 
>> the case you need to compare the behavior of two packaged versions of 
>> QEMU, for example. In such case, you can extract each package into a 
>> different directory and run them. Each QEMU executable will use 
>> relative paths to find modules tied with it.
> 
> This is exactly what I do, - comparing qemu binaries from different 
> packaged
> builds. And this is exactly where it fails, - because the "other" binary is
> not finding the binary files even if it were built with 
> --qemu-datadir=/usr/share/qemu/
> and all the binaries is there.
> 
> It should be the other way around: if you want this qemu to use a 
> non-standard
> path for firmware, - say, when you extracted "another" package in a temp 
> dir
> and need that one to use its own data dir - you run it wiht -L argument.

Let's focus on modules in this discussion. There should be no problem 
with data files here. You can pick data files from a different QEMU 
version and it should just work fine. And even if it unfortunately does 
not work, you can still use -L option to fix it.

That's not the case for modules. Modules are coupled with the executable 
so you need to copy them along with the executable and the executable 
should be able to find them.

Regards,
Akihiko Odaki

> 
> Thanks,
> 
> /mjt

Re: get_relocated_path: the configured paths are not looked for?

[Michael Tokarev]

23.04.2023 14:47, Akihiko Odaki пишет:

https://salsa.debian.org/qemu-team/qemu/-/commit/e017f53a8550d0bcaaca81c6dacac8ec34295cf0
fwiw.

> Also, that works only if the version changed. For example, consider the case where you patched a module downstream. To compare the behaviors of the 
> patched and unpatched ones, you'll need to copy the modules somewhere else.
> 
> Let's focus on modules in this discussion. There should be no problem with data files here. You can pick data files from a different QEMU version and 
> it should just work fine. And even if it unfortunately does not work, you can still use -L option to fix it.
> 
> That's not the case for modules. Modules are coupled with the executable so you need to copy them along with the executable and the executable should 
> be able to find them.

The modules case is actually trivial.  For this one, we have
$QEMU_MODULE_DIR which, if set, will be searched first.

There's no need to make tricks and turn --libdir or --datadir
specified at configure time as absolute paths, into something
entirely unpredictable.

/mjt

Re: get_relocated_path: the configured paths are not looked for?

[Akihiko Odaki]

On 2023/04/23 22:22, Michael Tokarev wrote:
> 23.04.2023 14:47, Akihiko Odaki пишет:
> 
> https://salsa.debian.org/qemu-team/qemu/-/commit/e017f53a8550d0bcaaca81c6dacac8ec34295cf0
> fwiw.

I seriously think you better consult GCC and other package maintainers 
to have consensus on handling this kind of scenario. Otherwise you don't 
get the behavior you expect from other packages.

> 
>> Also, that works only if the version changed. For example, consider 
>> the case where you patched a module downstream. To compare the 
>> behaviors of the patched and unpatched ones, you'll need to copy the 
>> modules somewhere else.
>>
>> Let's focus on modules in this discussion. There should be no problem 
>> with data files here. You can pick data files from a different QEMU 
>> version and it should just work fine. And even if it unfortunately 
>> does not work, you can still use -L option to fix it.
>>
>> That's not the case for modules. Modules are coupled with the 
>> executable so you need to copy them along with the executable and the 
>> executable should be able to find them.
> 
> The modules case is actually trivial.  For this one, we have
> $QEMU_MODULE_DIR which, if set, will be searched first.
> 
> There's no need to make tricks and turn --libdir or --datadir
> specified at configure time as absolute paths, into something
> entirely unpredictable.
> 
> /mjt

It is more preferable to use modules which are bundled with the 
executable by default since they are coupled with the executable and you 
should never want to use alternative modules unless you are debugging 
QEMU. The current logic can reliably find the modules if either relative 
paths or absolute paths of the executable and modules are preserved.

That said, it's very reasonable to specify absolute paths to --libdir 
when you want to relocate only the executable (e.g. moving 
bin/qemu-system-i386 into libexec/xen/qemu-system-i386) but keep other 
files in the configured path, and the current QEMU build scripts do not 
allow that. If you explain a convincing reason for doing that, I think 
there is a good chance to get a patch to cover such a case merged.

But of course that is something a maintainer decides and I'm not 
responsible here. util/cutils.c has no maintainer listed and the last 
change made for get_relocated_path() was merged by Paolo Bonzini. He is 
also a maintainer of the build infrastructure.

Regards,
Akihiko Odaki

Re: get_relocated_path: the configured paths are not looked for?

[Michael Tokarev]

23.04.2023 20:39, Akihiko Odaki пишет:
> On 2023/04/23 22:22, Michael Tokarev wrote:
>> 23.04.2023 14:47, Akihiko Odaki пишет:
>>
>> https://salsa.debian.org/qemu-team/qemu/-/commit/e017f53a8550d0bcaaca81c6dacac8ec34295cf0
>> fwiw.
> 
> I seriously think you better consult GCC and other package maintainers to have consensus on handling this kind of scenario. Otherwise you don't get 
> the behavior you expect from other packages.

I know no other software which does this.  It's interesting you mentioned GCC, -
I've seen it is doing that for years, wondered why.  But it works just fine
when moved elsewhere - I just tried compiling a hello.c program by
/tmp/gcc, it works exactly the same way as compiled by /usr/bin/gcc.
And wast majority of software available on linux does not do these funny
tricks with relative-to-executable paths (I can't say for *all* software,
but it is definitely uncommon and I know no other examples).  Only qemu
is broken in this context.  The patch above fixes this breakage.

/mjt

Re: get_relocated_path: the configured paths are not looked for?

[Akihiko Odaki]

On 2023/04/24 3:10, Michael Tokarev wrote:
> 23.04.2023 20:39, Akihiko Odaki пишет:
>> On 2023/04/23 22:22, Michael Tokarev wrote:
>>> 23.04.2023 14:47, Akihiko Odaki пишет:
>>>
>>> https://salsa.debian.org/qemu-team/qemu/-/commit/e017f53a8550d0bcaaca81c6dacac8ec34295cf0
>>> fwiw.
>>
>> I seriously think you better consult GCC and other package maintainers 
>> to have consensus on handling this kind of scenario. Otherwise you 
>> don't get the behavior you expect from other packages.
> 
> I know no other software which does this.  It's interesting you 
> mentioned GCC, -
> I've seen it is doing that for years, wondered why.  But it works just fine
> when moved elsewhere - I just tried compiling a hello.c program by
> /tmp/gcc, it works exactly the same way as compiled by /usr/bin/gcc.
> And wast majority of software available on linux does not do these funny
> tricks with relative-to-executable paths (I can't say for *all* software,
> but it is definitely uncommon and I know no other examples).  Only qemu
> is broken in this context.  The patch above fixes this breakage.
> 
> /mjt

Well for me GCC can't find cc1 as I pointed out earlier. The below is a 
simple reproduction case using Podman:
podman run --rm -i debian <<EOS
apt-get update
apt-get install -y gcc
cp /usr/bin/gcc /tmp
cat > a.c <<EOC
int main()
{
}
EOC
/tmp/gcc a.c
EOS

The output ends with:
gcc: fatal error: cannot execute 'cc1': execvp: No such file or directory
compilation terminated.

This is because GCC uses relative paths to find cc1 and other files.

Regards,
Akihiko Odaki

Re: get_relocated_path: the configured paths are not looked for?

[Michael Tokarev]

23.04.2023 21:24, Akihiko Odaki wrote:

> The output ends with:
> gcc: fatal error: cannot execute 'cc1': execvp: No such file or directory
> compilation terminated.

$ cd /tmp; printf '#include <stdio.h>\nint main(){puts("Hello!");return 0;}' > hello.c; cp /usr/bin/gcc .; ./gcc hello.c; ./a.out ; ./gcc --version; 
ls -l gcc; cd /tmp
Hello!
gcc (Debian 12.2.0-14) 12.2.0
Copyright (C) 2022 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

-rwxr-xr-x 1 mjt mjt 1301496 апр 23 21:30 gcc*

This is debian bookworm.  Ditto for debian bullseye (gcc-10 is in there).

It is interesting that it doesn't work, say, in a subdir of /tmp/, the
same way as in your case above.

Still, GCC is an exception. It is very rare.

/mjt

Re: get_relocated_path: the configured paths are not looked for?

[Michael Tokarev]

23.04.2023 21:33, Michael Tokarev пишет:

> $ cd /tmp; printf '#include <stdio.h>\nint main(){puts("Hello!");return 0;}' > hello.c; cp /usr/bin/gcc .; ./gcc hello.c; ./a.out ; ./gcc --version; 
> ls -l gcc; cd /tmp
> Hello!

execve("/tmp/../lib/gcc/x86_64-linux-gnu/12/cc1", [...])

Since this is merged-usr system, it finds its component just fine,
that's why it works here in /tmp, doesn't work in a subdir of /tmp,
and doesn't work in your case (non-merged /usr).

> Still, GCC is an exception. It is very rare.

/mjt

Re: get_relocated_path: the configured paths are not looked for?

[Akihiko Odaki]

On 2023/04/24 3:37, Michael Tokarev wrote:
> 23.04.2023 21:33, Michael Tokarev пишет:
> 
>> $ cd /tmp; printf '#include <stdio.h>\nint 
>> main(){puts("Hello!");return 0;}' > hello.c; cp /usr/bin/gcc .; ./gcc 
>> hello.c; ./a.out ; ./gcc --version; ls -l gcc; cd /tmp
>> Hello!
> 
> execve("/tmp/../lib/gcc/x86_64-linux-gnu/12/cc1", [...])
> 
> Since this is merged-usr system, it finds its component just fine,
> that's why it works here in /tmp, doesn't work in a subdir of /tmp,
> and doesn't work in your case (non-merged /usr).
> 
>> Still, GCC is an exception. It is very rare.

Oh, I was just wondering why it worked and digging what's going on. 
Thanks for the answer.

Well, I have nothing to say GCC is an exception or not, but it's 
certainly an important program so I still think it's better to talk with 
GCC packagers (and upstream developers perhaps?)

Regards,
Akihiko Odaki

> 
> /mjt
>