fsimage - no such file

fsimage - no such file

[Mohammad Hedayati]

I am having a problem with pygrub of the current (rev 25467)
xen-unstable pygrub in Ubuntu 12.04. pygrub can not read any image. I
have traced the python code and found the problem being in
fsimage.open(file, offset, bootfsoptions). It raises IOError "no such
file or directory". I even wrote the following code and got the same
error.

#!/usr/bin/python
import fsimage

print "Hello, World!"
try:
    fs = fsimage.open("/root/guest/mini.iso", 0, "")
except Exception as e:
    print "Exception ", e

It seems that libfsimage has a problem with current version of Ubuntu.

Re: fsimage - no such file

[Thanos Makatos]

I have the same problem on xen-unstable 54c8c9eaee92+. I think libfsimage comes with xen (in my repo it's under ./tools/libfsimage). I've investigated a bit and found that in my occasion libfsimage tries to access directory /usr/lib64/fs which doesn't exist, whereas /usr/lib/fs does. I linked /usr/lib64/fs to /usr/lib/fs but then I got other problems, don't remember what. I don't know if this has to do with the fact that my dom0 is 64bit (Debian unstable), or that the domU is 64bit (Debian unstable again), or both.

-----Original Message-----
From: xen-devel-bounces@lists.xen.org [mailto:xen-devel-bounces@lists.xen.org] On Behalf Of Mohammad Hedayati
Sent: 18 June 2012 23:24
To: xen-devel@lists.xensource.com
Subject: [Xen-devel] fsimage - no such file

I am having a problem with pygrub of the current (rev 25467) xen-unstable pygrub in Ubuntu 12.04. pygrub can not read any image. I have traced the python code and found the problem being in fsimage.open(file, offset, bootfsoptions). It raises IOError "no such file or directory". I even wrote the following code and got the same error.

#!/usr/bin/python
import fsimage

print "Hello, World!"
try:
    fs = fsimage.open("/root/guest/mini.iso", 0, "") except Exception as e:
    print "Exception ", e

It seems that libfsimage has a problem with current version of Ubuntu.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: fsimage - no such file

[Ian Campbell]

On Tue, 2012-06-19 at 09:35 +0100, Thanos Makatos wrote:
> I have the same problem on xen-unstable 54c8c9eaee92+. I think
> libfsimage comes with xen (in my repo it's under ./tools/libfsimage).
> I've investigated a bit and found that in my occasion libfsimage tries
> to access directory /usr/lib64/fs which doesn't exist,
> whereas /usr/lib/fs does. I linked /usr/lib64/fs to /usr/lib/fs but
> then I got other problems, don't remember what. I don't know if this
> has to do with the fact that my dom0 is 64bit (Debian unstable), or
> that the domU is 64bit (Debian unstable again), or both.

Aha I'd been puzzling over why people had been seeing this but I'm not! 

This very likely is to do with Debian and Ubuntu's transition to
"multiarch" which has necessitated removing the old compatibility
symlink /usr/lib64 -> /usr/lib.

lib64 is a RPMism which doesn't really apply to Debian and derivatives
and therefore you need to tweak your Xen config before building. I use
the following local configuration patch (I should probably do the
equivalent in .config):

# HG changeset patch
# User Ian Campbell <ian.campbell@citrix.com>
# Date 1339576073 -3600
# Node ID 6eabee6807b48c72bcc35a28170f0729500def85
# Parent  80c0677f0f8370a4542aab81ab93380b0dab25db
imported patch debian-lib-dir.patch

diff -r 80c0677f0f83 -r 6eabee6807b4 config/StdGNU.mk
--- a/config/StdGNU.mk	Wed Jun 13 09:27:53 2012 +0100
+++ b/config/StdGNU.mk	Wed Jun 13 09:27:53 2012 +0100
@@ -34,7 +34,7 @@ BINDIR = $(PREFIX)/bin
 INCLUDEDIR = $(PREFIX)/include
 LIBLEAFDIR = lib
 LIBLEAFDIR_x86_32 = lib
-LIBLEAFDIR_x86_64 ?= lib64
+LIBLEAFDIR_x86_64 ?= lib
 LIBDIR = $(PREFIX)/$(LIBLEAFDIR)
 LIBDIR_x86_32 = $(PREFIX)/$(LIBLEAFDIR_x86_32)
 LIBDIR_x86_64 = $(PREFIX)/$(LIBLEAFDIR_x86_64)

Can you try a fresh build with this applied and see if that helps.

If it does then I shall update
http://wiki.xen.org/wiki/Compiling_Xen_From_Source with this
information!

Roger -- is there any way we could automate this via autoconf for 4.2?

Ian.

Re: fsimage - no such file

[Mohammad Hedayati]

On Tue, Jun 19, 2012 at 2:27 PM, Ian Campbell <Ian.Campbell@citrix.com> wrote:
> On Tue, 2012-06-19 at 09:35 +0100, Thanos Makatos wrote:
>> I have the same problem on xen-unstable 54c8c9eaee92+. I think
>> libfsimage comes with xen (in my repo it's under ./tools/libfsimage).
>> I've investigated a bit and found that in my occasion libfsimage tries
>> to access directory /usr/lib64/fs which doesn't exist,
>> whereas /usr/lib/fs does. I linked /usr/lib64/fs to /usr/lib/fs but
>> then I got other problems, don't remember what. I don't know if this
>> has to do with the fact that my dom0 is 64bit (Debian unstable), or
>> that the domU is 64bit (Debian unstable again), or both.
>
> Aha I'd been puzzling over why people had been seeing this but I'm not!
>
> This very likely is to do with Debian and Ubuntu's transition to
> "multiarch" which has necessitated removing the old compatibility
> symlink /usr/lib64 -> /usr/lib.

Linking /usr/lib64/fs to /usr/lib/fs solved the problem for me. I have
patched StdGNU.mk so couldn't think of this to be the cause.

> lib64 is a RPMism which doesn't really apply to Debian and derivatives
> and therefore you need to tweak your Xen config before building. I use
> the following local configuration patch (I should probably do the
> equivalent in .config):
>
> # HG changeset patch
> # User Ian Campbell <ian.campbell@citrix.com>
> # Date 1339576073 -3600
> # Node ID 6eabee6807b48c72bcc35a28170f0729500def85
> # Parent  80c0677f0f8370a4542aab81ab93380b0dab25db
> imported patch debian-lib-dir.patch
>
> diff -r 80c0677f0f83 -r 6eabee6807b4 config/StdGNU.mk
> --- a/config/StdGNU.mk  Wed Jun 13 09:27:53 2012 +0100
> +++ b/config/StdGNU.mk  Wed Jun 13 09:27:53 2012 +0100
> @@ -34,7 +34,7 @@ BINDIR = $(PREFIX)/bin
>  INCLUDEDIR = $(PREFIX)/include
>  LIBLEAFDIR = lib
>  LIBLEAFDIR_x86_32 = lib
> -LIBLEAFDIR_x86_64 ?= lib64
> +LIBLEAFDIR_x86_64 ?= lib
>  LIBDIR = $(PREFIX)/$(LIBLEAFDIR)
>  LIBDIR_x86_32 = $(PREFIX)/$(LIBLEAFDIR_x86_32)
>  LIBDIR_x86_64 = $(PREFIX)/$(LIBLEAFDIR_x86_64)
>
> Can you try a fresh build with this applied and see if that helps.
I have applied this patch. I think libfsimage is somehow not being
affected with this. Yet, linking /usr/lib64/fs -> /usr/lib/fs solves
the problem.
>
> If it does then I shall update
> http://wiki.xen.org/wiki/Compiling_Xen_From_Source with this
> information!
>
> Roger -- is there any way we could automate this via autoconf for 4.2?
>
> Ian.
>

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: fsimage - no such file

[Matt Wilson]

On Tue, Jun 19, 2012 at 04:19:27AM -0700, Mohammad Hedayati wrote:
> > # Node ID 6eabee6807b48c72bcc35a28170f0729500def85
> > # Parent  80c0677f0f8370a4542aab81ab93380b0dab25db
> > imported patch debian-lib-dir.patch
> >
> > diff -r 80c0677f0f83 -r 6eabee6807b4 config/StdGNU.mk
> > --- a/config/StdGNU.mk  Wed Jun 13 09:27:53 2012 +0100
> > +++ b/config/StdGNU.mk  Wed Jun 13 09:27:53 2012 +0100
> > @@ -34,7 +34,7 @@ BINDIR = $(PREFIX)/bin
> >  INCLUDEDIR = $(PREFIX)/include
> >  LIBLEAFDIR = lib
> >  LIBLEAFDIR_x86_32 = lib
> > -LIBLEAFDIR_x86_64 ?= lib64
> > +LIBLEAFDIR_x86_64 ?= lib
> >  LIBDIR = $(PREFIX)/$(LIBLEAFDIR)
> >  LIBDIR_x86_32 = $(PREFIX)/$(LIBLEAFDIR_x86_32)
> >  LIBDIR_x86_64 = $(PREFIX)/$(LIBLEAFDIR_x86_64)
> >
> > Can you try a fresh build with this applied and see if that helps.
> I have applied this patch. I think libfsimage is somehow not being
> affected with this. Yet, linking /usr/lib64/fs -> /usr/lib/fs solves
> the problem.

libfsimage is going to blindly look in /usr/lib64 on non-Itanium
64-bit Linux platforms. See tools/libfsimage/common/fsimage_plugin.c:134

#if defined(FSIMAGE_FSDIR)
        if (fsdir == NULL)
                fsdir = FSIMAGE_FSDIR;
#elif defined(__sun__)
        if (fsdir == NULL)
                fsdir = "/usr/lib/fs";

        if (sizeof(void *) == 8)
                isadir = "64/";
#elif defined(__ia64__)
        if (fsdir == NULL)
                fsdir = "/usr/lib/fs";
#else
        if (fsdir == NULL) {
                if (sizeof(void *) == 8)
                        fsdir = "/usr/lib64/fs";
                else
                        fsdir = "/usr/lib/fs";
        }
#endif

Matt

Re: fsimage - no such file

[Ian Campbell]

On Tue, 2012-06-19 at 17:10 +0100, Matt Wilson wrote:
> On Tue, Jun 19, 2012 at 04:19:27AM -0700, Mohammad Hedayati wrote:
> > > # Node ID 6eabee6807b48c72bcc35a28170f0729500def85
> > > # Parent  80c0677f0f8370a4542aab81ab93380b0dab25db
> > > imported patch debian-lib-dir.patch
> > >
> > > diff -r 80c0677f0f83 -r 6eabee6807b4 config/StdGNU.mk
> > > --- a/config/StdGNU.mk  Wed Jun 13 09:27:53 2012 +0100
> > > +++ b/config/StdGNU.mk  Wed Jun 13 09:27:53 2012 +0100
> > > @@ -34,7 +34,7 @@ BINDIR = $(PREFIX)/bin
> > >  INCLUDEDIR = $(PREFIX)/include
> > >  LIBLEAFDIR = lib
> > >  LIBLEAFDIR_x86_32 = lib
> > > -LIBLEAFDIR_x86_64 ?= lib64
> > > +LIBLEAFDIR_x86_64 ?= lib
> > >  LIBDIR = $(PREFIX)/$(LIBLEAFDIR)
> > >  LIBDIR_x86_32 = $(PREFIX)/$(LIBLEAFDIR_x86_32)
> > >  LIBDIR_x86_64 = $(PREFIX)/$(LIBLEAFDIR_x86_64)
> > >
> > > Can you try a fresh build with this applied and see if that helps.
> > I have applied this patch. I think libfsimage is somehow not being
> > affected with this. Yet, linking /usr/lib64/fs -> /usr/lib/fs solves
> > the problem.
> 
> libfsimage is going to blindly look in /usr/lib64 on non-Itanium
> 64-bit Linux platforms. See tools/libfsimage/common/fsimage_plugin.c:134

Oh bloody hell, I hadn't spotted that.

We should definitely be setting FSIMAGE_FSDIR to something sane based on
LIBDIR and not letting all sorts of weird heuristics kick in.

Ian.

> 
> #if defined(FSIMAGE_FSDIR)
>         if (fsdir == NULL)
>                 fsdir = FSIMAGE_FSDIR;
> #elif defined(__sun__)
>         if (fsdir == NULL)
>                 fsdir = "/usr/lib/fs";
> 
>         if (sizeof(void *) == 8)
>                 isadir = "64/";
> #elif defined(__ia64__)
>         if (fsdir == NULL)
>                 fsdir = "/usr/lib/fs";
> #else
>         if (fsdir == NULL) {
>                 if (sizeof(void *) == 8)
>                         fsdir = "/usr/lib64/fs";
>                 else
>                         fsdir = "/usr/lib/fs";
>         }
> #endif
> 
> Matt

Re: fsimage - no such file

[cyberhawk001]

On 6/19/2012 12:36 PM, Ian Campbell wrote:
> On Tue, 2012-06-19 at 17:10 +0100, Matt Wilson wrote:
>> On Tue, Jun 19, 2012 at 04:19:27AM -0700, Mohammad Hedayati wrote:
>>>> # Node ID 6eabee6807b48c72bcc35a28170f0729500def85
>>>> # Parent  80c0677f0f8370a4542aab81ab93380b0dab25db
>>>> imported patch debian-lib-dir.patch
>>>>
>>>> diff -r 80c0677f0f83 -r 6eabee6807b4 config/StdGNU.mk
>>>> --- a/config/StdGNU.mk  Wed Jun 13 09:27:53 2012 +0100
>>>> +++ b/config/StdGNU.mk  Wed Jun 13 09:27:53 2012 +0100
>>>> @@ -34,7 +34,7 @@ BINDIR = $(PREFIX)/bin
>>>>   INCLUDEDIR = $(PREFIX)/include
>>>>   LIBLEAFDIR = lib
>>>>   LIBLEAFDIR_x86_32 = lib
>>>> -LIBLEAFDIR_x86_64 ?= lib64
>>>> +LIBLEAFDIR_x86_64 ?= lib
>>>>   LIBDIR = $(PREFIX)/$(LIBLEAFDIR)
>>>>   LIBDIR_x86_32 = $(PREFIX)/$(LIBLEAFDIR_x86_32)
>>>>   LIBDIR_x86_64 = $(PREFIX)/$(LIBLEAFDIR_x86_64)
>>>>
>>>> Can you try a fresh build with this applied and see if that helps.
>>> I have applied this patch. I think libfsimage is somehow not being
>>> affected with this. Yet, linking /usr/lib64/fs ->  /usr/lib/fs solves
>>> the problem.
>> libfsimage is going to blindly look in /usr/lib64 on non-Itanium
>> 64-bit Linux platforms. See tools/libfsimage/common/fsimage_plugin.c:134
> Oh bloody hell, I hadn't spotted that.
>
> We should definitely be setting FSIMAGE_FSDIR to something sane based on
> LIBDIR and not letting all sorts of weird heuristics kick in.
>
> Ian.
>
>> #if defined(FSIMAGE_FSDIR)
>>          if (fsdir == NULL)
>>                  fsdir = FSIMAGE_FSDIR;
>> #elif defined(__sun__)
>>          if (fsdir == NULL)
>>                  fsdir = "/usr/lib/fs";
>>
>>          if (sizeof(void *) == 8)
>>                  isadir = "64/";
>> #elif defined(__ia64__)
>>          if (fsdir == NULL)
>>                  fsdir = "/usr/lib/fs";
>> #else
>>          if (fsdir == NULL) {
>>                  if (sizeof(void *) == 8)
>>                          fsdir = "/usr/lib64/fs";
>>                  else
>>                          fsdir = "/usr/lib/fs";
>>          }
>> #endif
>>
>> Matt

SOOO, just wondering, in the above patch, should these:

-LIBLEAFDIR_x86_64 ?= lib64
+LIBLEAFDIR_x86_64 ?= lib


be also changed if installing the latest Xen 4.2-unstable on like Debian 
Wheezy? OR is this something that is not important that important?



Just curious since even in the latest Xen 4.2-unstable rev-25494, it 
still has the

LIBLEAFDIR_x86_32 = lib
LIBLEAFDIR_x86_64 ?= lib64


in the StdGNU.mk file.

Thanks

Re: fsimage - no such file

[Matt Wilson]

On Sat, Jun 23, 2012 at 01:22:29PM -0700, cyberhawk001@gmail.com wrote:
> >> On Tue, Jun 19, 2012 at 04:19:27AM -0700, Mohammad Hedayati wrote:
> >>>> # Node ID 6eabee6807b48c72bcc35a28170f0729500def85
> >>>> # Parent  80c0677f0f8370a4542aab81ab93380b0dab25db
> >>>> imported patch debian-lib-dir.patch
> >>>>
> >>>> diff -r 80c0677f0f83 -r 6eabee6807b4 config/StdGNU.mk
> >>>> --- a/config/StdGNU.mk  Wed Jun 13 09:27:53 2012 +0100
> >>>> +++ b/config/StdGNU.mk  Wed Jun 13 09:27:53 2012 +0100
> >>>> @@ -34,7 +34,7 @@ BINDIR = $(PREFIX)/bin
> >>>>   INCLUDEDIR = $(PREFIX)/include
> >>>>   LIBLEAFDIR = lib
> >>>>   LIBLEAFDIR_x86_32 = lib
> >>>> -LIBLEAFDIR_x86_64 ?= lib64
> >>>> +LIBLEAFDIR_x86_64 ?= lib
> >>>>   LIBDIR = $(PREFIX)/$(LIBLEAFDIR)
> >>>>   LIBDIR_x86_32 = $(PREFIX)/$(LIBLEAFDIR_x86_32)
> >>>>   LIBDIR_x86_64 = $(PREFIX)/$(LIBLEAFDIR_x86_64)
> >>>>
> 
> SOOO, just wondering, in the above patch, should these:
> 
> -LIBLEAFDIR_x86_64 ?= lib64
> +LIBLEAFDIR_x86_64 ?= lib
> 
> 
> be also changed if installing the latest Xen 4.2-unstable on like Debian 
> Wheezy? OR is this something that is not important that important?

Ultimately the LIBLEAFDIR bits should go away entirely. The person
building Xen should specify where libraries live at ./configure time
with, for example, --libdir=/usr/lib64 (for 64-bit OSs like Fedora,
RHEL, OpenSUSE and SLES) or --libdir=/usr/lib/x86_64-linux-gnu (for
64-bit systems like Ubuntu 12.04). More specifically, on Debian based
multiarch platforms, you'd use:
 ./configure --libdir=/usr/lib/$(dpkg-architecture -qDEB_HOST_MULTIARCH)

> Just curious since even in the latest Xen 4.2-unstable rev-25494, it 
> still has the
> 
> LIBLEAFDIR_x86_32 = lib
> LIBLEAFDIR_x86_64 ?= lib64
> 
> 
> in the StdGNU.mk file.

Right, nothing has been committed while I've been working through the
feedback on the patch. I'll post v3 in just a moment which hopefully
has addressed all concerns.

Matt

Re: fsimage - no such file

[Matt Wilson]

On Tue, Jun 19, 2012 at 03:57:33AM -0700, Ian Campbell wrote:
> On Tue, 2012-06-19 at 09:35 +0100, Thanos Makatos wrote:
> > I have the same problem on xen-unstable 54c8c9eaee92+. I think
> > libfsimage comes with xen (in my repo it's under ./tools/libfsimage).
> > I've investigated a bit and found that in my occasion libfsimage tries
> > to access directory /usr/lib64/fs which doesn't exist,
> > whereas /usr/lib/fs does. I linked /usr/lib64/fs to /usr/lib/fs but
> > then I got other problems, don't remember what. I don't know if this
> > has to do with the fact that my dom0 is 64bit (Debian unstable), or
> > that the domU is 64bit (Debian unstable again), or both.
> 
> Aha I'd been puzzling over why people had been seeing this but I'm not! 
> 
> This very likely is to do with Debian and Ubuntu's transition to
> "multiarch" which has necessitated removing the old compatibility
> symlink /usr/lib64 -> /usr/lib.
> 
> lib64 is a RPMism which doesn't really apply to Debian and derivatives
> and therefore you need to tweak your Xen config before building. I use
> the following local configuration patch (I should probably do the
> equivalent in .config):

lib64 isn't an RPMism, its a Filesystem Hiarchary Standardism [1].

Pretty much every distribution but Debian (and derivatives) adopted
lib64 for all platforms where running 32-bit binaries was common place
(read: all platforms but Alpha and Itanium).

Matt Taggart proposed multiarch for Debian in 2004 (where Red Hat and
SUSE already had support for running a mix of 32-bit and 64-bit
applications on one system), which is captured in the Debian wiki [2].
Ubuntu added support for co-installing 32-bit and 64-bit software in
11.04 [3].

I'm a bit surprised that Debian and Ubuntu can't continue to have a
symlink for lib64 that points at lib/x86_64-linux-gnu. No matter since
I suppose the correct fix is to install the plugins in /usr/lib/$target/fs 
and define FSIMAGE_FSDIR to point there when building libfsimage on
systems that implement multiarch.

Matt

[1] http://www.pathname.com/fhs/
[2] http://wiki.debian.org/Multiarch/TheCaseForMultiarch
[3] https://wiki.ubuntu.com/MultiarchSpec

Re: fsimage - no such file

[Ian Campbell]

On Tue, 2012-06-19 at 16:58 +0100, Matt Wilson wrote:
> On Tue, Jun 19, 2012 at 03:57:33AM -0700, Ian Campbell wrote:
> > On Tue, 2012-06-19 at 09:35 +0100, Thanos Makatos wrote:
> > lib64 is a RPMism which doesn't really apply to Debian and derivatives
> > and therefore you need to tweak your Xen config before building. I use
> > the following local configuration patch (I should probably do the
> > equivalent in .config):
> 
> lib64 isn't an RPMism, its a Filesystem Hiarchary Standardism [1].

Yes, right, I keep making that mistake.

[...]

> I'm a bit surprised that Debian and Ubuntu can't continue to have a
> symlink for lib64 that points at lib/x86_64-linux-gnu.

I'm not sure either.

> No matter since
> I suppose the correct fix is to install the plugins in /usr/lib/$target/fs 
> and define FSIMAGE_FSDIR to point there when building libfsimage on
> systems that implement multiarch.

I think the current use of $(LIBDIR)/fs/... is correct -- assuming you
actually set LIBDIR to the correct thing (which would
be /usr/lib/$target on a multiarch system).

In the meantime I think defaulting to autodetecting /usr/lib
vs. /usr/lib64 would make things continue to work. multiarch is more of
a packaging thing and the config snippets allow maintainers to do the
right thing).

Ian.

Re: fsimage - no such file

[Ian Campbell]

On Tue, 2012-06-19 at 17:13 +0100, Ian Campbell wrote:
> In the meantime I think defaulting to autodetecting /usr/lib
> vs. /usr/lib64 would make things continue to work. multiarch is more
> of a packaging thing and the config snippets allow maintainers to do
> the right thing). 

Or maybe we should ditch LIBLEAFDIR_x86_32 and LIBLEAFDIR_x86_64 etc and
just have LIBDIR which defaults to /usr/lib but is settable
via ./configure --libdir=...?

This seems to be the norm for configure using build systems so it
shouldn't be all that surprising.

Ian.