xend "sysctl operation failed"

xend "sysctl operation failed"

[Aron Griffis]

I'm seeing this failure trying to start xend:

==> xend-debug.log <==
Xend started at Wed Nov  7 09:30:01 2007.
sysctl operation failed -- need to rebuild the user-space tool set?
Exception starting xend: (13, 'Permission denied')

==> xend.log <==
[2007-11-07 09:30:01 3723] INFO (__init__:1072) Xend Daemon started
[2007-11-07 09:30:01 3723] INFO (__init__:1072) Xend changeset: Fri Nov 02 16:38:11 2007 +0000 16314:650cadd1b283.
[2007-11-07 09:30:01 3723] INFO (__init__:1072) Xend version: Unknown.
[2007-11-07 09:30:01 3723] ERROR (__init__:1072) Exception starting xend ((13, 'Permission denied'))
Traceback (most recent call last):
  File "//usr/lib/python/xen/xend/server/SrvDaemon.py", line 345, in run
    servers = SrvServer.create()
  File "//usr/lib/python/xen/xend/server/SrvServer.py", line 251, in create
    root.putChild('xend', SrvRoot())
  File "//usr/lib/python/xen/xend/server/SrvRoot.py", line 40, in __init__
    self.get(name)
  File "//usr/lib/python/xen/web/SrvDir.py", line 82, in get
    val = val.getobj()
  File "//usr/lib/python/xen/web/SrvDir.py", line 52, in getobj
    self.obj = klassobj()
  File "//usr/lib/python/xen/xend/server/SrvNode.py", line 30, in __init__
    self.xn = XendNode.instance()
  File "//usr/lib/python/xen/xend/XendNode.py", line 703, in instance
    inst = XendNode()
  File "//usr/lib/python/xen/xend/XendNode.py", line 83, in __init__
    self.other_config["xen_pagesize"] = self.xeninfo_dict()["xen_pagesize"]
  File "//usr/lib/python/xen/xend/XendNode.py", line 667, in xeninfo_dict
    return dict(self.xeninfo())
  File "//usr/lib/python/xen/xend/XendNode.py", line 617, in xeninfo
    info['xen_scheduler'] = self.xenschedinfo()
  File "//usr/lib/python/xen/xend/XendNode.py", line 607, in xenschedinfo
    sched_id = self.xc.sched_id_get()
Error: (13, 'Permission denied')

Anybody know what might be the problem here?  I'm sure the build is
good and matches the running hypervisor and kernel, which are csets
ia64/linux-2.6.18-xen.hg-210 and ia64/xen-unstable.hg-16314

Thanks,
Aron

Re: [Xen-devel] xend "sysctl operation failed"

[Keir Fraser]

On 7/11/07 15:55, "Aron Griffis" <aron@hp.com> wrote:

> Anybody know what might be the problem here?  I'm sure the build is
> good and matches the running hypervisor and kernel, which are csets
> ia64/linux-2.6.18-xen.hg-210 and ia64/xen-unstable.hg-16314

You need to reinstall the tools.

 -- Keir

Re: xend "sysctl operation failed"

[Aron Griffis]

Keir Fraser wrote:  [Wed Nov 07 2007, 10:57:09AM EST]
> On 7/11/07 15:55, "Aron Griffis" <aron@hp.com> wrote:
> 
> > Anybody know what might be the problem here?  I'm sure the build is
> > good and matches the running hypervisor and kernel, which are csets
> > ia64/linux-2.6.18-xen.hg-210 and ia64/xen-unstable.hg-16314
> 
> You need to reinstall the tools.

I have, multiple times, including "make uninstall" first.

What are you anticipating that a tools re-install will fix?

Thanks,
Aron

Re: [Xen-devel] xend "sysctl operation failed"

[Keir Fraser]

On 7/11/07 16:02, "Aron Griffis" <aron@hp.com> wrote:

>> You need to reinstall the tools.
> 
> I have, multiple times, including "make uninstall" first.
> 
> What are you anticipating that a tools re-install will fix?

The sysctl is almost certainly failing because of a interface version
mismatch. Could you be building the tools and Xen against different versions
of the public sysctl.h header file?

Try instrumenting xen/common/sysctl.c in the hypervisor and see if you are
indeed failing on the obvious version check at the top of the do_sysctl()
function.

 -- Keir

Re: [Xen-devel] xend "sysctl operation failed"

[Aron Griffis]

Keir Fraser wrote:  [Wed Nov 07 2007, 11:31:35AM EST]
> On 7/11/07 16:02, "Aron Griffis" <aron@hp.com> wrote:
> 
> >> You need to reinstall the tools.
> > 
> > I have, multiple times, including "make uninstall" first.
> > 
> > What are you anticipating that a tools re-install will fix?
> 
> The sysctl is almost certainly failing because of a interface version
> mismatch. Could you be building the tools and Xen against different versions
> of the public sysctl.h header file?

Yeah, I think that's it.  I'm building on one system, then tar/untar
to the test system to do the final "make install-tools"

Maybe it's rebuilding something it doesn't need to?  I'd probably
rather fix that than worry about keeping the userlands in perfect
sync, unless this is going to crop up at runtime regardless.

Thanks,
Aron

Re: [Xen-devel] xend "sysctl operation failed"

[Keir Fraser]

On 7/11/07 17:26, "Aron Griffis" <aron@hp.com> wrote:

>> The sysctl is almost certainly failing because of a interface version
>> mismatch. Could you be building the tools and Xen against different versions
>> of the public sysctl.h header file?
> 
> Yeah, I think that's it.  I'm building on one system, then tar/untar
> to the test system to do the final "make install-tools"
> 
> Maybe it's rebuilding something it doesn't need to?  I'd probably
> rather fix that than worry about keeping the userlands in perfect
> sync, unless this is going to crop up at runtime regardless.

Well, now you know the likely problem it doesn't sound like it will be very
hard to track down!

 -- Keir

Re: xend "sysctl operation failed"

[Aron Griffis]

Keir Fraser wrote:  [Wed Nov 07 2007, 12:30:29PM EST]
> Well, now you know the likely problem it doesn't sound like it will
> be very hard to track down!

Actually, it wasn't the problem at all...  My script was trying to
copy the kernel from dist/install/boot/vmlinuz but recently it's
changed to dist/install/boot/vmlinuz-2.6.18.8-xen.  Unfortunately
I wrote that part of the script to fail silently, so I wasn't aware
that my recent testing has been mismatched.

Thanks for the clues, they helped me finally track this down.

Aron