xen-ringwatch issues

xen-ringwatch issues

[moftah moftah]

[-- Attachment #1.1: Type: text/plain, Size: 2182 bytes --]

Hi All,
We are using Xenserver 6.2 with latest patches installed and we just
enabled email sending on the pool (the pool has been running for long time
with no major issues)

now 3 hosts out of 4 hosts in the pool are flooding us with xen-ringwatch
exceptions
like this
Traceback (most recent call last):
  File "/usr/sbin/xen-ringwatch", line 593, in ?
    watches.update()
  File "/usr/sbin/xen-ringwatch", line 414, in update
    entry = RingWatch.new(ring)
  File "/usr/sbin/xen-ringwatch", line 357, in new
    state = ring.read()
  File "/usr/sbin/xen-ringwatch", line 136, in read
    state = RingState.from_sysfs(self.path())
  File "/usr/sbin/xen-ringwatch", line 223, in from_sysfs
    req = cls.Req.from_sysfs(_req, size=nr_ents)
  File "/usr/sbin/xen-ringwatch", line 240, in from_sysfs
    raise Exception, "Malformed %s input: %s" % \
NameError: global name 's' is not defined

so after searching around we changed the file xen-ringwatch in order to see
the real issue
the changes are

--- /usr/sbin/xen-ringwatch     2013-07-22 13:52:19.000000000 +0200
+++ /usr/sbin/xen-ringwatch     2013-07-22 13:52:30.000000000 +0200
@@ -238,7 +238,7 @@
             match = cls._pattern.search(line)
             if not match:
                 raise Exception, "Malformed %s input: %s" % \
-                    (cls.__name__, repr(s))
+                    (cls.__name__, repr(line))

             i = iter(match.groups())
             for k in i:


now the issue we see is like this

Exception: Malformed Req input: 'req prod 3412900880 cons -882066416
event 3412900881'
Exception: Malformed Req input: 'req prod 3412902034 cons -882065262
event 3412902035''
and it is flooding us with this for every host every 2 minutes
and in each email the numbers changes


I have been trying to read the original article where this script was
introduced here
http://comments.gmane.org/gmane.comp.emulators.xen.devel/106601

but i am not sure whats wrong with the script
and why we have negative number here

if any one can help in fixing the script or at least inform us if it
is still in use as it seems it was done for specific version of the
kernel

Any help will be appreciated

Thanks

[-- Attachment #1.2: Type: text/html, Size: 5845 bytes --]

[-- Attachment #2: Type: text/plain, Size: 126 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: xen-ringwatch issues

[Ian Campbell]

On Thu, 2015-01-08 at 14:07 -0500, moftah moftah wrote:
> Hi All,
> We are using Xenserver 6.2

FYI xenserver is developed as a separate project over at
www.xenserver.org, so in general you should be reporting
issue/requesting help over on their forums and lists etc.

However, since xen-ringwatch is shipped by upstream Xen we can at least
try and help with that bit here.

[...]
> so after searching around we changed the file xen-ringwatch in order
> to see the real issue the changes are

Those look sensible, please could you send with a changelog message and
a Signed-off-by as described in
http://wiki.xen.org/wiki/Submitting_Xen_Patches . The S-o-b in
particular is required in order to be able to accept a code
contribution.

> --- /usr/sbin/xen-ringwatch     2013-07-22 13:52:19.000000000 +0200
> +++ /usr/sbin/xen-ringwatch     2013-07-22 13:52:30.000000000 +0200
> @@ -238,7 +238,7 @@
>              match = cls._pattern.search(line)
>              if not match:
>                  raise Exception, "Malformed %s input: %s" % \
> -                    (cls.__name__, repr(s))
> +                    (cls.__name__, repr(line))
>  
>              i = iter(match.groups())
>              for k in i:
> 
> now the issue we see is like this
> Exception: Malformed Req input: 'req prod 3412900880 cons -882066416 event 3412900881'

I bet the negative number is confusing things (ah, which you also said
further down). 

Really the kernel ought to be printing these as unsigned (for which
you'll need to speak to the xenserver.org folks, I think)

But the python code could also deal with them more gracefully when it
sees them. You'd need to start by allowing the regex used for the match
to accept an optional leading "-" on the numbers.

You probably also want to cast the result to the unsigned value during
the subsequent parsing, my Python-fu isn't sufficient to know off hand
how one would do that.

Ian.

Re: xen-ringwatch issues

[moftah moftah]

[-- Attachment #1.1: Type: text/plain, Size: 5522 bytes --]

Hi Ian,

OK, I have followed your suggestions and created a new patch

this new patch does fix the issue fully for the negative numbers

I didnt run the patch on 64bit OS since xenserver has 32bit Dom0

but i think it is safe to add

here is the patch in case you or anyone want to have look at and if you
have comments on before i send it
*** xen-ringwatch.old   2015-01-09 17:05:55.000000000 -0500
--- xen-ringwatch.new   2015-01-09 17:06:14.000000000 -0500
***************
*** 238,244 ****
              match = cls._pattern.search(line)
              if not match:
                  raise Exception, "Malformed %s input: %s" % \
!                     (cls.__name__, repr(s))

              i = iter(match.groups())
              for k in i:
--- 238,244 ----
              match = cls._pattern.search(line)
              if not match:
                  raise Exception, "Malformed %s input: %s" % \
!                     (cls.__name__, repr(line))

              i = iter(match.groups())
              for k in i:
***************
*** 251,263 ****

      class Req(Queue):

!         _pattern = Pattern("req (prod) (\d+) (cons) (\d+) (event) (\d+)")

          def __init__(self, prod, cons, event, **d):
              RingState.Queue.__init__(self, **d)
              self.prod  = int(prod)
              self.cons  = int(cons)
              self.event = int(event)

          def __repr__(self):
              return "%s(prod=%d, cons=%d, event=%d)" % \
--- 251,270 ----

      class Req(Queue):

!         _pattern = Pattern("req (prod) (\d+) (cons) ([-]*\d+) (event)
(\d+)")

          def __init__(self, prod, cons, event, **d):
              RingState.Queue.__init__(self, **d)
              self.prod  = int(prod)
              self.cons  = int(cons)
              self.event = int(event)
+
+             if self.cons < 0:
+                  self.cons = 4294967296 + self.cons
+             if self.prod < 0:
+                  self.prod = 4294967296 + self.prod
+             if self.event < 0:
+                  self.event = 4294967296 + self.event

          def __repr__(self):
              return "%s(prod=%d, cons=%d, event=%d)" % \
***************
*** 274,287 ****

      class Rsp(Queue):

!         _pattern = Pattern("rsp (prod) (\d+) (pvt) (\d+) (event) (\d+)")

          def __init__(self, prod, pvt, event, **d):
              RingState.Queue.__init__(self, **d)
              self.prod  = int(prod)
              self.pvt   = int(pvt)
              self.event = int(event)
!
          def __repr__(self):
              return "%s(prod=%d, pvt=%d, event=%d)" % \
                  (type(self).__name__, self.prod, self.pvt, self.event)
--- 281,301 ----

      class Rsp(Queue):

!         _pattern = Pattern("rsp (prod) (\d+) (pvt) ([-]*\d+) (event)
(\d+)")

          def __init__(self, prod, pvt, event, **d):
              RingState.Queue.__init__(self, **d)
              self.prod  = int(prod)
              self.pvt   = int(pvt)
              self.event = int(event)
!
!             if self.pvt < 0:
!                  self.pvt = 4294967296 + self.pvt
!             if self.prod < 0:
!                  self.prod = 4294967296 + self.prod
!             if self.event < 0:
!                  self.event = 4294967296 + self.event
!
          def __repr__(self):
              return "%s(prod=%d, pvt=%d, event=%d)" % \
                  (type(self).__name__, self.prod, self.pvt, self.event)


Thanks


On Fri, Jan 9, 2015 at 5:10 AM, Ian Campbell <Ian.Campbell@citrix.com>
wrote:

> On Thu, 2015-01-08 at 14:07 -0500, moftah moftah wrote:
> > Hi All,
> > We are using Xenserver 6.2
>
> FYI xenserver is developed as a separate project over at
> www.xenserver.org, so in general you should be reporting
> issue/requesting help over on their forums and lists etc.
>
> However, since xen-ringwatch is shipped by upstream Xen we can at least
> try and help with that bit here.
>
> [...]
> > so after searching around we changed the file xen-ringwatch in order
> > to see the real issue the changes are
>
> Those look sensible, please could you send with a changelog message and
> a Signed-off-by as described in
> http://wiki.xen.org/wiki/Submitting_Xen_Patches . The S-o-b in
> particular is required in order to be able to accept a code
> contribution.
>
> > --- /usr/sbin/xen-ringwatch     2013-07-22 13:52:19.000000000 +0200
> > +++ /usr/sbin/xen-ringwatch     2013-07-22 13:52:30.000000000 +0200
> > @@ -238,7 +238,7 @@
> >              match = cls._pattern.search(line)
> >              if not match:
> >                  raise Exception, "Malformed %s input: %s" % \
> > -                    (cls.__name__, repr(s))
> > +                    (cls.__name__, repr(line))
> >
> >              i = iter(match.groups())
> >              for k in i:
> >
> > now the issue we see is like this
> > Exception: Malformed Req input: 'req prod 3412900880 cons -882066416
> event 3412900881'
>
> I bet the negative number is confusing things (ah, which you also said
> further down).
>
> Really the kernel ought to be printing these as unsigned (for which
> you'll need to speak to the xenserver.org folks, I think)
>
> But the python code could also deal with them more gracefully when it
> sees them. You'd need to start by allowing the regex used for the match
> to accept an optional leading "-" on the numbers.
>
> You probably also want to cast the result to the unsigned value during
> the subsequent parsing, my Python-fu isn't sufficient to know off hand
> how one would do that.
>
> Ian.
>
>
>

[-- Attachment #1.2: Type: text/html, Size: 8166 bytes --]

[-- Attachment #2: Type: text/plain, Size: 126 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: xen-ringwatch issues

[Ian Campbell]

On Fri, 2015-01-09 at 17:00 -0500, moftah moftah wrote:
> Hi Ian,
> 
> 
> OK, I have followed your suggestions and created a new patch

Thanks, please submit it in the form describe in the wiki page
http://wiki.xen.org/wiki/Submitting_Xen_Patches which I referenced
before.

In particular, a Signed-off-by is a strict requirement for licensing
reasons, to indicate that you have read the DCO (which is at
http://wiki.xen.org/wiki/Submitting_Xen_Patches#Signing_off_a_patch ).

Please also provide a suitable changelog entry and use a unified diff
(diff -u).

I think rather than "[-]*" the right addition would be -? (i.e. a single
optional negative sign), unless Python's re syntax is lots different to
what I expect.

> this new patch does fix the issue fully for the negative numbers
>
> I didnt run the patch on 64bit OS since xenserver has 32bit Dom0

These fields are unsigned int, .i.e. always 32-bits. So I think your "if
self.conf < 0: self.cons = 4294967296 + self.cons" etc will work. The
Internet also suggests "ctypes.c_uint32(i).value", dunno if that is
better. In any case writing 4294967296 as 2**32 would be slightly
clearer.

Ian.