Xen 4.4, xenstored and WATCH requests

Xen 4.4, xenstored and WATCH requests

[Sergei Lebedev]

Hello list,

I’m working on a Python client library for XenStore [1]. The library implements two ways to access XenStore: via Unix socket and via /dev.  The /dev interface turned out to be a bit problematic, because it ignores the req_id field for WATCH requests.

The spec [2] requires all responses (except WATCH_EVENT) to copy req_id from request. So I wonder if it’s possible that the behaviour I observe is in fact a bug in xenstored?

Below is a Python script demonstrating the issue:

    import os
    import struct

    WATCH = 4
    WRITE = 11

    def send_packet(fd, op, rq_id, tx_id, payload):
        data = struct.pack(b"IIII", op, rq_id, tx_id, len(payload)) + payload
        os.write(fd, data)


    def print_reply(fd):
        op, rq_id, tx_id, size = struct.unpack(b"IIII", os.read(fd, 16))
        payload = os.read(fd, size) if size else b""
        print(op, rq_id, tx_id, payload)


    fd = os.open("/dev/xen/xenbus", os.O_RDWR)
    try:
        send_packet(fd, WRITE, 24, 0, b"/foo\x00bar\x00")
        print_reply(fd)  # ACK for WRITE with rq_id = 24.
        send_packet(fd, WATCH, 42, 0, b"/foo\x00token\x00")
        print_reply(fd)  # Spurious (?) WATCH_EVENT.
        print_reply(fd)  # ACK for WATCH with rq_id = 0.
    finally:
        os.close(fd)

Example output:

    (11, 24, 0, 'OK\x00')
    (15, 4294967295, 892960384, '/foo\x00token\x00')
    (4, 0, 0, 'OK\x00')

I’m running Xen 4.4 on Ubuntu 14.04 inside VirtualBox.

$ uname -a
Linux xen-devel 3.19.0-25-generic #26~14.04.1-Ubuntu SMP Fri Jul 24 21:16:20 UTC 2015 x86_64 x86_64 x86_64 GNU/Linux

Regards,
Sergei

[1]: http://github.com/selectel/pyxs
[2]: http://xenbits.xen.org/docs/4.4-testing/misc/xenstore.txt


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: Xen 4.4, xenstored and WATCH requests

[Wei Liu]

Hello

On Thu, Feb 25, 2016 at 03:54:14PM +0300, Sergei Lebedev wrote:
> Hello list,
> 
> I’m working on a Python client library for XenStore [1]. The library
> implements two ways to access XenStore: via Unix socket and via /dev.
> The /dev interface turned out to be a bit problematic, because it
> ignores the req_id field for WATCH requests.
> 

So the unix socket interface worked fine for you?

> The spec [2] requires all responses (except WATCH_EVENT) to copy
> req_id from request. So I wonder if it’s possible that the behaviour I
> observe is in fact a bug in xenstored?
> 
> Below is a Python script demonstrating the issue:
> 
>     import os
>     import struct
> 
>     WATCH = 4
>     WRITE = 11
> 
>     def send_packet(fd, op, rq_id, tx_id, payload):
>         data = struct.pack(b"IIII", op, rq_id, tx_id, len(payload)) + payload
>         os.write(fd, data)
> 
> 
>     def print_reply(fd):
>         op, rq_id, tx_id, size = struct.unpack(b"IIII", os.read(fd, 16))
>         payload = os.read(fd, size) if size else b""
>         print(op, rq_id, tx_id, payload)
> 
> 
>     fd = os.open("/dev/xen/xenbus", os.O_RDWR)
>     try:
>         send_packet(fd, WRITE, 24, 0, b"/foo\x00bar\x00")
>         print_reply(fd)  # ACK for WRITE with rq_id = 24.
>         send_packet(fd, WATCH, 42, 0, b"/foo\x00token\x00")
>         print_reply(fd)  # Spurious (?) WATCH_EVENT.
>         print_reply(fd)  # ACK for WATCH with rq_id = 0.
>     finally:
>         os.close(fd)
> 

> Example output:
> 
>     (11, 24, 0, 'OK\x00')
>     (15, 4294967295, 892960384, '/foo\x00token\x00')
>     (4, 0, 0, 'OK\x00')
> 
> I’m running Xen 4.4 on Ubuntu 14.04 inside VirtualBox.
> 
> $ uname -a
> Linux xen-devel 3.19.0-25-generic #26~14.04.1-Ubuntu SMP Fri Jul 24 21:16:20 UTC 2015 x86_64 x86_64 x86_64 GNU/Linux
> 

FWIW I tried you program in my own testbox running xen-unstable with
Debian's stock 3.16 kernel, the result is the same.

And in xen.git

  $ git diff origin/stable-4.4..origin/staging -- docs/misc/xenstore.txt

shows nothing.

So this is not a problem in 4.4 only.

Wei.

> Regards,
> Sergei
> 
> [1]: http://github.com/selectel/pyxs
> [2]: http://xenbits.xen.org/docs/4.4-testing/misc/xenstore.txt
> 
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xen.org
> http://lists.xen.org/xen-devel

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: Xen 4.4, xenstored and WATCH requests

[Wei Liu]

On Fri, Feb 26, 2016 at 11:26:50AM +0000, Wei Liu wrote:
> Hello
> 
> On Thu, Feb 25, 2016 at 03:54:14PM +0300, Sergei Lebedev wrote:
> > Hello list,
> > 
> > I’m working on a Python client library for XenStore [1]. The library
> > implements two ways to access XenStore: via Unix socket and via /dev.
> > The /dev interface turned out to be a bit problematic, because it
> > ignores the req_id field for WATCH requests.
> > 
> 
> So the unix socket interface worked fine for you?
> 

BTW can you paste in the result of

  $ ps aux | grep xenstored

here?

I need to know which variant of xenstored you're running.

Wei.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: Xen 4.4, xenstored and WATCH requests

[Sergei Lebedev]

[-- Attachment #1.1: Type: text/plain, Size: 3582 bytes --]

Hi Wei,

Yes, the Unix socket interface works okay. Here’s a relevant part of the script changed to use the socket:

    import socket
    sock = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM)
    sock.connect("/var/run/xenstored/socket")
    fd = os.dup(sock.fileno())
    try:
        send_packet(fd, WRITE, 24, 0, b"/foo\x00bar\x00")
        print_reply(fd)  # Responds with rq_id = 24.
        send_packet(fd, WATCH, 42, 0, b"/foo\x00token\x00")
        print_reply(fd)  # ACK for WATCH with rq_id = 42.
    finally:
        os.close(fd)

> BTW can you paste in the result of
> 
>  $ ps aux | grep xenstored
> 
> here?


$ ps aux | grep xenstored
root       807  0.0  0.2  10932  2132 ?        S    00:34   0:01 /usr/lib/xen-4.4/bin/xenstored --p
id-file=/var/run/xenstore.pid

Sergei


> On 26 Feb 2016, at 14:26, Wei Liu <wei.liu2@citrix.com> wrote:
> 
> Hello
> 
> On Thu, Feb 25, 2016 at 03:54:14PM +0300, Sergei Lebedev wrote:
>> Hello list,
>> 
>> I’m working on a Python client library for XenStore [1]. The library
>> implements two ways to access XenStore: via Unix socket and via /dev.
>> The /dev interface turned out to be a bit problematic, because it
>> ignores the req_id field for WATCH requests.
>> 
> 
> So the unix socket interface worked fine for you?
> 
>> The spec [2] requires all responses (except WATCH_EVENT) to copy
>> req_id from request. So I wonder if it’s possible that the behaviour I
>> observe is in fact a bug in xenstored?
>> 
>> Below is a Python script demonstrating the issue:
>> 
>>    import os
>>    import struct
>> 
>>    WATCH = 4
>>    WRITE = 11
>> 
>>    def send_packet(fd, op, rq_id, tx_id, payload):
>>        data = struct.pack(b"IIII", op, rq_id, tx_id, len(payload)) + payload
>>        os.write(fd, data)
>> 
>> 
>>    def print_reply(fd):
>>        op, rq_id, tx_id, size = struct.unpack(b"IIII", os.read(fd, 16))
>>        payload = os.read(fd, size) if size else b""
>>        print(op, rq_id, tx_id, payload)
>> 
>> 
>>    fd = os.open("/dev/xen/xenbus", os.O_RDWR)
>>    try:
>>        send_packet(fd, WRITE, 24, 0, b"/foo\x00bar\x00")
>>        print_reply(fd)  # ACK for WRITE with rq_id = 24.
>>        send_packet(fd, WATCH, 42, 0, b"/foo\x00token\x00")
>>        print_reply(fd)  # Spurious (?) WATCH_EVENT.
>>        print_reply(fd)  # ACK for WATCH with rq_id = 0.
>>    finally:
>>        os.close(fd)
>> 
> 
>> Example output:
>> 
>>    (11, 24, 0, 'OK\x00')
>>    (15, 4294967295, 892960384, '/foo\x00token\x00')
>>    (4, 0, 0, 'OK\x00')
>> 
>> I’m running Xen 4.4 on Ubuntu 14.04 inside VirtualBox.
>> 
>> $ uname -a
>> Linux xen-devel 3.19.0-25-generic #26~14.04.1-Ubuntu SMP Fri Jul 24 21:16:20 UTC 2015 x86_64 x86_64 x86_64 GNU/Linux
>> 
> 
> FWIW I tried you program in my own testbox running xen-unstable with
> Debian's stock 3.16 kernel, the result is the same.
> 
> And in xen.git
> 
>  $ git diff origin/stable-4.4..origin/staging -- docs/misc/xenstore.txt
> 
> shows nothing.
> 
> So this is not a problem in 4.4 only.
> 
> Wei.
> 
>> Regards,
>> Sergei
>> 
>> [1]: http://github.com/selectel/pyxs <http://github.com/selectel/pyxs>
>> [2]: http://xenbits.xen.org/docs/4.4-testing/misc/xenstore.txt <http://xenbits.xen.org/docs/4.4-testing/misc/xenstore.txt>
>> 
>> 
>> _______________________________________________
>> Xen-devel mailing list
>> Xen-devel@lists.xen.org <mailto:Xen-devel@lists.xen.org>
>> http://lists.xen.org/xen-devel <http://lists.xen.org/xen-devel>

[-- Attachment #1.2: Type: text/html, Size: 18023 bytes --]

[-- Attachment #2: Type: text/plain, Size: 126 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: Xen 4.4, xenstored and WATCH requests

[Wei Liu]

(CC'ing Linux maintainers)

OK. Thanks for all the information.

This seems to be a bug in Linux xenbus driver.

Relevant code snippet in
drivers/xen/xenbus/xenbus_dev_frontend.c:xenbus_write_watch()


  /* Success.  Synthesize a reply to say all is OK. */
  {
          struct {
                  struct xsd_sockmsg hdr;
                  char body[3];
          } __packed reply = {
                  {
                          .type = msg_type,
                          .len = sizeof(reply.body)
                  },
                  "OK"
          };
  
          mutex_lock(&u->reply_mutex);
          rc = queue_reply(&u->read_buffers, &reply, sizeof(reply));
          wake_up(&u->read_waitq);
          mutex_unlock(&u->reply_mutex);
  }

The synthesised reply doesn't echo the req_id back to user.

Wei.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel