[Qemu-devel] bind interdomain ioctl error xen-kvm.c

[Qemu-devel] bind interdomain ioctl error xen-kvm.c

[Rishi Ranjan]

[-- Attachment #1: Type: text/plain, Size: 670 bytes --]

I am trying to use Xen as accelerator for my Qemu machine. I have created a
guest domain with following xl config:

builder = "hvm"
name = "qemu-hvm"
memory = "512"
vcpus = 1
vif = ['']
vnc = 1
boot="c"


When I try to run with following parameters:

-machine q35,accel=xen -cpu qemu64 -bios ./pc-bios/bios-256k.bin -xen-domid
"Domain id of guest"

I am getting follwing error from xen-hvm.c:

"bind interdomain ioctl error" in xen_hvm_init while calling
state->shared_vmport_page =
            xc_map_foreign_range(xen_xc, xen_domid, XC_PAGE_SIZE,
                                 PROT_READ|PROT_WRITE, ioreq_pfn);

Can someone help me get this working?

Thanks,
Rishi

[-- Attachment #2: Type: text/html, Size: 1144 bytes --]

Re: [Qemu-devel] bind interdomain ioctl error xen-kvm.c

[Stefano Stabellini]

[-- Attachment #1: Type: text/plain, Size: 1159 bytes --]

On Tue, 30 Dec 2014, Rishi Ranjan wrote:
> I am trying to use Xen as accelerator for my Qemu machine. I have created a guest domain with following xl config: 
> builder = "hvm"
> name = "qemu-hvm"
> memory = "512"
> vcpus = 1
> vif = ['']
> vnc = 1
> boot="c"
> 
> 
> When I try to run with following parameters: 
> 
> -machine q35,accel=xen -cpu qemu64 -bios ./pc-bios/bios-256k.bin -xen-domid "Domain id of guest"

You should know that q35 emulation is not fully working on Xen yet.
However Anthony (CC'ed) should have some patches for it.

That said, it does not look like this error has something to do with
q35.


> I am getting follwing error from xen-hvm.c: 
> 
> "bind interdomain ioctl error" in xen_hvm_init while calling         state->shared_vmport_page =
>             xc_map_foreign_range(xen_xc, xen_domid, XC_PAGE_SIZE,
>                                  PROT_READ|PROT_WRITE, ioreq_pfn); 
> 
> Can someone help me get this working? 

Can you post the full output of the logs?
What is the Xen version that you are running?
Did you execute the xencommons init script at boot time?

Re: [Qemu-devel] bind interdomain ioctl error xen-kvm.c

[Rishi Ranjan]

[-- Attachment #1.1: Type: text/plain, Size: 2377 bytes --]

Hi Stefano,
                 Please find my answers inline.

>>>> However Anthony (CC'ed) should have some patches for it.
Anthony, can you please share any patch that can help me with this?


>>>> Can you post the full output of the logs?
I have attached the output of "sudo xl -v create /etc/xen/qemu-pv.cfg"
as xl_create.txt. I have also enabled DEBUG_XEN_HVM in xen-hvm.c and pasted
output of "sudo ./x86_64-softmmu/qemu-system-x86_64  -machine q35,accel=xen
-cpu qemu64 -xen-domid 13" below:


xen: shared page at pfn feffd
xen: buffered io page at pfn feffb
bind interdomain ioctl error 22
xen hardware virtual machine initialisation failed


>>>> What is the Xen version that you are running?

I am using XEN 4.4.1 as this is the default on Ubuntu 14.04. I have
attached the output of "xl info" command as xl_info.txt.

>>>>Did you execute the xencommons init script at boot time?
On Ubuntu I don't see /etc/init.d/xencommon but there is a /etc/init.d/xen
script which starts xenstored and xenconsoled. I did confirm from ps aufx
that both the daemons are running. I have attched log for "ps aufx" as
ps_aufx_grep_xen.txt .






On Mon, Jan 5, 2015 at 4:48 AM, Stefano Stabellini <
stefano.stabellini@eu.citrix.com> wrote:

> On Tue, 30 Dec 2014, Rishi Ranjan wrote:
> > I am trying to use Xen as accelerator for my Qemu machine. I have
> created a guest domain with following xl config:
> > builder = "hvm"
> > name = "qemu-hvm"
> > memory = "512"
> > vcpus = 1
> > vif = ['']
> > vnc = 1
> > boot="c"
> >
> >
> > When I try to run with following parameters:
> >
> > -machine q35,accel=xen -cpu qemu64 -bios ./pc-bios/bios-256k.bin
> -xen-domid "Domain id of guest"
>
> You should know that q35 emulation is not fully working on Xen yet.
> However Anthony (CC'ed) should have some patches for it.
>
> That said, it does not look like this error has something to do with
> q35.
>
>
> > I am getting follwing error from xen-hvm.c:
> >
> > "bind interdomain ioctl error" in xen_hvm_init while calling
> state->shared_vmport_page =
> >             xc_map_foreign_range(xen_xc, xen_domid, XC_PAGE_SIZE,
> >                                  PROT_READ|PROT_WRITE, ioreq_pfn);
> >
> > Can someone help me get this working?
>
> Can you post the full output of the logs?
> What is the Xen version that you are running?
> Did you execute the xencommons init script at boot time?

[-- Attachment #1.2: Type: text/html, Size: 3957 bytes --]

[-- Attachment #2: xl_info.txt --]
[-- Type: text/plain, Size: 1290 bytes --]

host                   : rishi-700-327c
release                : 3.13.0-43-generic
version                : #72-Ubuntu SMP Mon Dec 8 19:35:06 UTC 2014
machine                : x86_64
nr_cpus                : 8
max_cpu_id             : 7
nr_nodes               : 1
cores_per_socket       : 4
threads_per_core       : 2
cpu_mhz                : 3392
hw_caps                : bfebfbff:2c100800:00000000:00007f00:77fafbff:00000000:00000021:00002fbb
virt_caps              : hvm hvm_directio
total_memory           : 16289
free_memory            : 128
sharing_freed_memory   : 0
sharing_used_memory    : 0
outstanding_claims     : 0
free_cpus              : 0
xen_major              : 4
xen_minor              : 4
xen_extra              : .1
xen_version            : 4.4.1
xen_caps               : xen-3.0-x86_64 xen-3.0-x86_32p hvm-3.0-x86_32 hvm-3.0-x86_32p hvm-3.0-x86_64 
xen_scheduler          : credit
xen_pagesize           : 4096
platform_params        : virt_start=0xffff800000000000
xen_changeset          : 
xen_commandline        : placeholder no-real-mode edd=off
cc_compiler            : gcc (Ubuntu 4.8.2-19ubuntu1) 4.8.2
cc_compile_by          : stefan.bader
cc_compile_domain      : canonical.com
cc_compile_date        : Wed Nov 26 14:12:33 UTC 2014
xend_config_format     : 4

[-- Attachment #3: ps_aufx_grep_xen.txt --]
[-- Type: text/plain, Size: 772 bytes --]

root        65  0.0  0.0      0     0 ?        S    09:33   0:00  \_ [xenwatch]
root        66  0.0  0.0      0     0 ?        S    09:33   0:00  \_ [xenbus]
root        72  0.0  0.0      0     0 ?        S<   09:33   0:00  \_ [xenbus_frontend]
root      1112  0.0  0.0  10232  2408 ?        Ss   09:33   0:00 dhclient -1 -v -pf /run/dhclient.xenbr0.pid -lf /var/lib/dhcp/dhclient.xenbr0.leases xenbr0
root      1524  0.0  0.0  10992   960 ?        S    09:33   0:00 /usr/lib/xen-4.4/bin/xenstored --pid-file=/var/run/xenstore.pid
root      1530  0.0  0.0  82572   632 ?        SLl  09:33   0:00 /usr/lib/xen-4.4/bin/xenconsoled --pid-file=/var/run/xenconsoled.pid
rishi     4309  0.0  0.0  17444   932 pts/1    S+   10:44   0:00          |       \_ grep --color=auto xen

[-- Attachment #4: xl_create.txt --]
[-- Type: text/plain, Size: 765 bytes --]

sudo xl -v create /etc/xen/qemu-pv.cfg 
Parsing config from /etc/xen/qemu-pv.cfg
libxl: detail: libxl_dom.c:195:numa_place_domain: NUMA placement candidate with 1 nodes, 8 cpus and 4477 KB free selected
xc: detail: elf_parse_binary: phdr: paddr=0x100000 memsz=0xa7a84
xc: detail: elf_parse_binary: memory: 0x100000 -> 0x1a7a84
xc: detail: VIRTUAL MEMORY ARRANGEMENT:
  Loader:        0000000000100000->00000000001a7a84
  Modules:       0000000000000000->0000000000000000
  TOTAL:         0000000000000000->00000000ff800000
  ENTRY ADDRESS: 0000000000100620
xc: detail: PHYSICAL MEMORY ALLOCATION:
  4KB PAGES: 0x0000000000000200
  2MB PAGES: 0x00000000000003fb
  1GB PAGES: 0x0000000000000002
xc: detail: elf_load_binary: phdr 0 at 0x7fe78bc23000 -> 0x7fe78bcc190d

Re: [Qemu-devel] bind interdomain ioctl error xen-kvm.c

[Don Slutz]

On 01/05/15 14:10, Rishi Ranjan wrote:
> Hi Stefano,
>                  Please find my answers inline. 
> 
>>>>> However Anthony (CC'ed) should have some patches for it.
> Anthony, can you please share any patch that can help me with this?
> 
> 
>>>>> Can you post the full output of the logs?
> I have attached the output of "sudo xl -v create /etc/xen/qemu-pv.cfg"
> as xl_create.txt. I have also enabled DEBUG_XEN_HVM in xen-hvm.c and
> pasted output of "sudo ./x86_64-softmmu/qemu-system-x86_64  -machine
> q35,accel=xen -cpu qemu64 -xen-domid 13" below: 
> 

The output of "sudo xl -vvv create /etc/xen/qemu-pv.cfg" will help
since it includes how xen invokes qemu.

> 
> xen: shared page at pfn feffd
> xen: buffered io page at pfn feffb
> bind interdomain ioctl error 22
> xen hardware virtual machine initialisation failed


This says that where you are reporting the error is not
correct.

The message "buffered io page at pfn feffb" is output
after the code that contains shared_vmport_page.

Also posting the data in /var/log/xen/qemu-dm-qemu-hvm.log
will help.


It looks like you are re-starting QEMU on a domain.  This is not
supported as far as I know (and not clear that you are starting the same
version that xen tried).


> 
> 
>>>>> What is the Xen version that you are running?
> 
> I am using XEN 4.4.1 as this is the default on Ubuntu 14.04. I have
> attached the output of "xl info" command as xl_info.txt. 
> 

You are running QEMU 2.2.0 or later based on having
shared_vmport_page in the code.


>>>>>Did you execute the xencommons init script at boot time?
> On Ubuntu I don't see /etc/init.d/xencommon but there is a
> /etc/init.d/xen script which starts xenstored and xenconsoled. I did
> confirm from ps aufx that both the daemons are running. I have attched
> log for "ps aufx" as ps_aufx_grep_xen.txt .
> 
> 
> 
> 
> 
> 
> On Mon, Jan 5, 2015 at 4:48 AM, Stefano Stabellini
> <stefano.stabellini@eu.citrix.com
> <mailto:stefano.stabellini@eu.citrix.com>> wrote:
> 
>     On Tue, 30 Dec 2014, Rishi Ranjan wrote:
>     > I am trying to use Xen as accelerator for my Qemu machine. I have created a guest domain with following xl config: 
>     > builder = "hvm"
>     > name = "qemu-hvm"
>     > memory = "512"
>     > vcpus = 1
>     > vif = ['']
>     > vnc = 1
>     > boot="c"
>     >
>     >
>     > When I try to run with following parameters: 
>     >
>     > -machine q35,accel=xen -cpu qemu64 -bios ./pc-bios/bios-256k.bin -xen-domid "Domain id of guest"
> 
>     You should know that q35 emulation is not fully working on Xen yet.
>     However Anthony (CC'ed) should have some patches for it.
> 
>     That said, it does not look like this error has something to do with
>     q35.
> 
> 
>     > I am getting follwing error from xen-hvm.c: 
>     >
>     > "bind interdomain ioctl error" in xen_hvm_init while calling         state->shared_vmport_page =
>     >             xc_map_foreign_range(xen_xc, xen_domid, XC_PAGE_SIZE,
>     >                                  PROT_READ|PROT_WRITE, ioreq_pfn); 

To get here, xen_get_vmport_regs_pfn() needs to return 0.

In include/hw/xen/xen_common.h:


#ifdef HVM_PARAM_VMPORT_REGS_PFN
static inline int xen_get_vmport_regs_pfn(XenXC xc, domid_t dom,
                                          unsigned long *vmport_regs_pfn)
{
    return xc_get_hvm_param(xc, dom, HVM_PARAM_VMPORT_REGS_PFN,
                            vmport_regs_pfn);
}
#else
static inline int xen_get_vmport_regs_pfn(XenXC xc, domid_t dom,
                                          unsigned long *vmport_regs_pfn)
{
    return -ENOSYS;
}
#endif


requires HVM_PARAM_VMPORT_REGS_PFN to be defined before it will return
0.

I am sure that xen 4.4.1 does not define this (patch that does is
waiting for xen 4.6 to open up).


   -Don Slutz

>     >
>     > Can someone help me get this working? 
> 
>     Can you post the full output of the logs?
>     What is the Xen version that you are running?
>     Did you execute the xencommons init script at boot time?
> 
> 

Re: [Qemu-devel] [Xen-devel] bind interdomain ioctl error xen-kvm.c

[Ian Campbell]

On Mon, 2015-01-05 at 11:10 -0800, Rishi Ranjan wrote:
> Hi Stefano,
>                  Please find my answers inline. 
> 
> 
> >>>> However Anthony (CC'ed) should have some patches for it.
> 
> Anthony, can you please share any patch that can help me with this?
> 
> 
> 
> 
> >>>> Can you post the full output of the logs?
> I have attached the output of "sudo xl -v create /etc/xen/qemu-pv.cfg"
> as xl_create.txt. I have also enabled DEBUG_XEN_HVM in xen-hvm.c and
> pasted output of "sudo ./x86_64-softmmu/qemu-system-x86_64  -machine
> q35,accel=xen -cpu qemu64 -xen-domid 13" below: 

Have you done anything which stops "xl create" from also launching a
qemu? The guest cfg file you posted earlier didn't suggest so. Running
two qemu's against the same domain doesn't seem likely to result in good
things...

> >>>>Did you execute the xencommons init script at boot time?
> On Ubuntu I don't see /etc/init.d/xencommon but there is
> a /etc/init.d/xen script which starts xenstored and xenconsoled.

Yes, Debian and Ubuntu don't use the upstream initscript but have their
own.

Ian.

Re: [Qemu-devel] bind interdomain ioctl error xen-kvm.c

[Anthony PERARD]

On Mon, Jan 05, 2015 at 11:10:34AM -0800, Rishi Ranjan wrote:
> >>>> However Anthony (CC'ed) should have some patches for it.
> Anthony, can you please share any patch that can help me with this?

Hi,

The "patches" are in two repos:
git://xenbits.xen.org/people/aperard/xen-unstable.git
branch: machine-q35-wip

git://xenbits.xen.org/people/aperard/qemu-dm.git
branch: xen-q35-wip

For the xen tree, it's the 3 top commit, and the top commit for qemu.
Once applied, hvmloader will only work with Q35 and not with the
default machine.

-- 
Anthony PERARD

Re: [Qemu-devel] bind interdomain ioctl error xen-kvm.c

[Anthony PERARD]

On Tue, Jan 06, 2015 at 11:11:48AM +0000, Anthony PERARD wrote:
> On Mon, Jan 05, 2015 at 11:10:34AM -0800, Rishi Ranjan wrote:
> > >>>> However Anthony (CC'ed) should have some patches for it.
> > Anthony, can you please share any patch that can help me with this?
> 
> Hi,
> 
> The "patches" are in two repos:
> git://xenbits.xen.org/people/aperard/xen-unstable.git
> branch: machine-q35-wip
> 
> git://xenbits.xen.org/people/aperard/qemu-dm.git
> branch: xen-q35-wip
> 
> For the xen tree, it's the 3 top commit, and the top commit for qemu.
> Once applied, hvmloader will only work with Q35 and not with the
> default machine.

To start a guest, I have this in the guest config:
disk = [ 'phy:/dev/mapper/vg-guest_q35,sda,w' ]
machine='q35'

If you wish, you can recompile only hvmloader and qemu and have
something like this in your guest config:
firmware_override      = '/root/hvmloader-q35'
device_model_override = '/root/qemu-xen-q35'
device_model_version = 'qemu-xen'
device_model_args_hvm = [ '-machine', 'q35,accel=xen' ]


Hope that help.

-- 
Anthony PERARD