Re: segfault in xl create for HVM with PCI passthrough

[Atom2 <ariel.atom2@web2web.at>]

[-- Attachment #1: Type: text/plain, Size: 4721 bytes --]

To keep the thread together I am again submitting the relevant parts of 
my last answer (which due to an error on my part originally went out to 
Ian only and I only forward it to the list afterwards which resulted in 
an out-of-thread appeareance) together with the (new) results of my gdb 
excercise. Sorry for any confusion this may(might have) cause(d).

Am 28.10.14 um 17:04 schrieb Ian Campbell:
[...]
>> With regards to gdb: I can certainly run the command under gdb after
>> including debug support to the executables - that's no big deal.
>> I would, however, ask for your advice as to what I need to recompile
>> with debugger support? Is xen-tools (which includes xl) sufficient
>
> I think just the Xen bits would be sufficient, at least to start with.
>
>>   or
>> would you think that I also need to include debug support for gcc as the
>> library that is mentioned in /var/log/messages (libgcc_s.so.1) seems to
>> belong to the gcc package? Or is this library a red herring that just
>> works as the catch-all code getting and finally handling the segfault?
>
> I'd recommend ignoring it for now, in the event that the backtrace from
> just the xen bits suggests a gcc issue that might change. My money right
> now is on it being a xen issue though.
>
After recompiling xen-tools with gdb debug support I started the 
following command:
# gdb --args /usr/sbin/xl create pfsense -c

Please find the command's screen output after its start up to the 
segfault including the output of the bt command after the segfault in 
the attached document named "create".

Furthermore I did the same for the destroy command:
# gdb --args /usr/sbin/xl destroy pfsense

The output of this command is in the attached document named "destroy".

I haven't got much experience with gdb yet so I am unable to interpret 
the outcome of either. Also if there's more/different stuff required, 
please advise me what to do next. Tx.

>>> [...]
>>>> pci          = [ '04:00.0', '0a:08.0', '0a:0b.0' ]
>>>
>>> You say in $subject that the failure is with PCI, is that because you've
>>> tried an HVM domain without and it is ok, or is it just that all your
>>> HVM domains happen to have passthrough enabled?
>> I haven't tried HVM domains without PCI passthrough (but PV domains w/o
>> PCI passthrough and they did not segfault) so far as all my HVM domains
>> require PCI devices (either at least a network card for pfsense - in
>> actual facts it's more than one that's being passed through - or a SATA
>> controller for my second HVM which is used as a storage VM).
>
> The VM doesn't need to be fully functional, it just needs to boot
> without crashing the toolstack. Just running your existing VM with the
> pci line commented out would be useful.
Before re-compiling the xen-tools I made a quick test as you suggested
and commented out the pci line from my config file ... and the boot menu 
showed up (which it did not before when the segfault happened).
I did not boot the pfsense vm any further as this might lead to a change 
in my configuration due to missing devices, but to me this at first 
sight seemed to indicate that is has to do with the PCI passthrough 
functionality.
Although as I did not want to boot the machine (and "xl shutdown" did
not work, not even with -F) I then decided to
     xl destroy pfsense
and that printed a segmentation fault message (in both the shell window
where I started the command from and the console window where the
boot-menu was shown) despite no PCI devices being passed through.

To also check PCI passthrough with a PV domain: I added a pci device to
a config file for a PV domain and started that with
     xl create voip -c
The boot menu appeared without issues. I then also tried
     xl destroy voip
from another window and that issued the following error messages in the
shell window (without using any -vvv option):

# xl destroy voip
libxl: error: libxl_pci.c:1247:do_pci_remove: xc_domain_irq_permission
irq=17
libxl: error: libxl_device.c:1127:libxl__wait_for_backend: Backend
/local/domain/0/backend/pci/4/0 not ready
libxl: error: libxl_pci.c:1247:do_pci_remove: xc_domain_irq_permission
irq=16
libxl: error: libxl_device.c:1127:libxl__wait_for_backend: Backend
/local/domain/0/backend/pci/4/0 not ready
libxl: error: libxl_pci.c:1247:do_pci_remove: xc_domain_irq_permission
irq=23
libxl: error: libxl_device.c:1127:libxl__wait_for_backend: Backend
/local/domain/0/backend/pci/4/0 not ready
Segmentation fault

The "Segmentation fault" message also appeared in both the console
window for the domU and the shell window.

This all seems a bit strange to me at the moement, but I am sure with
your help we will arrive at the grounds of this.

Thanks and regards Atom

[-- Attachment #2: create --]
[-- Type: text/plain, Size: 2863 bytes --]

GNU gdb (Gentoo 7.6.2 p1) 7.6.2
Copyright (C) 2013 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-pc-linux-gnu".
For bug reporting instructions, please see:
<http://bugs.gentoo.org/>...
Reading symbols from /usr/sbin/xl...Reading symbols from /usr/lib64/debug/usr/sbin/xl.debug...done.
done.
(gdb) run
Starting program: /usr/sbin/xl create pfsense -c
warning: no loadable sections found in added symbol-file system-supplied DSO at 0x7ffff7ffa000
warning: Could not load shared library symbols for linux-vdso.so.1.
Do you need "set solib-search-path" or "set sysroot"?
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib64/libthread_db.so.1".
Parsing config from pfsense
xc: info: VIRTUAL MEMORY ARRANGEMENT:
  Loader:        0000000000100000->00000000001c12a4
  Modules:       0000000000000000->0000000000000000
  TOTAL:         0000000000000000->000000001f800000
  ENTRY ADDRESS: 0000000000100000
xc: info: PHYSICAL MEMORY ALLOCATION:
  4KB PAGES: 0x0000000000000200
  2MB PAGES: 0x00000000000000fb
  1GB PAGES: 0x0000000000000000
[New Thread 0x7ffff7ff5700 (LWP 13464)]
[New Thread 0x7ffff7fe6700 (LWP 13574)]

Program received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0x7ffff7fe6700 (LWP 13574)]
0x00007ffff5882b64 in ?? () from /usr/lib/gcc/x86_64-pc-linux-gnu/4.8.3/libgcc_s.so.1
(gdb) bt
#0  0x00007ffff5882b64 in ?? () from /usr/lib/gcc/x86_64-pc-linux-gnu/4.8.3/libgcc_s.so.1
#1  0x00007ffff58835cc in ?? () from /usr/lib/gcc/x86_64-pc-linux-gnu/4.8.3/libgcc_s.so.1
#2  0x00007ffff5883945 in ?? () from /usr/lib/gcc/x86_64-pc-linux-gnu/4.8.3/libgcc_s.so.1
#3  0x00007ffff58845c6 in ?? () from /usr/lib/gcc/x86_64-pc-linux-gnu/4.8.3/libgcc_s.so.1
#4  0x00007ffff588494c in _Unwind_ForcedUnwind () from /usr/lib/gcc/x86_64-pc-linux-gnu/4.8.3/libgcc_s.so.1
#5  0x00007ffff731a733 in __pthread_unwind () from /lib64/libpthread.so.0
#6  0x00007ffff7311b49 in sigcancel_handler () from /lib64/libpthread.so.0
#7  <signal handler called>
#8  0x00007ffff731ae4d in read () from /lib64/libpthread.so.0
#9  0x00007ffff6b17b53 in read (__nbytes=16, __buf=0x7fffe80008d0, __fd=18) at /usr/include/bits/unistd.h:44
#10 read_all (fd=18, data=data@entry=0x7fffe80008d0, len=len@entry=16, nonblocking=nonblocking@entry=0) at xs.c:374
#11 0x00007ffff6b17c94 in read_message (h=h@entry=0x555555785670, nonblocking=nonblocking@entry=0) at xs.c:1139
#12 0x00007ffff6b18626 in read_thread (arg=0x555555785670) at xs.c:1211
#13 0x00007ffff731332d in start_thread () from /lib64/libpthread.so.0
#14 0x00007ffff704a19d in clone () from /lib64/libc.so.6
(gdb)

[-- Attachment #3: destroy --]
[-- Type: text/plain, Size: 2431 bytes --]

GNU gdb (Gentoo 7.6.2 p1) 7.6.2
Copyright (C) 2013 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-pc-linux-gnu".
For bug reporting instructions, please see:
<http://bugs.gentoo.org/>...
Reading symbols from /usr/sbin/xl...Reading symbols from /usr/lib64/debug/usr/sbin/xl.debug...done.
done.
(gdb) run
Starting program: /usr/sbin/xl destroy pfsense
warning: no loadable sections found in added symbol-file system-supplied DSO at 0x7ffff7ffa000
warning: Could not load shared library symbols for linux-vdso.so.1.
Do you need "set solib-search-path" or "set sysroot"?
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib64/libthread_db.so.1".
[New Thread 0x7ffff7ff6700 (LWP 13639)]

Program received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0x7ffff7ff6700 (LWP 13639)]
0x00007ffff5882b64 in ?? () from /usr/lib/gcc/x86_64-pc-linux-gnu/4.8.3/libgcc_s.so.1
(gdb) bt
#0  0x00007ffff5882b64 in ?? () from /usr/lib/gcc/x86_64-pc-linux-gnu/4.8.3/libgcc_s.so.1
#1  0x00007ffff58835cc in ?? () from /usr/lib/gcc/x86_64-pc-linux-gnu/4.8.3/libgcc_s.so.1
#2  0x00007ffff5883945 in ?? () from /usr/lib/gcc/x86_64-pc-linux-gnu/4.8.3/libgcc_s.so.1
#3  0x00007ffff58845c6 in ?? () from /usr/lib/gcc/x86_64-pc-linux-gnu/4.8.3/libgcc_s.so.1
#4  0x00007ffff588494c in _Unwind_ForcedUnwind () from /usr/lib/gcc/x86_64-pc-linux-gnu/4.8.3/libgcc_s.so.1
#5  0x00007ffff731a733 in __pthread_unwind () from /lib64/libpthread.so.0
#6  0x00007ffff7311b49 in sigcancel_handler () from /lib64/libpthread.so.0
#7  <signal handler called>
#8  0x00007ffff731ae4d in read () from /lib64/libpthread.so.0
#9  0x00007ffff6b17b53 in read (__nbytes=16, __buf=0x7ffff0000a10, __fd=10) at /usr/include/bits/unistd.h:44
#10 read_all (fd=10, data=data@entry=0x7ffff0000a10, len=len@entry=16, nonblocking=nonblocking@entry=0) at xs.c:374
#11 0x00007ffff6b17c94 in read_message (h=h@entry=0x55555577ed40, nonblocking=nonblocking@entry=0) at xs.c:1139
#12 0x00007ffff6b18626 in read_thread (arg=0x55555577ed40) at xs.c:1211
#13 0x00007ffff731332d in start_thread () from /lib64/libpthread.so.0
#14 0x00007ffff704a19d in clone () from /lib64/libc.so.6
(gdb)

[-- Attachment #4: Type: text/plain, Size: 126 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel