Domain-0 kernel boot failure with xen-unstable and linux-2.6.18-xen

Domain-0 kernel boot failure with xen-unstable and linux-2.6.18-xen

[Masaki Kanno]

Hi,

I have encountered a problem with xen-unstable and linux-2.6.18-xen.
The problem is that Domain-0 kernel is not booted.  I'm pinpointing 
a changeset causing the problem.

xen-unstable   linux-2.6.18-xen   result
cs:20597       cs:959             success
cs:20654       cs:962             success
cs:20702       cs:967             success

cs:20744       cs:970             failure
cs:20779       cs:981             failure

Best regards,
 Kan

Re: Domain-0 kernel boot failure with xen-unstable andlinux-2.6.18-xen

[Masaki Kanno]

Thu, 14 Jan 2010 10:51:45 +0900, Masaki Kanno wrote:

>Hi,
>
>I have encountered a problem with xen-unstable and linux-2.6.18-xen.
>The problem is that Domain-0 kernel is not booted.  I'm pinpointing 
>a changeset causing the problem.
>
>xen-unstable   linux-2.6.18-xen   result
>cs:20597       cs:959             success
>cs:20654       cs:962             success
>cs:20702       cs:967             success

 cs:20714       cs:969             failure
 cs:20729       cs:970             failure
>cs:20744       cs:970             failure
>cs:20779       cs:981             failure
>

Best regards,
 Kan

Re: Domain-0 kernel boot failure with xen-unstableandlinux-2.6.18-xen

[Masaki Kanno]

Thu, 14 Jan 2010 11:57:20 +0900, Masaki Kanno wrote:

>Thu, 14 Jan 2010 10:51:45 +0900, Masaki Kanno wrote:
>
>>Hi,
>>
>>I have encountered a problem with xen-unstable and linux-2.6.18-xen.
>>The problem is that Domain-0 kernel is not booted.  I'm pinpointing 
>>a changeset causing the problem.
>>
>>xen-unstable   linux-2.6.18-xen   result
>>cs:20597       cs:959             success
>>cs:20654       cs:962             success
>>cs:20702       cs:967             success
  cs:20704       cs:967             success

  cs:20710       cs:967             failure
> cs:20714       cs:969             failure
> cs:20729       cs:970             failure
>>cs:20744       cs:970             failure
>>cs:20779       cs:981             failure

Best regards,
 Kan

Re: Domain-0 kernel boot failure withxen-unstableandlinux-2.6.18-xen

[Masaki Kanno]

Thu, 14 Jan 2010 14:45:07 +0900, Masaki Kanno wrote:

>Thu, 14 Jan 2010 11:57:20 +0900, Masaki Kanno wrote:
>
>>Thu, 14 Jan 2010 10:51:45 +0900, Masaki Kanno wrote:
>>
>>>Hi,
>>>
>>>I have encountered a problem with xen-unstable and linux-2.6.18-xen.
>>>The problem is that Domain-0 kernel is not booted.  I'm pinpointing 
>>>a changeset causing the problem.
>>>
>>>xen-unstable   linux-2.6.18-xen   result
>>>cs:20597       cs:959             success
>>>cs:20654       cs:962             success
>>>cs:20702       cs:967             success
>  cs:20704       cs:967             success
   cs:20705       cs:967             failure
>  cs:20710       cs:967             failure
>> cs:20714       cs:969             failure
>> cs:20729       cs:970             failure
>>>cs:20744       cs:970             failure
>>>cs:20779       cs:981             failure

The problem occurs by xen-unstable changeset 20705.
I can pinpoint the changeset causing the problem, but I cannot find out 
a bug in the changeset...

Best regards,
 Kan

Re: Domain-0 kernel boot failure withxen-unstableandlinux-2.6.18-xen

[Jan Beulich]

You forgot to tell us in what way the boot fails.

Jan

>>> Masaki Kanno <kanno.masaki@jp.fujitsu.com> 14.01.10 07:18 >>>
Thu, 14 Jan 2010 14:45:07 +0900, Masaki Kanno wrote:

>Thu, 14 Jan 2010 11:57:20 +0900, Masaki Kanno wrote:
>
>>Thu, 14 Jan 2010 10:51:45 +0900, Masaki Kanno wrote:
>>
>>>Hi,
>>>
>>>I have encountered a problem with xen-unstable and linux-2.6.18-xen.
>>>The problem is that Domain-0 kernel is not booted.  I'm pinpointing 
>>>a changeset causing the problem.
>>>
>>>xen-unstable   linux-2.6.18-xen   result
>>>cs:20597       cs:959             success
>>>cs:20654       cs:962             success
>>>cs:20702       cs:967             success
>  cs:20704       cs:967             success
   cs:20705       cs:967             failure
>  cs:20710       cs:967             failure
>> cs:20714       cs:969             failure
>> cs:20729       cs:970             failure
>>>cs:20744       cs:970             failure
>>>cs:20779       cs:981             failure

The problem occurs by xen-unstable changeset 20705.
I can pinpoint the changeset causing the problem, but I cannot find out 
a bug in the changeset...

Best regards,
 Kan



_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com 
http://lists.xensource.com/xen-devel

Re: Domain-0 kernel boot failure withxen-unstableandlinux-2.6.18-xen

[Keir Fraser]

On 14/01/2010 06:18, "Masaki Kanno" <kanno.masaki@jp.fujitsu.com> wrote:

>    cs:20705       cs:967             failure
>>  cs:20710       cs:967             failure
>>> cs:20714       cs:969             failure
>>> cs:20729       cs:970             failure
>>>> cs:20744       cs:970             failure
>>>> cs:20779       cs:981             failure
> 
> The problem occurs by xen-unstable changeset 20705.
> I can pinpoint the changeset causing the problem, but I cannot find out
> a bug in the changeset...

Yes, the changeset does look pretty harmless, at least as far as dom0
booting is concerned. Obviously dom0 boot is working for lots of other
people, so perhaps this is also related to the machine you are running on?

 -- Keir

Re: Domain-0 kernel boot failurewithxen-unstableandlinux-2.6.18-xen

[Masaki Kanno]

Hi Jan,

Thu, 14 Jan 2010 09:05:32 +0000, "Jan Beulich" wrote:

>You forgot to tell us in what way the boot fails.

Oops!  Could you refer to the following about a serial log when the 
problem occurred?

http://lists.xensource.com/archives/html/xen-devel/2010-01/msg00338.html

Accurately, the problem is not "Domain-0 kernel boot fails" but "Domain-
0 kernel boot does not start".

Best regards,
 Kan

>Jan
>
>>>> Masaki Kanno <kanno.masaki@jp.fujitsu.com> 14.01.10 07:18 >>>
>Thu, 14 Jan 2010 14:45:07 +0900, Masaki Kanno wrote:
>
>>Thu, 14 Jan 2010 11:57:20 +0900, Masaki Kanno wrote:
>>
>>>Thu, 14 Jan 2010 10:51:45 +0900, Masaki Kanno wrote:
>>>
>>>>Hi,
>>>>
>>>>I have encountered a problem with xen-unstable and linux-2.6.18-xen.
>>>>The problem is that Domain-0 kernel is not booted.  I'm pinpointing 
>>>>a changeset causing the problem.
>>>>
>>>>xen-unstable   linux-2.6.18-xen   result
>>>>cs:20597       cs:959             success
>>>>cs:20654       cs:962             success
>>>>cs:20702       cs:967             success
>>  cs:20704       cs:967             success
>   cs:20705       cs:967             failure
>>  cs:20710       cs:967             failure
>>> cs:20714       cs:969             failure
>>> cs:20729       cs:970             failure
>>>>cs:20744       cs:970             failure
>>>>cs:20779       cs:981             failure
>
>The problem occurs by xen-unstable changeset 20705.
>I can pinpoint the changeset causing the problem, but I cannot find out 
>a bug in the changeset...
>
>Best regards,
> Kan
>
>
>
>_______________________________________________
>Xen-devel mailing list
>Xen-devel@lists.xensource.com 
>http://lists.xensource.com/xen-devel
>
>
>_______________________________________________
>Xen-devel mailing list
>Xen-devel@lists.xensource.com
>http://lists.xensource.com/xen-devel

Re: Domain-0 kernel boot failurewithxen-unstableandlinux-2.6.18-xen

[Keir Fraser]

On 14/01/2010 11:07, "Masaki Kanno" <kanno.masaki@jp.fujitsu.com> wrote:

> Hi Jan,
> 
> Thu, 14 Jan 2010 09:05:32 +0000, "Jan Beulich" wrote:
> 
>> You forgot to tell us in what way the boot fails.
> 
> Oops!  Could you refer to the following about a serial log when the
> problem occurred?
> 
> http://lists.xensource.com/archives/html/xen-devel/2010-01/msg00338.html
> 
> Accurately, the problem is not "Domain-0 kernel boot fails" but "Domain-
> 0 kernel boot does not start".

I don't know how much experience with kernel issues you have, but I would
take the EIP from the CPU0 guest dump in that log file (the EIP is c01edc67)
and look at that code address in a disassembly of the dom0 kernel image
(objdump -d vmlinux). Seeing what function you're in would be a good start.
'addr2line -e vmlinux c01edc67' would also be useful: tells you the source
file and line number.

 -- Keir

Re: Domain-0 kernel boot failurewithxen-unstableandlinux-2.6.18-xen

[Masaki Kanno]

Thu, 14 Jan 2010 09:31:53 +0000, Keir Fraser wrote:

>On 14/01/2010 06:18, "Masaki Kanno" <kanno.masaki@jp.fujitsu.com> wrote:
>
>>    cs:20705       cs:967             failure
>>>  cs:20710       cs:967             failure
>>>> cs:20714       cs:969             failure
>>>> cs:20729       cs:970             failure
>>>>> cs:20744       cs:970             failure
>>>>> cs:20779       cs:981             failure
>> 
>> The problem occurs by xen-unstable changeset 20705.
>> I can pinpoint the changeset causing the problem, but I cannot find out
>> a bug in the changeset...
>
>Yes, the changeset does look pretty harmless, at least as far as dom0
>booting is concerned. Obviously dom0 boot is working for lots of other
>people, so perhaps this is also related to the machine you are running on?

Hi Keir,

Pentium III 1GHz...  I will switch off my machine.

Best regards,
 Kan

Re: Domain-0 kernel bootfailurewithxen-unstableandlinux-2.6.18-xen

[Masaki Kanno]

Thu, 14 Jan 2010 11:26:12 +0000, Keir Fraser wrote:

>On 14/01/2010 11:07, "Masaki Kanno" <kanno.masaki@jp.fujitsu.com> wrote:
>
>> Hi Jan,
>> 
>> Thu, 14 Jan 2010 09:05:32 +0000, "Jan Beulich" wrote:
>> 
>>> You forgot to tell us in what way the boot fails.
>> 
>> Oops!  Could you refer to the following about a serial log when the
>> problem occurred?
>> 
>> http://lists.xensource.com/archives/html/xen-devel/2010-01/msg00338.html
>> 
>> Accurately, the problem is not "Domain-0 kernel boot fails" but "Domain-
>> 0 kernel boot does not start".
>
>I don't know how much experience with kernel issues you have, but I would
>take the EIP from the CPU0 guest dump in that log file (the EIP is c01edc67)
>and look at that code address in a disassembly of the dom0 kernel image
>(objdump -d vmlinux). Seeing what function you're in would be a good start.
>'addr2line -e vmlinux c01edc67' would also be useful: tells you the source
>file and line number.
>

Thanks Keir.  I will try it tomorrow.

Best regards,
 Kan

Re: Domain-0 kernelbootfailurewithxen-unstableandlinux-2.6.18-xen

[Masaki Kanno]

[-- Attachment #1: Mail message body --]
[-- Type: text/plain, Size: 1906 bytes --]

Thu, 14 Jan 2010 20:42:07 +0900, Masaki Kanno wrote:

>Thu, 14 Jan 2010 11:26:12 +0000, Keir Fraser wrote:
>
>>On 14/01/2010 11:07, "Masaki Kanno" <kanno.masaki@jp.fujitsu.com> wrote:
>>
>>> Hi Jan,
>>> 
>>> Thu, 14 Jan 2010 09:05:32 +0000, "Jan Beulich" wrote:
>>> 
>>>> You forgot to tell us in what way the boot fails.
>>> 
>>> Oops!  Could you refer to the following about a serial log when the
>>> problem occurred?
>>> 
>>> http://lists.xensource.com/archives/html/xen-devel/2010-01/msg00338.html
>>> 
>>> Accurately, the problem is not "Domain-0 kernel boot fails" but "Domain-
>>> 0 kernel boot does not start".
>>
>>I don't know how much experience with kernel issues you have, but I would
>>take the EIP from the CPU0 guest dump in that log file (the EIP is c01edc67)
>>and look at that code address in a disassembly of the dom0 kernel image
>>(objdump -d vmlinux). Seeing what function you're in would be a good start.
>>'addr2line -e vmlinux c01edc67' would also be useful: tells you the source
>>file and line number.
>>

I was not able to have enough working times to investigate the 
problem today.  The problem still occurs with xen-4.0.0-rc2-pre.
Attached the serial log.  EIP:c01ed6c7 is the following function.

c01ed6b0 <read_current_timer>:
c01ed6b0:       81 3d 54 1f 37 c0 80    cmpl   $0xc01ed680,0xc0371f54
c01ed6b7:       d6 1e c0 
c01ed6ba:       89 c1                   mov    %eax,%ecx
c01ed6bc:       b8 ff ff ff ff          mov    $0xffffffff,%eax
c01ed6c1:       74 02                   je     c01ed6c5 <read_current_timer+0x15>
c01ed6c3:       f3 c3                   repz ret 
c01ed6c5:       0f 31                   rdtsc  
c01ed6c7:       89 01                   mov    %eax,(%ecx)
c01ed6c9:       31 c0                   xor    %eax,%eax
c01ed6cb:       c3                      ret    
c01ed6cc:       8d 74 26 00             lea    0x0(%esi),%esi

Best regards,
 Kan


[-- Attachment #2: COM1_20100115.log --]
[-- Type: application/octet-stream, Size: 12315 bytes --]

 __  __            _  _    ___   ___              ____                    
 \ \/ /___ _ __   | || |  / _ \ / _ \    _ __ ___|___ \    _ __  _ __ ___ 
  \  // _ \ '_ \  | || |_| | | | | | |__| '__/ __| __) |__| '_ \| '__/ _ \
  /  \  __/ | | | |__   _| |_| | |_| |__| | | (__ / __/|__| |_) | | |  __/
 /_/\_\___|_| |_|    |_|(_)___(_)___/   |_|  \___|_____|  | .__/|_|  \___|
                                                          |_|             
(XEN) Xen version 4.0.0-rc2-pre (root@sky.yk.fujitsu.co.jp) (gcc version 4.1.2 20070626 (Red Hat 4.1.2-14)) Fri Jan 15 10:06:25 JST 2010
(XEN) Latest ChangeSet: Thu Jan 14 14:11:25 2010 +0000 20808:db8a882f5515
(XEN) Console output is synchronous.
(XEN) Command line: loglvl=all guest_loglvl=all sync_console console_to_ring com1=9600,8n1 console=com1
(XEN) Video information:
(XEN)  VGA is text mode 80x25, font 8x16
(XEN)  VBE/DDC methods: V2; EDID transfer time: 2 seconds
(XEN) Disc information:
(XEN)  Found 1 MBR signatures
(XEN)  Found 1 EDD information structures
(XEN) Xen-e820 RAM map:
(XEN)  0000000000000000 - 000000000009ec00 (usable)
(XEN)  000000000009ec00 - 00000000000a0000 (reserved)
(XEN)  00000000000e0000 - 0000000000100000 (reserved)
(XEN)  0000000000100000 - 000000003fef0000 (usable)
(XEN)  000000003fef0000 - 000000003feff000 (reserved)
(XEN)  000000003feff000 - 000000003ff00000 (ACPI NVS)
(XEN)  000000003ff00000 - 0000000040000000 (usable)
(XEN)  00000000fec00000 - 00000000fec10000 (reserved)
(XEN)  00000000fee00000 - 00000000fee01000 (reserved)
(XEN)  00000000fff80000 - 0000000100000000 (reserved)
(XEN) System RAM: 1002MB (1026604kB)
(XEN) ACPI: RSDP 000F8010, 0014 (r0 PTLTD )
(XEN) ACPI: RSDT 3FEFD053, 0034 (r1 PTLTD    RSDT    13E0000  LTP        0)
(XEN) ACPI: FACP 3FEFEEA0, 0074 (r1 QUANTA S31       13E0000 MSFT     1000)
(XEN) ACPI: DSDT 3FEFD087, 1E19 (r1 Quanta      S31  13E0000 MSFT  100000D)
(XEN) ACPI: FACS 3FEFFFC0, 0040
(XEN) ACPI: SPCR 3FEFEF14, 0050 (r1 PTLTD  $UCRTBL$  13E0000 PTL         1)
(XEN) ACPI: APIC 3FEFEF64, 0074 (r1 PTLTD  	 APIC    13E0000  LTP        0)
(XEN) ACPI: BOOT 3FEFEFD8, 0028 (r1 PTLTD  $SBFTBL$  13E0000  LTP        1)
(XEN) No NUMA configuration found
(XEN) Faking a node at 0000000000000000-0000000040000000
(XEN) Xen heap: 9MB (9320kB)
(XEN) Domain heap initialised
(XEN) found SMP MP-table at 000f8080
(XEN) DMI present.
(XEN) Using APIC driver default
(XEN) ACPI: PM-Timer IO Port: 0x508
(XEN) ACPI: ACPI SLEEP INFO: pm1x_cnt[544,504], pm1x_evt[540,500]
(XEN) ACPI:                  wakeup_vec[3fefffcc], vec_size[20]
(XEN) ACPI: Local APIC address 0xfee00000
(XEN) ACPI: LAPIC (acpi_id[0x00] lapic_id[0x03] enabled)
(XEN) Processor #3 6:11 APIC version 17
(XEN) ACPI: LAPIC (acpi_id[0x01] lapic_id[0x00] enabled)
(XEN) Processor #0 6:11 APIC version 17
(XEN) ACPI: LAPIC_NMI (acpi_id[0x00] high edge lint[0x1])
(XEN) ACPI: LAPIC_NMI (acpi_id[0x01] high edge lint[0x1])
(XEN) ACPI: IOAPIC (id[0x01] address[0xfec00000] gsi_base[0])
(XEN) IOAPIC[0]: apic_id 1, version 17, address 0xfec00000, GSI 0-15
(XEN) ACPI: IOAPIC (id[0x02] address[0xfec01000] gsi_base[16])
(XEN) IOAPIC[1]: apic_id 2, version 17, address 0xfec01000, GSI 16-31
(XEN) ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 high edge)
(XEN) ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 low level)
(XEN) ACPI: IRQ0 used by override.
(XEN) ACPI: IRQ2 used by override.
(XEN) ACPI: IRQ9 used by override.
(XEN) Enabling APIC mode:  Flat.  Using 2 I/O APICs
(XEN) Using ACPI (MADT) for SMP configuration information
(XEN) Using scheduler: SMP Credit Scheduler (credit)
(XEN) Initializing CPU#0
(XEN) Detected 997.483 MHz processor.
(XEN) Initing memory sharing.
(XEN) CPU: L1 I cache: 16K, L1 D cache: 16K
(XEN) CPU: L2 cache: 512K
(XEN) Intel machine check reporting enabled on CPU#0.
(XEN) CMCI: CPU0 has no CMCI support
(XEN) I/O virtualisation disabled
(XEN) CPU0: Intel(R) Pentium(R) III CPU - S         1000MHz stepping 04
(XEN) Booting processor 1/0 eip 8c000
(XEN) Initializing CPU#1
(XEN) CPU: L1 I cache: 16K, L1 D cache: 16K
(XEN) CPU: L2 cache: 512K
(XEN) Intel machine check reporting enabled on CPU#1.
(XEN) CMCI: CPU1 has no CMCI support
(XEN) CPU1: Intel(R) Pentium(R) III CPU - S         1000MHz stepping 04
(XEN) Total of 2 processors activated.
(XEN) ENABLING IO-APIC IRQs
(XEN)  -> Using new ACK method
(XEN) ..TIMER: vector=0xF0 apic1=0 pin1=2 apic2=-1 pin2=-1
(XEN) ..MP-BIOS bug: 8254 timer not connected to IO-APIC
(XEN) ...trying to set up timer (IRQ0) through the 8259A ...  failed.
(XEN) ...trying to set up timer as Virtual Wire IRQ... works.
(XEN) checking TSC synchronization across 2 CPUs: passed.
(XEN) Platform timer is 3.579MHz ACPI PM Timer
(XEN) Brought up 2 CPUs
(XEN) microcode.c:73:d32767 microcode: CPU1 resumed
(XEN) CPUIDLE: disabled due to no HPET. Force enable with 'cpuidle'.
(XEN) ACPI sleep modes: S3
(XEN) mcheck_poll: Machine check polling timer started.
(XEN) *** LOADING DOMAIN 0 ***
(XEN) elf_parse_binary: phdr: paddr=0xc0100000 memsz=0x261424
(XEN) elf_parse_binary: phdr: paddr=0xc0362000 memsz=0x12943c
(XEN) elf_parse_binary: memory: 0xc0100000 -> 0xc048b43c
(XEN) elf_xen_parse_note: GUEST_OS = "linux"
(XEN) elf_xen_parse_note: GUEST_VERSION = "2.6"
(XEN) elf_xen_parse_note: XEN_VERSION = "xen-3.0"
(XEN) elf_xen_parse_note: VIRT_BASE = 0xc0000000
(XEN) elf_xen_parse_note: PADDR_OFFSET = 0xc0000000
(XEN) elf_xen_parse_note: ENT = 0xc0100000
(XEN) elf_xen_parse_note: HYPERCALL_PAGE = 0xc0101000
(XEN) elf_xen_parse_note: HV_START_LOW = 0xf5800000
(XEN) elf_xen_parse_note: FEATURES = "writable_page_tables|writable_descriptor_tables|auto_translated_physmap|pae_pgdir_above_4gb|supervisor_mode_kernel"
(XEN) elf_xen_parse_note: PAE_MODE = "yes"
(XEN) elf_xen_parse_note: unknown xen elf note (0xd)
(XEN) elf_xen_parse_note: LOADER = "generic"
(XEN) elf_xen_parse_note: SUSPEND_CANCEL = 0x1
(XEN) elf_xen_addr_calc_check: addresses:
(XEN)     virt_base        = 0xc0000000
(XEN)     elf_paddr_offset = 0xc0000000
(XEN)     virt_offset      = 0x0
(XEN)     virt_kstart      = 0xc0100000
(XEN)     virt_kend        = 0xc048b43c
(XEN)     virt_entry       = 0xc0100000
(XEN)     p2m_base         = 0xffffffffffffffff
(XEN)  Xen  kernel: 32-bit, PAE, lsb
(XEN)  Dom0 kernel: 32-bit, PAE, lsb, paddr 0xc0100000 -> 0xc048b43c
(XEN) PHYSICAL MEMORY ARRANGEMENT:
(XEN)  Dom0 alloc.:   000000003d000000->000000003e000000 (236849 pages to be allocated)
(XEN) VIRTUAL MEMORY ARRANGEMENT:
(XEN)  Loaded kernel: c0100000->c048b43c
(XEN)  Init. ramdisk: c048c000->c09d2800
(XEN)  Phys-Mach map: c09d3000->c0abe4c4
(XEN)  Start info:    c0abf000->c0abf47c
(XEN)  Page tables:   c0ac0000->c0acb000
(XEN)  Boot stack:    c0acb000->c0acc000
(XEN)  TOTAL:         c0000000->c0c00000
(XEN)  ENTRY ADDRESS: c0100000
(XEN) Dom0 has maximum 2 VCPUs
(XEN) elf_load_binary: phdr 0 at 0xc0100000 -> 0xc0361424
(XEN) elf_load_binary: phdr 1 at 0xc0362000 -> 0xc041d9e4
(XEN) Scrubbing Free RAM: done.
(XEN) Xen trace buffers: disabled
(XEN) Std. Loglevel: All
(XEN) Guest Loglevel: All
(XEN) **********************************************
(XEN) ******* WARNING: CONSOLE OUTPUT IS SYNCHRONOUS
(XEN) ******* This option is intended to aid debugging of Xen by ensuring
(XEN) ******* that all output is synchronously delivered on the serial line.
(XEN) ******* However it can introduce SIGNIFICANT latencies and affect
(XEN) ******* timekeeping. It is NOT recommended for production use!
(XEN) **********************************************
(XEN) 3... 2... 1... 
(XEN) *** Serial input -> DOM0 (type 'CTRL-a' three times to switch input to Xen)
(XEN) Freed 132kB init memory.
(XEN) *** Serial input -> Xen (type 'CTRL-a' three times to switch input to DOM0)
(XEN) 'd' pressed -> dumping registers
(XEN) 
(XEN) *** Dumping CPU0 host state: ***
(XEN) ----[ Xen-4.0.0-rc2-pre  x86_32p  debug=y  Tainted:    C ]----
(XEN) CPU:    0
(XEN) EIP:    e008:[<ff10e17d>] __dump_execstate+0x7/0x61
(XEN) EFLAGS: 00010282   CONTEXT: hypervisor
(XEN) eax: 00000000   ebx: 00000064   ecx: ff20b748   edx: ff20b748
(XEN) esi: 00000064   edi: ff20b7d8   ebp: ff2b3e88   esp: ff2b3e70
(XEN) cr0: 8005003b   cr4: 000006f0   cr3: 3dac0000   cr2: f5505000
(XEN) ds: e010   es: e010   fs: 0000   gs: 0000   ss: e010   cs: e008
(XEN) Xen stack trace from esp=ff2b3e70:
(XEN)    00000064 ff20b7d8 ff2c3142 ff2b3e94 00000064 00000064 ff2b3ea8 ff10ead0
(XEN)    00000000 00000000 ff2b3ea8 ff12d91d ff20b334 00000064 ff2b3ec8 ff10e253
(XEN)    00000064 ff2b3fb4 ff2b3ef8 00000000 ff2b3fb4 00000064 ff2b3ed8 ff12daee
(XEN)    00000064 ff2b3fb4 ff2b3ee8 ff12dbb4 ff12db2f ff20b780 ff2b3f18 ff12eca1
(XEN)    00000064 ff2b3fb4 00000282 ff2b3f1c ff13fd58 ff20b7e0 64000286 00000061
(XEN)    ff2c75e0 ff20b780 ff2b3f38 ff12e3b1 ff20b780 ff2b3fb4 ffb8029c 00000000
(XEN)    ff2b3fb4 00000004 ff2b3fa8 ff14cfe5 00000004 ff20b780 ff2b3fb4 ff2e7e44
(XEN)    ff2e7e40 ff230100 00000000 35c347b5 00000004 ff2c7600 0003db39 ffb80280
(XEN)    00000000 ff2b3fb4 00000000 ff2b3f9c ff11b190 00000000 ff2b3fac 00d4c037
(XEN)    ffb7a000 0000e021 0000e021 ffff8ad1 0000e021 0000e021 00d4c037 ff145f56
(XEN)    ff2b3fb4 ffff8ad1 c03e7fb4 00000005 ffff8ad0 c17802e4 c03e7fb4 497c9019
(XEN)    00f10000 c01ed6c7 0000e019 00000246 c03e7f88 0000e021 0000e021 0000e021
(XEN)    00000000 00000000 00000000 ffb7a000
(XEN) Xen call trace:
(XEN)    [<ff10e17d>] __dump_execstate+0x7/0x61
(XEN)    [<ff10ead0>] dump_registers+0x54/0x124
(XEN)    [<ff10e253>] handle_keypress+0x50/0x73
(XEN)    [<ff12daee>] __serial_rx+0x20/0x61
(XEN)    [<ff12dbb4>] serial_rx+0x85/0x89
(XEN)    [<ff12eca1>] serial_rx_interrupt+0xa4/0xb1
(XEN)    [<ff12e3b1>] ns16550_interrupt+0x48/0x60
(XEN)    [<ff14cfe5>] do_IRQ+0x5a4/0x64e
(XEN)    [<ff145f56>] common_interrupt+0x56/0x60
(XEN)    
(XEN) *** Dumping CPU0 guest state: ***
(XEN) ----[ Xen-4.0.0-rc2-pre  x86_32p  debug=y  Tainted:    C ]----
(XEN) CPU:    0
(XEN) EIP:    e019:[<c01ed6c7>]
(XEN) EFLAGS: 00000246   EM: 0   CONTEXT: pv guest
(XEN) eax: 497c9019   ebx: ffff8ad1   ecx: c03e7fb4   edx: 00000005
(XEN) esi: ffff8ad0   edi: c17802e4   ebp: c03e7fb4   esp: c03e7f88
(XEN) cr0: 8005003b   cr4: 000006f0   cr3: 3dac0000   cr2: f5505000
(XEN) ds: e021   es: e021   fs: 0000   gs: 0000   ss: e021   cs: e019
(XEN) Guest stack trace from esp=c03e7f88:
(XEN)    c0102986 c0177076 00000008 00000006 00000000 00000000 00000000 00000000
(XEN)    c1774b6c c17802e4 000038e4 497c8ff0 497c8ff0 c177ca00 00000001 c17802e4
(XEN)    000038e4 c03ec6ea 00000035 c03ec220 00000000 00000020 00008000 0136a900
(XEN)    c0421aa0 00000004 c0abf000 00000000 00000000 00000000
(XEN) 
(XEN) *** Dumping CPU1 host state: ***
(XEN) ----[ Xen-4.0.0-rc2-pre  x86_32p  debug=y  Tainted:    C ]----
(XEN) CPU:    1
(XEN) EIP:    e008:[<ff10e17d>] __dump_execstate+0x7/0x61
(XEN) EFLAGS: 00010086   CONTEXT: hypervisor
(XEN) eax: 00000080   ebx: ff10e176   ecx: 00000000   edx: ffbebfb4
(XEN) esi: 0000e010   edi: 0000e010   ebp: ffbebf48   esp: ffbebf30
(XEN) cr0: 8005003b   cr4: 000006f0   cr3: 00100120   cr2: 00000000
(XEN) ds: e010   es: e010   fs: e010   gs: e010   ss: e010   cs: e008
(XEN) Xen stack trace from esp=ffbebf30:
(XEN)    00000000 ff2e7e88 ff20b75c ff232100 ff20b408 ff10e176 ffbebf58 ff15dbee
(XEN)    00000000 00000000 ffbebf68 ff15e1cf 0000e010 ffbebfb4 00414077 ff13f02b
(XEN)    ffbebf74 ffbebfb4 00000000 00000080 00000001 00000000 ffbebfa0 00000080
(XEN)    00fb0000 ff142b43 0000e008 00000246 ffbebfb0 ff142b18 00000000 ffbec000
(XEN)    ffbebf60 940f01c2 04c083c2 84104189 e8850fd2 83fffffe e901f045 fffffdf6
(XEN)    042444c7 0000002e 8908558b a5e82414 830000b1 850f01c0 fffffd56 fffebee9
(XEN)    2444c7ff 00002704 00000001 ffbec000
(XEN) Xen call trace:
(XEN)    [<ff10e17d>] __dump_execstate+0x7/0x61
(XEN)    [<ff15dbee>] __smp_call_function_interrupt+0x4d/0x8b
(XEN)    [<ff15e1cf>] smp_call_function_interrupt+0x52/0x73
(XEN)    [<ff13f02b>] call_function_interrupt+0x5b/0x70
(XEN)    [<ff142b43>] default_idle+0x21/0x26
(XEN)    [<ff142b18>] idle_loop+0x61/0x6b
(XEN)    
(XEN) *** Dumping CPU1 guest state: ***
(XEN) No guest context (CPU is idle).
(XEN) 

[-- Attachment #3: Type: text/plain, Size: 138 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Re: Domain-0 kernel boot failure withxen-unstableandlinux-2.6.18-xen

[Keir Fraser]

[-- Attachment #1: Type: text/plain, Size: 623 bytes --]

On 14/01/2010 06:18, "Masaki Kanno" <kanno.masaki@jp.fujitsu.com> wrote:

>    cs:20705       cs:967             failure
>>  cs:20710       cs:967             failure
>>> cs:20714       cs:969             failure
>>> cs:20729       cs:970             failure
>>>> cs:20744       cs:970             failure
>>>> cs:20779       cs:981             failure
> 
> The problem occurs by xen-unstable changeset 20705.
> I can pinpoint the changeset causing the problem, but I cannot find out
> a bug in the changeset...

Masaki,

Please try the attached patch. We can put it in 4.0.0-rc2 if it works well
for you.

 Thanks,
 Keir


[-- Attachment #2: 00-tsc --]
[-- Type: application/octet-stream, Size: 3819 bytes --]

diff -r d7e8b6a66a3d xen/arch/x86/cpu/amd.c
--- a/xen/arch/x86/cpu/amd.c	Mon Jan 18 14:49:00 2010 +0000
+++ b/xen/arch/x86/cpu/amd.c	Mon Jan 18 15:16:36 2010 +0000
@@ -465,6 +465,8 @@
 		if (c->x86_power & (1<<8)) {
 			set_bit(X86_FEATURE_CONSTANT_TSC, c->x86_capability);
 			set_bit(X86_FEATURE_NONSTOP_TSC, c->x86_capability);
+			if (c->x86 != 0x11)
+				set_bit(X86_FEATURE_TSC_RELIABLE, c->x86_capability);
 		}
 	}
 
diff -r d7e8b6a66a3d xen/arch/x86/cpu/intel.c
--- a/xen/arch/x86/cpu/intel.c	Mon Jan 18 14:49:00 2010 +0000
+++ b/xen/arch/x86/cpu/intel.c	Mon Jan 18 15:16:36 2010 +0000
@@ -212,6 +212,7 @@
 	if (cpuid_edx(0x80000007) & (1u<<8)) {
 		set_bit(X86_FEATURE_CONSTANT_TSC, c->x86_capability);
 		set_bit(X86_FEATURE_NONSTOP_TSC, c->x86_capability);
+		set_bit(X86_FEATURE_TSC_RELIABLE, c->x86_capability);
 	}
 	if ((c->cpuid_level >= 0x00000006) &&
 	    (cpuid_eax(0x00000006) & (1u<<2)))
diff -r d7e8b6a66a3d xen/arch/x86/time.c
--- a/xen/arch/x86/time.c	Mon Jan 18 14:49:00 2010 +0000
+++ b/xen/arch/x86/time.c	Mon Jan 18 15:16:36 2010 +0000
@@ -1372,7 +1372,18 @@
 /* Late init function (after all CPUs are booted). */
 int __init init_xen_time(void)
 {
-    extern unsigned int max_cstate;
+    if ( boot_cpu_has(X86_FEATURE_TSC_RELIABLE) )
+    {
+        /*
+         * Sadly, despite processor vendors' best design guidance efforts, on
+         * some systems, cpus may come out of reset improperly synchronized.
+         * So we must verify there is no warp and we can't do that until all
+         * CPUs are booted.
+         */
+        tsc_check_reliability();
+        if ( tsc_max_warp )
+            clear_bit(X86_FEATURE_TSC_RELIABLE, boot_cpu_data.x86_capability);
+    }
 
     /* If we have constant-rate TSCs then scale factor can be shared. */
     if ( boot_cpu_has(X86_FEATURE_CONSTANT_TSC) )
@@ -1380,22 +1391,10 @@
         int cpu;
         for_each_possible_cpu ( cpu )
             per_cpu(cpu_time, cpu).tsc_scale = per_cpu(cpu_time, 0).tsc_scale;
+        /* If TSCs are not marked as 'reliable', re-sync during rendezvous. */
+        if ( !boot_cpu_has(X86_FEATURE_TSC_RELIABLE) )
+            time_calibration_rendezvous_fn = time_calibration_tsc_rendezvous;
     }
-    if ( (boot_cpu_has(X86_FEATURE_CONSTANT_TSC) && max_cstate <= 2) ||
-         boot_cpu_has(X86_FEATURE_NONSTOP_TSC) )
-    {
-        /*
-         * Sadly, despite processor vendors' best design guidance efforts,
-         * on some systems, cpus may come out of reset improperly
-         * synchronized.  So we must verify there is no warp and we
-         * can't do that until all CPUs are booted
-         */
-        tsc_check_reliability();
-        if ( tsc_max_warp == 0 )
-            set_boot_cpu_bit(X86_FEATURE_TSC_RELIABLE);
-    }
-    if ( !boot_cpu_has(X86_FEATURE_TSC_RELIABLE) )
-            time_calibration_rendezvous_fn = time_calibration_tsc_rendezvous;
 
     open_softirq(TIME_CALIBRATE_SOFTIRQ, local_time_calibration);
 
@@ -1630,11 +1629,7 @@
 
 int host_tsc_is_safe(void)
 {
-    if ( boot_cpu_has(X86_FEATURE_TSC_RELIABLE) )
-        return 1;
-    if ( num_online_cpus() == 1 )
-        return 1;
-    return 0;
+    return boot_cpu_has(X86_FEATURE_TSC_RELIABLE) || (num_online_cpus() == 1);
 }
 
 void cpuid_time_leaf(uint32_t sub_idx, uint32_t *eax, uint32_t *ebx,
diff -r d7e8b6a66a3d xen/include/asm-x86/cpufeature.h
--- a/xen/include/asm-x86/cpufeature.h	Mon Jan 18 14:49:00 2010 +0000
+++ b/xen/include/asm-x86/cpufeature.h	Mon Jan 18 15:16:36 2010 +0000
@@ -132,7 +132,6 @@
 
 #define cpu_has(c, bit)		test_bit(bit, (c)->x86_capability)
 #define boot_cpu_has(bit)	test_bit(bit, boot_cpu_data.x86_capability)
-#define set_boot_cpu_bit(bit)	set_bit(bit, boot_cpu_data.x86_capability)
 
 #ifdef __i386__
 #define cpu_has_vme		boot_cpu_has(X86_FEATURE_VME)

[-- Attachment #3: Type: text/plain, Size: 138 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Re: Domain-0 kernel boot failurewithxen-unstableandlinux-2.6.18-xen

[Masaki Kanno]

Hi Keir,

Great!!  Domain-0 kernel is running on my machine by the patch.

Best regards,
 Kan

Mon, 18 Jan 2010 15:18:08 +0000, Keir Fraser wrote:

>On 14/01/2010 06:18, "Masaki Kanno" <kanno.masaki@jp.fujitsu.com> wrote:
>
>>    cs:20705       cs:967             failure
>>>  cs:20710       cs:967             failure
>>>> cs:20714       cs:969             failure
>>>> cs:20729       cs:970             failure
>>>>> cs:20744       cs:970             failure
>>>>> cs:20779       cs:981             failure
>> 
>> The problem occurs by xen-unstable changeset 20705.
>> I can pinpoint the changeset causing the problem, but I cannot find out
>> a bug in the changeset...
>
>Masaki,
>
>Please try the attached patch. We can put it in 4.0.0-rc2 if it works well
>for you.
>
> Thanks,
> Keir
>
>
>-------------------------------text/plain-------------------------------
>_______________________________________________
>Xen-devel mailing list
>Xen-devel@lists.xensource.com
>http://lists.xensource.com/xen-devel