Oops

Oops

[Gerd Hoffmann]

[-- Attachment #1: Type: text/plain, Size: 125 bytes --]

  Hi,

paravirt kernel doesn't boot as xen guest for me, see attachment.

cheers,
  Gerd

-- 
Gerd Hoffmann <kraxel@suse.de>

[-- Attachment #2: boot.log --]
[-- Type: text/x-log, Size: 5717 bytes --]

Reserving virtual address space above 0xf57fe000
CPU: Vendor unknown, using generic init.
CPU: Your system may be unstable.
Linux version 2.6.20-rc4-pae-hg689 (kraxel@master-xen) (gcc version 4.1.2 20061115 (prerelease) (SUSE Linux)) #7 Tue Jan 9 14:22:16 CET 2007
BIOS-provided physical RAM map:
 Xen: 0000000000000000 - 0000000002000000 (usable)
0MB HIGHMEM available.
32MB LOWMEM available.
NX (Execute Disable) protection: active
about to switch to new pagetable c039e000...
done
Zone PFN ranges:
  DMA             0 ->     4096
  Normal       4096 ->     8192
  HighMem      8192 ->     8192
early_node_map[1] active PFN ranges
    0:        0 ->     8192
DMI not present or invalid.
Allocating PCI resources starting at 10000000 (gap: 02000000:fe000000)
Detected 3793.006 MHz processor.
Built 1 zonelists.  Total pages: 8128
Kernel command line: root=/dev/ram0 ro 
Local APIC disabled by BIOS -- you can enable it with "lapic"
Enabling fast FPU save and restore... done.
Enabling unmasked SIMD FPU exception support... done.
Initializing CPU#0
PID hash table entries: 128 (order: 7, 512 bytes)
Xen reported: 3792.882 MHz processor.
Dentry cache hash table entries: 4096 (order: 2, 16384 bytes)
Inode-cache hash table entries: 2048 (order: 1, 8192 bytes)
Memory: 25172k/32768k available (1741k kernel code, 7596k reserved, 754k data, 172k init, 0k highmem)
virtual kernel memory layout:
    fixmap  : 0xf57ac000 - 0xf57fd000   ( 324 kB)
    pkmap   : 0xf5400000 - 0xf5600000   (2048 kB)
    vmalloc : 0xc2800000 - 0xf53fe000   ( 811 MB)
    lowmem  : 0xc0000000 - 0xc2000000   (  32 MB)
      .init : 0xc0372000 - 0xc039d000   ( 172 kB)
      .data : 0xc02b3637 - 0xc036feb0   ( 754 kB)
      .text : 0xc0100000 - 0xc02b3637   (1741 kB)
Checking if this processor honours the WP bit even in supervisor mode... Ok.
Calibrating delay using timer specific routine.. 7589.84 BogoMIPS (lpj=37949208)
Mount-cache hash table entries: 512
monitor/mwait feature present.
CPU: Trace cache: 12K uops, L1 D cache: 16K
CPU: L2 cache: 2048K
Intel machine check architecture supported.
Intel machine check reporting enabled on CPU#0.
CPU0: Intel P4/Xeon Extended MCE MSRs (24) available
CPU0: Thermal monitoring enabled
CPU: Intel Genuine Intel(R) CPU 3.80GHz stepping 06
Booting paravirtualized kernel on Xen
Hypervisor signature: xen-3.0-x86_32p
Grant table initialized
NET: Registered protocol family 16
Setting up standard PCI resources
NET: Registered protocol family 2
IP route cache hash table entries: 1024 (order: 0, 4096 bytes)
TCP established hash table entries: 1024 (order: 0, 4096 bytes)
TCP bind hash table entries: 512 (order: -1, 2048 bytes)
TCP: Hash tables configured (established 1024 bind 512)
TCP reno registered
checking if image is initramfs...it isn't (bad gzip magic numbers); looks like an initrd
Freeing initrd memory: 4096k freed
Machine check exception polling timer started.
IA-32 Microcode Update Driver: v1.14a <tigran@aivazian.fsnet.co.uk>
Total HugeTLB memory allocated, 0
Installing knfsd (copyright (C) 1996 okir@monad.swb.de).
io scheduler noop registered
io scheduler anticipatory registered (default)
io scheduler deadline registered
io scheduler cfq registered
rtc: IRQ 8 is not free.
Hangcheck: starting hangcheck timer 0.9.0 (tick is 180 seconds, margin is 60 seconds).
Hangcheck: Using get_cycles().
Serial: 8250/16550 driver $Revision: 1.90 $ 4 ports, IRQ sharing disabled
RAMDISK driver initialized: 16 RAM disks of 4096K size 1024 blocksize
loop: loaded (max 8 devices)
Registering block device major 202
netconsole: not configured, aborting
Xen virtual console successfully installed as tty1
netfront: Initialising virtual ethernet driver.
i8042.c: No controller found.
mice: PS/2 mouse device common for all mice
device-mapper: ioctl: 4.11.0-ioctl (2006-10-12) initialised: dm-devel@redhat.com
oprofile: using timer interrupt.
TCP cubic registered
NET: Registered protocol family 1
NET: Registered protocol family 10
IPv6 over IPv4 tunneling driver
NET: Registered protocol family 17
Using IPI Shortcut mode
Time: xen clocksource has been installed.
blkfront: xvda: barriers enabled
 xvda: xvda1 xvda2
RAMDISK: ext2 filesystem found at block 0
RAMDISK: Loading 4096KiB [1 disk] into ram disk... done.
VFS: Mounted root (ext2 filesystem) readonly.
Freeing unused kernel memory: 172k freed
BUG: unable to handle kernel paging request at virtual address c2ffffe0
 printing eip:
c0102545
*pde = 00000000
Oops: 0000 [#1]
Modules linked in:
CPU:    0
EIP:    0061:[<c0102545>]    Not tainted VLI
EFLAGS: 00010202   (2.6.20-rc4-pae-hg689 #7)
EIP is at pgd_walk+0x7d/0x139
eax: 01ffffe0   ebx: c1915d60   ecx: fffff0a1   edx: 00000000
esi: c1000000   edi: c1911000   ebp: c1911000   esp: c1067ee0
ds: 007b   es: 007b   ss: 0069
Process init (pid: 1, ti=c1066000 task=c1064a70 task.ti=c1066000)
Stack: 00000061 80000000 01915000 00000000 c1821570 00000009 c1911000 00000000 
       00000000 c1b3b030 c010261c 00000003 c1230f3c c12305f4 c1230f3c 00000000 
       c0116755 c1230f00 c0112031 c0149219 c1067fb8 bf8ab844 00000011 00000000 
Call Trace:
 [<c010261c>] xen_pgd_pin+0x1b/0x67
 [<c0116755>] copy_process+0x96e/0xe50
 [<c0112031>] paravirt_restore_flags+0x6/0x7
 [<c0149219>] kmem_cache_alloc+0x50/0x5c
 [<c0116e3b>] do_fork+0x9e/0x17d
 [<c01fd699>] copy_to_user+0x2d/0x44
 [<c0102c0d>] sys_fork+0x2c/0x30
 [<c01045d8>] syscall_call+0x7/0xb
 =======================
Code: 8b 03 8b 53 04 ff 15 cc f0 34 c0 85 c0 74 5e 8b 35 c0 d6 3e c0 85 f6 74 33 8b 03 8b 53 04 ff 15 cc f0 34 c0 0f ac d0 0c c1 e0 05 <8b> 04 30 c1 e8 1e 69 c0 44 01 00 00 05 e0 0b 35 c0 8b 90 30 01 
EIP: [<c0102545>] pgd_walk+0x7d/0x139 SS:ESP 0069:c1067ee0
 <0>Kernel panic - not syncing: Attempted to kill init!

[-- Attachment #3: Type: text/plain, Size: 165 bytes --]

_______________________________________________
Virtualization mailing list
Virtualization@lists.osdl.org
https://lists.osdl.org/mailman/listinfo/virtualization

Re: Oops

[Jeremy Fitzhardinge]

Gerd Hoffmann wrote:
>   Hi,
>
> paravirt kernel doesn't boot as xen guest for me, see attachment.
>   

Hm, I just booted a pae kernel from the current paravirt repo and got it
to usermode.  What's your .config?  Also, what line is getting the fault?

    J

Re: Oops

[Gerd Hoffmann]

[-- Attachment #1: Type: text/plain, Size: 1475 bytes --]

Jeremy Fitzhardinge wrote:
> Gerd Hoffmann wrote:
>>   Hi,
>>
>> paravirt kernel doesn't boot as xen guest for me, see attachment.
>>   
> 
> Hm, I just booted a pae kernel from the current paravirt repo and got it
> to usermode.  What's your .config?  Also, what line is getting the fault?

config is attached.

It faults in fork syscall, down in pgd_walk().  It finds a pmd entry
pointing to a machine page where the mfn_to_pfn translation returns -1
aka 0xffffffff.  Trying to find a struct page for that one doesn't work ...

kernel with debug printk's:

xen_pgd_pin c1abb000
   g     0:  1abe001 1ae61001                <- pgd (index, val, val_ma)
     m   0:   80b067 1a394067                <- pmd (same)
     m   1:   80c067 1a393067
     m   2:   80d067 1a392067
     m   3:   80e067 1a391067
     m   4:   80f067 1a390067
     m   5:     2067 1b621067
     m   6:     3067 1b620067
     m   7:     4067  dbf7067
     m   8:     5067  dbf6067
     m   9:     6067  dbf5067
     m  10:     7067  dbf4067
     m  11:     8067 19c57067
     m  12:     9067 19c56067
     m  13:     a067 19c55067
     m  14:     b067 19c54067
     m  15:     c067 1a57f067
     m  20:  107e067 19d21067
     m  64:  1808067 1d297067
     m 426:     e067 1a57d067
     m 427:     d067 1a57e067
mfn_to_pfn: mfn 3f400 pfn ffffffff
mfn_to_pfn: mfn 3f400 pfn ffffffff
     m 428: fffff0a1 3f4000a1                <- here is the broken one

cheers,
  Gerd

-- 
Gerd Hoffmann <kraxel@suse.de>

[-- Attachment #2: pae.config --]
[-- Type: application/x-config, Size: 20112 bytes --]

[-- Attachment #3: Type: text/plain, Size: 165 bytes --]

_______________________________________________
Virtualization mailing list
Virtualization@lists.osdl.org
https://lists.osdl.org/mailman/listinfo/virtualization

Re: Oops

[Gerd Hoffmann]

Gerd Hoffmann wrote:
> Jeremy Fitzhardinge wrote:
>> Gerd Hoffmann wrote:
>>>   Hi,
>>>
>>> paravirt kernel doesn't boot as xen guest for me, see attachment.
>>>   
>> Hm, I just booted a pae kernel from the current paravirt repo and got it
>> to usermode.  What's your .config?  Also, what line is getting the fault?
> 
> config is attached.
> 
> It faults in fork syscall, down in pgd_walk().  It finds a pmd entry
> pointing to a machine page where the mfn_to_pfn translation returns -1
> aka 0xffffffff.  Trying to find a struct page for that one doesn't work ...

Looks like a slot-3 pmd got reused for slot-0, with some stale entries
for the hypervisor hole in there for some reason ...

quick and dirty sledge hammer fix:

--- paravirt-2.6.20-rc4-hg691.orig/arch/i386/paravirt-xen/enlighten.c
+++ paravirt-2.6.20-rc4-hg691/arch/i386/paravirt-xen/enlighten.c
@@ -522,6 +522,7 @@ static fastcall void xen_alloc_pd(u32 pf
 static fastcall void xen_release_pd(u32 pfn)
 {
        make_lowmem_page_readwrite(__va(PFN_PHYS(pfn)));
+       memset(__va(PFN_PHYS(pfn)), 0, PAGE_SIZE);
 }

 static fastcall void xen_release_pt(u32 pfn)

OK, now I'm back to the old buggy state where the ttylinux rootfs fails
to boot up due to init being killed instantly or something like that ...

cheers,
  Gerd

-- 
Gerd Hoffmann <kraxel@suse.de>

Re: Oops

[Gerd Hoffmann]

Gerd Hoffmann wrote:
> OK, now I'm back to the old buggy state where the ttylinux rootfs fails
> to boot up due to init being killed instantly or something like that ...

That was something completely different.  Machine booted up just fine,
but without any output.  Due to initialization in link order and xen/
coming after serial/ in drivers/ the serial console becomes default when
enabled.  Ouch.  Move it up.

--- paravirt-2.6.20-rc4-hg691.orig/drivers/Makefile
+++ paravirt-2.6.20-rc4-hg691/drivers/Makefile
@@ -18,6 +18,7 @@ obj-$(CONFIG_ARM_AMBA)                += amba/
 # char/ comes before serial/ etc so that the VT console is the boot-time
 # default.
 obj-y                          += char/
+obj-$(CONFIG_XEN)              += xen/

 obj-$(CONFIG_CONNECTOR)                += connector/

@@ -31,7 +32,6 @@ obj-y                         += base/ block/ misc/
mfd/ net/
 obj-$(CONFIG_NUBUS)            += nubus/
 obj-$(CONFIG_ATM)              += atm/
 obj-$(CONFIG_PPC_PMAC)         += macintosh/
-obj-$(CONFIG_XEN)              += xen/
 obj-$(CONFIG_IDE)              += ide/
 obj-$(CONFIG_FC4)              += fc4/
 obj-$(CONFIG_SCSI)             += scsi/

cheers,
  Gerd

-- 
Gerd Hoffmann <kraxel@suse.de>

Re: Oops

[Jeremy Fitzhardinge]

Gerd Hoffmann wrote:
> config is attached.
>
> It faults in fork syscall, down in pgd_walk().  It finds a pmd entry
> pointing to a machine page where the mfn_to_pfn translation returns -1
> aka 0xffffffff.  Trying to find a struct page for that one doesn't work ...
>   

Hm, that's interesting but curious.  Hm, the mapping is in userspace? 
How did userspace get a foreign/special mapping?

Is this the very first fork()?  Nothing should be mapped down there at
all, so maybe there's a bug in initial pagetable setup.

    J

Re: Oops

[Jeremy Fitzhardinge]

Gerd Hoffmann wrote:
> quick and dirty sledge hammer fix:
>
> --- paravirt-2.6.20-rc4-hg691.orig/arch/i386/paravirt-xen/enlighten.c
> +++ paravirt-2.6.20-rc4-hg691/arch/i386/paravirt-xen/enlighten.c
> @@ -522,6 +522,7 @@ static fastcall void xen_alloc_pd(u32 pf
>  static fastcall void xen_release_pd(u32 pfn)
>  {
>         make_lowmem_page_readwrite(__va(PFN_PHYS(pfn)));
> +       memset(__va(PFN_PHYS(pfn)), 0, PAGE_SIZE);
>  }
>
>  static fastcall void xen_release_pt(u32 pfn)
>   
Ah, OK.

    J

Re: Oops

[Jeremy Fitzhardinge]

Gerd Hoffmann wrote:
> Gerd Hoffmann wrote:
>   
>> OK, now I'm back to the old buggy state where the ttylinux rootfs fails
>> to boot up due to init being killed instantly or something like that ...
>>     
>
> That was something completely different.  Machine booted up just fine,
> but without any output.  Due to initialization in link order and xen/
> coming after serial/ in drivers/ the serial console becomes default when
> enabled.  Ouch.  Move it up.
>   

Erk.  I hate these link-order dependencies.

    J

Re: Oops

[Jeremy Fitzhardinge]

Gerd Hoffmann wrote:
> Gerd Hoffmann wrote:
>   
>> Jeremy Fitzhardinge wrote:
>>     
>>> Gerd Hoffmann wrote:
>>>       
>>>>   Hi,
>>>>
>>>> paravirt kernel doesn't boot as xen guest for me, see attachment.
>>>>   
>>>>         
>>> Hm, I just booted a pae kernel from the current paravirt repo and got it
>>> to usermode.  What's your .config?  Also, what line is getting the fault?
>>>       
>> config is attached.
>>
>> It faults in fork syscall, down in pgd_walk().  It finds a pmd entry
>> pointing to a machine page where the mfn_to_pfn translation returns -1
>> aka 0xffffffff.  Trying to find a struct page for that one doesn't work ...
>>     
>
> Looks like a slot-3 pmd got reused for slot-0, with some stale entries
> for the hypervisor hole in there for some reason ...
>
> quick and dirty sledge hammer fix:
>
> --- paravirt-2.6.20-rc4-hg691.orig/arch/i386/paravirt-xen/enlighten.c
> +++ paravirt-2.6.20-rc4-hg691/arch/i386/paravirt-xen/enlighten.c
> @@ -522,6 +522,7 @@ static fastcall void xen_alloc_pd(u32 pf
>  static fastcall void xen_release_pd(u32 pfn)
>  {
>         make_lowmem_page_readwrite(__va(PFN_PHYS(pfn)));
> +       memset(__va(PFN_PHYS(pfn)), 0, PAGE_SIZE);
>  }
>
>  static fastcall void xen_release_pt(u32 pfn)
>   

Does this work?

diff -r 252f3ed87072 arch/i386/mm/pgtable.c
--- a/arch/i386/mm/pgtable.c    Mon Jan 08 16:57:56 2007 -0800
+++ b/arch/i386/mm/pgtable.c    Wed Jan 10 13:13:40 2007 -0800
@@ -265,6 +265,7 @@ static void pgd_ctor(pgd_t *pgd)
                                        swapper_pg_dir + USER_PTRS_PER_PGD,
                                        KERNEL_PGD_PTRS);
                } else {
+                       memset(pgd, 0, USER_PTRS_PER_PGD*sizeof(pgd_t));
                        spin_lock_irqsave(&pgd_lock, flags);
                        pgd_list_add(pgd);
                        spin_unlock_irqrestore(&pgd_lock, flags);

Re: Oops

[Chris Wright]

* Gerd Hoffmann (kraxel@suse.de) wrote:
> Gerd Hoffmann wrote:
> > OK, now I'm back to the old buggy state where the ttylinux rootfs fails
> > to boot up due to init being killed instantly or something like that ...
> 
> That was something completely different.  Machine booted up just fine,
> but without any output.  Due to initialization in link order and xen/
> coming after serial/ in drivers/ the serial console becomes default when
> enabled.  Ouch.  Move it up.

This should not be needed, the console should be xcv0, completely decoupled
from serial.

thanks,
-chris

Re: Oops

[Jeremy Fitzhardinge]

Chris Wright wrote:
> This should not be needed, the console should be xcv0, completely decoupled
> from serial.

Isn't it just a matter of what gets used by default without a
command-line option?  If you compile in serial console, does it try to
do its thing even if there aren't any real serial ports?

    J

Re: Oops

[Gerd Hoffmann]

Chris Wright wrote:
> * Gerd Hoffmann (kraxel@suse.de) wrote:
>> Gerd Hoffmann wrote:
>>> OK, now I'm back to the old buggy state where the ttylinux rootfs fails
>>> to boot up due to init being killed instantly or something like that ...
>> That was something completely different.  Machine booted up just fine,
>> but without any output.  Due to initialization in link order and xen/
>> coming after serial/ in drivers/ the serial console becomes default when
>> enabled.  Ouch.  Move it up.
> 
> This should not be needed, the console should be xcv0, completely decoupled
> from serial.

Wrong.  In't not about whenever the xen console is registered as ttyS0
tty1 or xcv0, but whenever the xen console or the serial console (as in
"8250 driver") is registered first and thus becomes the default console.
 drivers/char/ (with the vt code) is linked before drivers/serial/ for
the very same reason.

cheers,
  Gerd

-- 
Gerd Hoffmann <kraxel@suse.de>

Re: Oops

[Gerd Hoffmann]

> If you compile in serial console, does it try to
> do its thing even if there aren't any real serial ports?

Yes (otherwise the issue wouldn't show up in xen domUs).

cheers,
  Gerd

-- 
Gerd Hoffmann <kraxel@suse.de>

Re: Oops

[Gerd Hoffmann]

Jeremy Fitzhardinge wrote:
>> Looks like a slot-3 pmd got reused for slot-0, with some stale entries
>> for the hypervisor hole in there for some reason ...
>>
>> quick and dirty sledge hammer fix:
>>
>> --- paravirt-2.6.20-rc4-hg691.orig/arch/i386/paravirt-xen/enlighten.c
>> +++ paravirt-2.6.20-rc4-hg691/arch/i386/paravirt-xen/enlighten.c
>> @@ -522,6 +522,7 @@ static fastcall void xen_alloc_pd(u32 pf
>>  static fastcall void xen_release_pd(u32 pfn)
>>  {
>>         make_lowmem_page_readwrite(__va(PFN_PHYS(pfn)));
>> +       memset(__va(PFN_PHYS(pfn)), 0, PAGE_SIZE);
>>  }
>>
>>  static fastcall void xen_release_pt(u32 pfn)
>>   
> 
> Does this work?
> 
> diff -r 252f3ed87072 arch/i386/mm/pgtable.c
> --- a/arch/i386/mm/pgtable.c    Mon Jan 08 16:57:56 2007 -0800
> +++ b/arch/i386/mm/pgtable.c    Wed Jan 10 13:13:40 2007 -0800
> @@ -265,6 +265,7 @@ static void pgd_ctor(pgd_t *pgd)
>                                         swapper_pg_dir + USER_PTRS_PER_PGD,
>                                         KERNEL_PGD_PTRS);
>                 } else {
> +                       memset(pgd, 0, USER_PTRS_PER_PGD*sizeof(pgd_t));
>                         spin_lock_irqsave(&pgd_lock, flags);
>                         pgd_list_add(pgd);
>                         spin_unlock_irqrestore(&pgd_lock, flags);

Didn't try (yet), but I don't think so.  It's not the pgd which is
broken, but the pmd.  And I think this way:

  (1) pmd is created
  (2) pmd is taken out of the slabcache and used for the kernel/xen
      address space (i.e. slot-3 in the PAE pgd).
  (3) xen fills in the page table entries for the hypervisor hole
  (4) pmd released and put back into the slab cache.
  (5) pmd gets reused, but for userspace addresses this time (pgd
      slot 0-2).
  (6) xen_pin() finds the stale entries for the hypervisor hole
  ==> Oops.

cheers,
  Gerd

-- 
Gerd Hoffmann <kraxel@suse.de>

Re: Oops

[Gerd Hoffmann]

[-- Attachment #1: Type: text/plain, Size: 1132 bytes --]

Chris Wright wrote:
> 
> This should not be needed, the console should be xcv0, completely decoupled
> from serial.

You've made the whole thing even more complicated with the last commit.
Enabling VT is possible now.  Good.  No way around that.  The kernel
hangs now though.  Fix is attached:  better don't try to setup the vga
console for xen guests which don't have the hardware.

Next problem:  The default for xencons (tty) conflicts with the virtual
consoles.  I'm tempted to drop the complete xencons=foobar stuff into
the waste basket and leave in xencons=xvc only.  And maybe xencons=off.
 xencons=tty conflicts with the VT subsystem.  xencons=ttyS conflicts
with the serial driver.  Disabling the offending drivers is completely
out of question for a kernel which is supposed to work both native and
paravirtualized.

One more issue:  What should be the default console?  Right now it is
the vt console (using the dummy device).  Not very good.  vgacon doesn't
work.  fbcon doesn't work either (yet).  So you'll end up with a
non-functional console by default.  Bummer.

cheers,
  Gerd

-- 
Gerd Hoffmann <kraxel@suse.de>

[-- Attachment #2: vga --]
[-- Type: text/plain, Size: 1009 bytes --]

Index: paravirt-2.6.20-rc4-hg699/arch/i386/kernel/setup.c
===================================================================
--- paravirt-2.6.20-rc4-hg699.orig/arch/i386/kernel/setup.c
+++ paravirt-2.6.20-rc4-hg699/arch/i386/kernel/setup.c
@@ -61,6 +61,7 @@
 #include <asm/ist.h>
 #include <asm/io.h>
 #include <asm/vmi.h>
+#include <asm/hypervisor.h>
 #include <setup_arch.h>
 #include <bios_ebda.h>
 
@@ -651,12 +652,17 @@ void __init setup_arch(char **cmdline_p)
 	e820_register_memory();
 
 #ifdef CONFIG_VT
-#if defined(CONFIG_VGA_CONSOLE)
-	if (!efi_enabled || (efi_mem_type(0xa0000) != EFI_CONVENTIONAL_MEMORY))
-		conswitchp = &vga_con;
-#elif defined(CONFIG_DUMMY_CONSOLE)
+#if defined(CONFIG_DUMMY_CONSOLE)
 	conswitchp = &dummy_con;
 #endif
+#if defined(CONFIG_VGA_CONSOLE)
+	if (efi_enabled && (efi_mem_type(0xa0000) == EFI_CONVENTIONAL_MEMORY))
+		goto novga;
+	if (is_running_on_xen() && !is_initial_xendomain())
+		goto novga;
+	conswitchp = &vga_con;
+novga:
+#endif
 #endif
 	tsc_init();
 }

[-- Attachment #3: Type: text/plain, Size: 165 bytes --]

_______________________________________________
Virtualization mailing list
Virtualization@lists.osdl.org
https://lists.osdl.org/mailman/listinfo/virtualization

Re: Oops

[Chris Wright]

* Gerd Hoffmann (kraxel@suse.de) wrote:
> Chris Wright wrote:
> > 
> > This should not be needed, the console should be xcv0, completely decoupled
> > from serial.
> 
> You've made the whole thing even more complicated with the last commit.
> Enabling VT is possible now.  Good.  No way around that.  The kernel
> hangs now though.  Fix is attached:  better don't try to setup the vga
> console for xen guests which don't have the hardware.

I'm not really sure what, the last commit I did was lhype?

> Next problem:  The default for xencons (tty) conflicts with the virtual
> consoles.  I'm tempted to drop the complete xencons=foobar stuff into
> the waste basket and leave in xencons=xvc only.  And maybe xencons=off.
>  xencons=tty conflicts with the VT subsystem.  xencons=ttyS conflicts
> with the serial driver.  Disabling the offending drivers is completely
> out of question for a kernel which is supposed to work both native and
> paravirtualized.

Yes, all of tty ttyS xencons goes away.  Here the default is xvc.

> One more issue:  What should be the default console?  Right now it is
> the vt console (using the dummy device).  Not very good.  vgacon doesn't
> work.  fbcon doesn't work either (yet).  So you'll end up with a
> non-functional console by default.  Bummer.

default console should be xvc in the guest.

thanks,
-chris

Re: Oops

[Jeremy Fitzhardinge]

Gerd Hoffmann wrote:
> Didn't try (yet), but I don't think so.  It's not the pgd which is
> broken, but the pmd.  And I think this way:
>
>   (1) pmd is created
>   (2) pmd is taken out of the slabcache and used for the kernel/xen
>       address space (i.e. slot-3 in the PAE pgd).
>   (3) xen fills in the page table entries for the hypervisor hole
>   (4) pmd released and put back into the slab cache.
>   (5) pmd gets reused, but for userspace addresses this time (pgd
>       slot 0-2).
>   (6) xen_pin() finds the stale entries for the hypervisor hole
>   ==> Oops.
Ah, right.

    J

Re: Oops

[Rusty Russell]

On Thu, 2007-01-11 at 16:45 +0100, Gerd Hoffmann wrote:
> Chris Wright wrote:
> > 
> > This should not be needed, the console should be xcv0, completely decoupled
> > from serial.
> 
> You've made the whole thing even more complicated with the last commit.
> Enabling VT is possible now.  Good.  No way around that.  The kernel
> hangs now though.  Fix is attached:  better don't try to setup the vga
> console for xen guests which don't have the hardware.

lguest (lhype) does the following:

	/* FIXME: Better way? */
	/* Suppress vgacon startup code */
	SCREEN_INFO.orig_video_isVGA = VIDEO_TYPE_VLFB;

So I'd prefer a generic way of disabling the VGA stuff...

Thanks,
Rusty.

Re: Oops

[Chris Wright]

* Rusty Russell (rusty@rustcorp.com.au) wrote:
> So I'd prefer a generic way of disabling the VGA stuff...

Heh, me too, like the patch at the bottom. I toyed with putting it into
vgacon directly since we'll need in on x86_64 too, like so:

static int vgacon_disabled;
void vgacon_disable(void)
{
	vgacon_disabled = 1;
}
EXPORT_SYMBOL_GPL(vgacon_disable);

vgacon_startup()
...
-	if (ORIG_VIDEO_ISVGA == VIDEO_TYPE_VLFB)
+	if (vgacon_disabled || ORIG_VIDEO_ISVGA == VIDEO_TYPE_VLFB)
...

but opted for the more arch specific way:

diff -r e1974f36aa13 arch/i386/kernel/setup.c
--- a/arch/i386/kernel/setup.c	Wed Jan 10 18:19:52 2007 -0800
+++ b/arch/i386/kernel/setup.c	Thu Jan 11 18:32:22 2007 -0800
@@ -69,6 +69,9 @@ unsigned long init_pg_tables_end __initd
 unsigned long init_pg_tables_end __initdata = ~0UL;
 
 int disable_pse __devinitdata = 0;
+
+/* use to runtime disable vga_con */
+int vgacon_enabled = 1;
 
 /*
  * Machine setup..
@@ -643,7 +646,7 @@ void __init setup_arch(char **cmdline_p)
 
 #ifdef CONFIG_VT
 #if defined(CONFIG_VGA_CONSOLE)
-	if (!efi_enabled || (efi_mem_type(0xa0000) != EFI_CONVENTIONAL_MEMORY))
+	if (vgacon_enabled && (!efi_enabled || (efi_mem_type(0xa0000) != EFI_CONVENTIONAL_MEMORY)))
 		conswitchp = &vga_con;
 #elif defined(CONFIG_DUMMY_CONSOLE)
 	conswitchp = &dummy_con;
diff -r e1974f36aa13 arch/i386/paravirt-xen/enlighten.c
--- a/arch/i386/paravirt-xen/enlighten.c	Wed Jan 10 18:19:52 2007 -0800
+++ b/arch/i386/paravirt-xen/enlighten.c	Thu Jan 11 18:37:40 2007 -0800
@@ -7,6 +7,7 @@
 #include <linux/start_kernel.h>
 #include <linux/sched.h>
 #include <linux/bootmem.h>
+#include <linux/console.h>
 
 #include <xen/interface/xen.h>
 #include <xen/features.h>
@@ -779,6 +780,9 @@ static asmlinkage void __init xen_start_
 	new_cpu_data.hard_math = 1;
 	identify_cpu(&new_cpu_data);
 
+	/* prepare for xen console */
+	vgacon_enabled = 0;
+
 	/* Poke various useful things into boot_params */
 	LOADER_TYPE = (9 << 4) | 0;
 	INITRD_START = xen_start_info->mod_start ? __pa(xen_start_info->mod_start) : 0;
diff -r e1974f36aa13 include/asm-i386/setup.h
--- a/include/asm-i386/setup.h	Wed Jan 10 18:19:52 2007 -0800
+++ b/include/asm-i386/setup.h	Thu Jan 11 18:36:42 2007 -0800
@@ -77,6 +77,8 @@ void __init add_memory_region(unsigned l
 void __init add_memory_region(unsigned long long start,
 			      unsigned long long size, int type);
 
+extern int vgacon_enabled;
+
 #endif /* __ASSEMBLY__ */
 
 #endif  /*  __KERNEL__  */

Re: Oops

[Gerd Hoffmann]

[-- Attachment #1: Type: text/plain, Size: 1616 bytes --]

Chris Wright wrote:
> * Gerd Hoffmann (kraxel@suse.de) wrote:
>> Chris Wright wrote:
>>> This should not be needed, the console should be xcv0, completely decoupled
>>> from serial.
>> You've made the whole thing even more complicated with the last commit.
>> Enabling VT is possible now.  Good.  No way around that.  The kernel
>> hangs now though.  Fix is attached:  better don't try to setup the vga
>> console for xen guests which don't have the hardware.
> 
> I'm not really sure what, the last commit I did was lhype?

Oh, sorry, was jeremy, wrong line in "hg view" ...

>> Next problem:  The default for xencons (tty) conflicts with the virtual
>> consoles.  I'm tempted to drop the complete xencons=foobar stuff into
>> the waste basket and leave in xencons=xvc only.  And maybe xencons=off.
>>  xencons=tty conflicts with the VT subsystem.  xencons=ttyS conflicts
>> with the serial driver.  Disabling the offending drivers is completely
>> out of question for a kernel which is supposed to work both native and
>> paravirtualized.
> 
> Yes, all of tty ttyS xencons goes away.  Here the default is xvc.

Huh?  It's not yet in the patch paravirt patch queue.  Patch attached.

>> One more issue:  What should be the default console?  Right now it is
>> the vt console (using the dummy device).  Not very good.  vgacon doesn't
>> work.  fbcon doesn't work either (yet).  So you'll end up with a
>> non-functional console by default.  Bummer.
> 
> default console should be xvc in the guest.

Ok, then xen/ must be before char/ in drivers/.  Patch attached.

cheers,
  Gerd

-- 
Gerd Hoffmann <kraxel@suse.de>

[-- Attachment #2: drop-xencons --]
[-- Type: text/plain, Size: 7239 bytes --]

Index: paravirt-2.6.20-rc4-hg702/drivers/xen/console/console.c
===================================================================
--- paravirt-2.6.20-rc4-hg702.orig/drivers/xen/console/console.c
+++ paravirt-2.6.20-rc4-hg702/drivers/xen/console/console.c
@@ -61,20 +61,7 @@
 
 MODULE_LICENSE("Dual BSD/GPL");
 
-/*
- * Modes:
- *  'xencons=off'  [XC_OFF]:     Console is disabled.
- *  'xencons=tty'  [XC_TTY]:     Console attached to '/dev/tty[0-9]+'.
- *  'xencons=ttyS' [XC_SERIAL]:  Console attached to '/dev/ttyS[0-9]+'.
- *  'xencons=xvc'  [XC_XVC]:     Console attached to '/dev/xvc0'.
- *                 [XC_DEFAULT]: DOM0 -> XC_SERIAL ; all others -> XC_TTY.
- * 
- * NB. In mode XC_TTY, we create dummy consoles for tty2-63. This suppresses
- * warnings from standard distro startup scripts.
- */
-static enum {
-	XC_OFF, XC_DEFAULT, XC_TTY, XC_SERIAL, XC_XVC
-} xc_mode = XC_DEFAULT;
+static int xc_disabled = 0;
 static int xc_num = -1;
 
 /* /dev/xvc0 device number allocated by lanana.org. */
@@ -87,27 +74,8 @@ static unsigned long sysrq_requested;
 
 static int __init xencons_setup(char *str)
 {
-	char *q;
-	int n;
-
-	if (!strncmp(str, "ttyS", 4)) {
-		xc_mode = XC_SERIAL;
-		str += 4;
-	} else if (!strncmp(str, "tty", 3)) {
-		xc_mode = XC_TTY;
-		str += 3;
-	} else if (!strncmp(str, "xvc", 3)) {
-		xc_mode = XC_XVC;
-		str += 3;
-	} else if (!strncmp(str, "off", 3)) {
-		xc_mode = XC_OFF;
-		str += 3;
-	}
-
-	n = simple_strtol(str, &q, 10);
-	if (q != str)
-		xc_num = n;
-
+	if (!strcmp(str, "off"))
+		xc_disabled = 1;
 	return 1;
 }
 __setup("xencons=", xencons_setup);
@@ -222,6 +190,7 @@ static struct tty_driver *kcons_device(s
 }
 
 static struct console kcons_info = {
+	.name   = "xvc",
 	.device	= kcons_device,
 	.flags	= CON_PRINTBUFFER | CON_ENABLED,
 	.index	= -1,
@@ -231,42 +200,18 @@ static int __init xen_console_init(void)
 {
 	if (!is_running_on_xen())
 		goto out;
+	if (xc_disabled)
+		goto out;
 
 	if (is_initial_xendomain()) {
-		if (xc_mode == XC_DEFAULT)
-			xc_mode = XC_SERIAL;
 		kcons_info.write = kcons_write_dom0;
 	} else {
 		if (!xen_start_info->console.domU.evtchn)
 			goto out;
-		if (xc_mode == XC_DEFAULT)
-			xc_mode = XC_TTY;
-		kcons_info.flags |= CON_ENABLED;
 		kcons_info.write = kcons_write;
 	}
-
-	switch (xc_mode) {
-	case XC_XVC:
-		strcpy(kcons_info.name, "xvc");
-		if (xc_num == -1)
-			xc_num = 0;
-		break;
-
-	case XC_SERIAL:
-		strcpy(kcons_info.name, "ttyS");
-		if (xc_num == -1)
-			xc_num = 0;
-		break;
-
-	case XC_TTY:
-		strcpy(kcons_info.name, "tty");
-		if (xc_num == -1)
-			xc_num = 1;
-		break;
-
-	default:
-		goto out;
-	}
+	if (xc_num == -1)
+		xc_num = 0;
 
 	wbuf = alloc_bootmem(wbuf_size);
 
@@ -303,11 +248,9 @@ void xencons_force_flush(void)
 /******************** User-space console driver (/dev/console) ************/
 
 #define DRV(_d)         (_d)
-#define DUMMY_TTY(_tty) ((xc_mode == XC_TTY) &&		\
-			 ((_tty)->index != (xc_num - 1)))
 
-static struct termios *xencons_termios[MAX_NR_CONSOLES];
-static struct termios *xencons_termios_locked[MAX_NR_CONSOLES];
+static struct ktermios *xencons_termios[1];
+static struct ktermios *xencons_termios_locked[1];
 static struct tty_struct *xencons_tty;
 static int xencons_priv_irq;
 static char x_char;
@@ -427,9 +370,6 @@ static void xencons_send_xchar(struct tt
 {
 	unsigned long flags;
 
-	if (DUMMY_TTY(tty))
-		return;
-
 	spin_lock_irqsave(&xencons_lock, flags);
 	x_char = ch;
 	__xencons_tx_flush();
@@ -438,18 +378,12 @@ static void xencons_send_xchar(struct tt
 
 static void xencons_throttle(struct tty_struct *tty)
 {
-	if (DUMMY_TTY(tty))
-		return;
-
 	if (I_IXOFF(tty))
 		xencons_send_xchar(tty, STOP_CHAR(tty));
 }
 
 static void xencons_unthrottle(struct tty_struct *tty)
 {
-	if (DUMMY_TTY(tty))
-		return;
-
 	if (I_IXOFF(tty)) {
 		if (x_char != 0)
 			x_char = 0;
@@ -462,9 +396,6 @@ static void xencons_flush_buffer(struct 
 {
 	unsigned long flags;
 
-	if (DUMMY_TTY(tty))
-		return;
-
 	spin_lock_irqsave(&xencons_lock, flags);
 	wc = wp = 0;
 	spin_unlock_irqrestore(&xencons_lock, flags);
@@ -487,9 +418,6 @@ static int xencons_write(
 	int i;
 	unsigned long flags;
 
-	if (DUMMY_TTY(tty))
-		return count;
-
 	spin_lock_irqsave(&xencons_lock, flags);
 
 	for (i = 0; i < count; i++)
@@ -508,9 +436,6 @@ static void xencons_put_char(struct tty_
 {
 	unsigned long flags;
 
-	if (DUMMY_TTY(tty))
-		return;
-
 	spin_lock_irqsave(&xencons_lock, flags);
 	(void)__xencons_put_char(ch);
 	spin_unlock_irqrestore(&xencons_lock, flags);
@@ -520,9 +445,6 @@ static void xencons_flush_chars(struct t
 {
 	unsigned long flags;
 
-	if (DUMMY_TTY(tty))
-		return;
-
 	spin_lock_irqsave(&xencons_lock, flags);
 	__xencons_tx_flush();
 	spin_unlock_irqrestore(&xencons_lock, flags);
@@ -532,9 +454,6 @@ static void xencons_wait_until_sent(stru
 {
 	unsigned long orig_jiffies = jiffies;
 
-	if (DUMMY_TTY(tty))
-		return;
-
 	while (DRV(tty->driver)->chars_in_buffer(tty)) {
 		set_current_state(TASK_INTERRUPTIBLE);
 		schedule_timeout(1);
@@ -551,9 +470,6 @@ static int xencons_open(struct tty_struc
 {
 	unsigned long flags;
 
-	if (DUMMY_TTY(tty))
-		return 0;
-
 	spin_lock_irqsave(&xencons_lock, flags);
 	tty->driver_data = NULL;
 	if (xencons_tty == NULL)
@@ -568,9 +484,6 @@ static void xencons_close(struct tty_str
 {
 	unsigned long flags;
 
-	if (DUMMY_TTY(tty))
-		return;
-
 	mutex_lock(&tty_mutex);
 
 	if (tty->count != 1) {
@@ -616,7 +529,7 @@ static int __init xencons_init(void)
 	if (!is_running_on_xen())
 		return -ENODEV;
 
-	if (xc_mode == XC_OFF)
+	if (xc_disabled)
 		return 0;
 
 	if (!is_initial_xendomain()) {
@@ -625,13 +538,14 @@ static int __init xencons_init(void)
 			return rc;
 	}
 
-	xencons_driver = alloc_tty_driver((xc_mode == XC_TTY) ?
-					  MAX_NR_CONSOLES : 1);
+	xencons_driver = alloc_tty_driver(1);
 	if (xencons_driver == NULL)
 		return -ENOMEM;
 
-	DRV(xencons_driver)->name            = "xencons";
-	DRV(xencons_driver)->major           = TTY_MAJOR;
+	DRV(xencons_driver)->name            = "xvc";
+	DRV(xencons_driver)->major           = XEN_XVC_MAJOR;
+	DRV(xencons_driver)->minor_start     = XEN_XVC_MINOR;
+	DRV(xencons_driver)->name_base       = xc_num;
 	DRV(xencons_driver)->type            = TTY_DRIVER_TYPE_SERIAL;
 	DRV(xencons_driver)->subtype         = SERIAL_TYPE_NORMAL;
 	DRV(xencons_driver)->init_termios    = tty_std_termios;
@@ -642,25 +556,6 @@ static int __init xencons_init(void)
 	DRV(xencons_driver)->termios         = xencons_termios;
 	DRV(xencons_driver)->termios_locked  = xencons_termios_locked;
 
-	switch (xc_mode) {
-	case XC_XVC:
-		DRV(xencons_driver)->name        = "xvc";
-		DRV(xencons_driver)->major       = XEN_XVC_MAJOR;
-		DRV(xencons_driver)->minor_start = XEN_XVC_MINOR;
-		DRV(xencons_driver)->name_base   = xc_num;
-		break;
-	case XC_SERIAL:
-		DRV(xencons_driver)->name        = "ttyS";
-		DRV(xencons_driver)->minor_start = 64 + xc_num;
-		DRV(xencons_driver)->name_base   = xc_num;
-		break;
-	default:
-		DRV(xencons_driver)->name        = "tty";
-		DRV(xencons_driver)->minor_start = 1;
-		DRV(xencons_driver)->name_base   = 1;
-		break;
-	}
-
 	tty_set_operations(xencons_driver, &xencons_ops);
 
 	if ((rc = tty_register_driver(DRV(xencons_driver))) != 0) {

[-- Attachment #3: console-order --]
[-- Type: text/plain, Size: 466 bytes --]

Index: paravirt-2.6.20-rc4-hg702/drivers/Makefile
===================================================================
--- paravirt-2.6.20-rc4-hg702.orig/drivers/Makefile
+++ paravirt-2.6.20-rc4-hg702/drivers/Makefile
@@ -17,8 +17,8 @@ obj-$(CONFIG_ARM_AMBA)		+= amba/
 
 # char/ comes before serial/ etc so that the VT console is the boot-time
 # default.
-obj-y				+= char/
 obj-$(CONFIG_XEN)		+= xen/
+obj-y				+= char/
 
 obj-$(CONFIG_CONNECTOR)		+= connector/
 

[-- Attachment #4: Type: text/plain, Size: 165 bytes --]

_______________________________________________
Virtualization mailing list
Virtualization@lists.osdl.org
https://lists.osdl.org/mailman/listinfo/virtualization

Re: Oops

[Gerd Hoffmann]

> diff -r e1974f36aa13 arch/i386/kernel/setup.c
> --- a/arch/i386/kernel/setup.c	Wed Jan 10 18:19:52 2007 -0800
> +++ b/arch/i386/kernel/setup.c	Thu Jan 11 18:32:22 2007 -0800
[ ... ]
> @@ -643,7 +646,7 @@ void __init setup_arch(char **cmdline_p)
>  
>  #ifdef CONFIG_VT
>  #if defined(CONFIG_VGA_CONSOLE)
> -	if (!efi_enabled || (efi_mem_type(0xa0000) != EFI_CONVENTIONAL_MEMORY))
> +	if (vgacon_enabled && (!efi_enabled || (efi_mem_type(0xa0000) != EFI_CONVENTIONAL_MEMORY)))
>  		conswitchp = &vga_con;
>  #elif defined(CONFIG_DUMMY_CONSOLE)
>  	conswitchp = &dummy_con;
[ ... ]

Approach looks fine to me, please put into the paravirt patch queue.

Well, maybe we can make the efi stuff just set vgacon_disabled too to
make the whole thing more readable.

Hmm, one more thing:  I think we should set conswitchp to dummy_con in
case vgacon is disabled.  Otherwise the vt subsystem isn't initialized
at all and we'll have problems making fbcon+pvfb work later on ...

cheers,

  Gerd

-- 
Gerd Hoffmann <kraxel@suse.de>