Re: dma_alloc_coherent issue with tg3 in x86_64 build

[Jerone Young <jyoung5@us.ibm.com>]

What build are you using? I have a dual opteron with the same card &
driver (tg3) ... and works fine. I'll try today's build. But the build
I'm using is only a few days old and has no problems with x86-64. I'll
try todays and see what happens.

On Thu, 2005-07-28 at 12:36 -0700, Chris Wright wrote:
> During boot of dom0, I get the following.  Box is remote, so I'm
> now waiting for proxy power button pusher ;-)  tg3 init fails doing
> dma_alloc_coherent (I assume this is from tg3_init_one -> tg3_test_dma ->
> pci_alloc_consistent -> dma_alloc_coherent).  Looks like an issue with
> the amount of memory in dom0.  Is this known?
> 
> thanks,
> -chris
> 
> root (hd0,0)
>  Filesystem type is ext2fs, partition type 0x83
> kernel /xen.gz com1=115200,8n1 console=com1,vga
>    [Multiboot-elf, <0x100000:0x7c3e0:0x29c20>, shtab=0x1a6078, entry=0x100000]
> module /vmlinuz-2.6.12-xen0 root=/dev/VolGroup00/LogVol00 ro console=tty0,ttyS0
>    [Multiboot-module @ 0x1a7000, 0x6b43f8 bytes]
> module /initrd-generic-2.6.img
>    [Multiboot-module @ 0x85c000, 0x2ab400 bytes]
> 
>  __  __            _____  ___         _                _ 
>  \ \/ /___ _ __   |___ / / _ \     __| | _____   _____| |
>   \  // _ \ '_ \    |_ \| | | |__ / _` |/ _ \ \ / / _ \ |
>   /  \  __/ | | |  ___) | |_| |__| (_| |  __/\ V /  __/ |
>  /_/\_\___|_| |_| |____(_)___/    \__,_|\___| \_/ \___|_|
>                                                          
>  http://www.cl.cam.ac.uk/netos/xen
>  University of Cambridge Computer Laboratory
> 
>  Xen version 3.0-devel (chrisw@pdx.osdl.net) (gcc version 3.4.3 20041125 (Red Hat 3.4.3-7)) Thu Jul 28 10:54:44 PDT 2005
>  Latest ChangeSet: Thu Jul 28 05:34:45 2005 501a70f3ae968e46e27b9003febf05253f4cf949
> 
> (XEN) Physical RAM map:
> (XEN)  0000000000000000 - 000000000009dc00 (usable)
> (XEN)  000000000009dc00 - 00000000000a0000 (reserved)
> (XEN)  00000000000d0000 - 0000000000100000 (reserved)
> (XEN)  0000000000100000 - 000000007ff60000 (usable)
> (XEN)  000000007ff60000 - 000000007ff72000 (ACPI data)
> (XEN)  000000007ff72000 - 000000007ff80000 (ACPI NVS)
> (XEN)  000000007ff80000 - 0000000080000000 (reserved)
> (XEN)  00000000fec00000 - 00000000fec00400 (reserved)
> (XEN)  00000000fee00000 - 00000000fee01000 (reserved)
> (XEN)  00000000fff80000 - 0000000100000000 (reserved)
> (XEN) System RAM: 2046MB (2096116kB)
> (XEN) Xen heap: 14MB (14632kB)
> (XEN) found SMP MP-table at 000f7270
> (XEN) DMI present.
> (XEN) Using APIC driver default
> (XEN) ACPI: RSDP (v002 PTLTD                                 ) @ 0x00000000000f7210
> (XEN) ACPI: XSDT (v001 PTLTD  	 XSDT   0x06040000  LTP 0x00000000) @ 0x000000007ff6d5a7
> (XEN) ACPI: FADT (v003 AMD    HAMMER   0x06040000 PTEC 0x000f4240) @ 0x000000007ff71d0f
> (XEN) ACPI: SRAT (v001 AMD    HAMMER   0x06040000 AMD  0x00000001) @ 0x000000007ff71e03
> (XEN) ACPI: MADT (v001 PTLTD  	 APIC   0x06040000  LTP 0x00000000) @ 0x000000007ff71ecb
> (XEN) ACPI: ASF! (v016    MBI     CETP 0x06040000 PTL  0x00000001) @ 0x000000007ff71f59
> (XEN) ACPI: DSDT (v001 AMD-K8  AMDACPI 0x06040000 MSFT 0x0100000e) @ 0x0000000000000000
> (XEN) ACPI: Local APIC address 0xfee00000
> (XEN) ACPI: LAPIC (acpi_id[0x00] lapic_id[0x00] enabled)
> (XEN) Processor #0 15:5 APIC version 16
> (XEN) ACPI: LAPIC (acpi_id[0x01] lapic_id[0x01] enabled)
> (XEN) Processor #1 15:5 APIC version 16
> (XEN) ACPI: LAPIC_NMI (acpi_id[0x00] high edge lint[0x1])
> (XEN) ACPI: LAPIC_NMI (acpi_id[0x01] high edge lint[0x1])
> (XEN) ACPI: IOAPIC (id[0x02] address[0xfec00000] gsi_base[0])
> (XEN) IOAPIC[0]: apic_id 2, version 17, address 0xfec00000, GSI 0-23
> (XEN) ACPI: IOAPIC (id[0x03] address[0xc0000000] gsi_base[24])
> (XEN) IOAPIC[1]: apic_id 3, version 17, address 0xc0000000, GSI 24-27
> (XEN) ACPI: IOAPIC (id[0x04] address[0xc0001000] gsi_base[28])
> (XEN) IOAPIC[2]: apic_id 4, version 17, address 0xc0001000, GSI 28-31
> (XEN) ACPI: IOAPIC (id[0x05] address[0xc0600000] gsi_base[32])
> (XEN) IOAPIC[3]: apic_id 5, version 17, address 0xc0600000, GSI 32-35
> (XEN) ACPI: IOAPIC (id[0x06] address[0xc0601000] gsi_base[36])
> (XEN) IOAPIC[4]: apic_id 6, version 17, address 0xc0601000, GSI 36-39
> (XEN) ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 high edge)
> (XEN) ACPI: IRQ0 used by override.
> (XEN) ACPI: IRQ2 used by override.
> (XEN) Enabling APIC mode:  Flat.  Using 5 I/O APICs
> (XEN) Using ACPI (MADT) for SMP configuration information
> (XEN) Initializing CPU#0
> (XEN) Detected 1994.377 MHz processor.
> (XEN) Using scheduler: Simple EDF Scheduler (sedf)
> (XEN) CPU: L1 I Cache: 64K (64 bytes/line), D cache 64K (64 bytes/line)
> (XEN) CPU: L2 Cache: 1024K (64 bytes/line)
> (XEN) CPU0: AMD Opteron(tm) Processor 246 stepping 08
> (XEN) Booting processor 1/1 eip 90000
> (XEN) Initializing CPU#1
> (XEN) CPU: L1 I Cache: 64K (64 bytes/line), D cache 64K (64 bytes/line)
> (XEN) CPU: L2 Cache: 1024K (64 bytes/line)
> (XEN) CPU1: AMD Opteron(tm) Processor 246 stepping 08
> (XEN) Total of 2 processors activated.
> (XEN) ENABLING IO-APIC IRQs
> (XEN) ..TIMER: vector=0x31 pin1=2 pin2=-1
> (XEN) checking TSC synchronization across 2 CPUs: passed.
> (XEN) Platform timer is 1.193MHz PIT
> (XEN) Brought up 2 CPUs
> (XEN) mtrr: v2.0 (20020519)
> (XEN) mtrr: your CPUs had inconsistent fixed MTRR settings
> (XEN) mtrr: probably your BIOS does not setup all CPUs.
> (XEN) mtrr: corrected configuration.
> (XEN) *** LOADING DOMAIN 0 ***
> (XEN) Xen-ELF header found: 'GUEST_OS=linux,GUEST_VER=2.6,XEN_VER=3.0,VIRT_BASE=0xffffffff80000000,LOADER=generic'
> (XEN) PHYSICAL MEMORY ARRANGEMENT:
> (XEN)  Dom0 alloc.:   0000000020000000->0000000040000000 (382799 pages to be allocated)
> (XEN) VIRTUAL MEMORY ARRANGEMENT:
> (XEN)  Loaded kernel: ffffffff80100000->ffffffff805ab086
> (XEN)  Init. ramdisk: ffffffff805ac000->ffffffff80857400
> (XEN)  Phys-Mach map: ffffffff80858000->ffffffff80a4dd3c
> (XEN)  Page tables:   ffffffff80a4e000->ffffffff80a57000
> (XEN)  Start info:    ffffffff80a57000->ffffffff80a58000
> (XEN)  Boot stack:    ffffffff80a58000->ffffffff80a59000
> (XEN)  TOTAL:         ffffffff80000000->ffffffff80c00000
> (XEN)  ENTRY ADDRESS: ffffffff80100000
> (XEN) Initrd len 0x2ab400, start at 0xffffffff805ac000
> (XEN) Scrubbing Free RAM: .....................done.
> (XEN) Xen trace buffers: initialised
> (XEN) *** Serial input -> DOM0 (type 'CTRL-a' three times to switch input to Xen).
> Linux version 2.6.12.2-xen0 (chrisw@fuzzy.pdx.osdl.net) (gcc version 3.4.3 20041125 (Red Hat 3.4.3-7)) #6 Thu Jul 28 10:53:46 PDT 2005
> kernel direct mapping tables upto ffff88007d74f000 @ c00000-fef000
> Registering memory for bootmem: from  1000000, size = 7c74f000
> ACPI: LAPIC (acpi_id[0x00] lapic_id[0x00] enabled)
> ACPI: LAPIC (acpi_id[0x01] lapic_id[0x01] enabled)
> ACPI: LAPIC_NMI (acpi_id[0x00] high edge lint[0x1])
> ACPI: LAPIC_NMI (acpi_id[0x01] high edge lint[0x1])
> ACPI: IOAPIC (id[0x02] address[0xfec00000] gsi_base[0])
> IOAPIC[0]: apic_id 2, version 17, address 0xfec00000, GSI 0-23
> ACPI: IOAPIC (id[0x03] address[0xc0000000] gsi_base[24])
> IOAPIC[1]: apic_id 3, version 17, address 0xc0000000, GSI 24-27
> ACPI: IOAPIC (id[0x04] address[0xc0001000] gsi_base[28])
> IOAPIC[2]: apic_id 4, version 17, address 0xc0001000, GSI 28-31
> ACPI: IOAPIC (id[0x05] address[0xc0600000] gsi_base[32])
> IOAPIC[3]: apic_id 5, version 17, address 0xc0600000, GSI 32-35
> ACPI: IOAPIC (id[0x06] address[0xc0601000] gsi_base[36])
> IOAPIC[4]: apic_id 6, version 17, address 0xc0601000, GSI 36-39
> ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 high edge)
> Setting APIC routing to flat
> Using ACPI (MADT) for SMP configuration information
> Allocating PCI resources starting at 7d800000 (gap: 7d74f000:828b1000)
> Built 1 zonelists
> Kernel command line: root=/dev/VolGroup00/LogVol00 ro console=tty0,ttyS0
> Initializing CPU#0
> PID hash table entries: 4096 (order: 12, 131072 bytes)
> Xen reported: 1994.377 MHz processor.
> Console: colour VGA+ 80x25
> Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes)
> Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes)
> Memory: 2007552k/2055484k available (2820k kernel code, 31364k reserved, 1119k data, 272k init)
> Mount-cache hash table entries: 256
> CPU: L1 I Cache: 64K (64 bytes/line), D cache 64K (64 bytes/line)
> CPU: L2 Cache: 1024K (64 bytes/line)
> CPU: AMD Opteron(tm) Processor 246 stepping 08
> checking if image is initramfs... it is
> NET: Registered protocol family 16
> PCI: Using configuration type 1
> ACPI: Subsystem revision 20050309
> ACPI: Interpreter enabled
> ACPI: Using IOAPIC for interrupt routing
> ACPI: PCI Root Bridge [PCI0] (0000:00)
> PCI: Probing PCI hardware (bus 00)
> ACPI: PCI Interrupt Link [LNKA] (IRQs 3 *5 10 11)
> ACPI: PCI Interrupt Link [LNKB] (IRQs 3 5 10 *11)
> ACPI: PCI Interrupt Link [LNKC] (IRQs 3 5 *10 11)
> ACPI: PCI Interrupt Link [LNKD] (IRQs 3 5 10 *11)
> ACPI: PCI Root Bridge [PCI1] (0000:08)
> PCI: Probing PCI hardware (bus 08)
> xen_mem: Initialising balloon driver.
> SCSI subsystem initialized
> PCI: Using ACPI for IRQ routing
> PCI: If a device doesn't work, try "pci=routeirq".  If it helps, post a report
> Grant table initialized
> IA-32 Microcode Update Driver: v1.14 <tigran@veritas.com>
> IA32 emulation $Id: sys_ia32.c,v 1.32 2002/03/24 13:02:28 ak Exp $
> Initializing Cryptographic API
> PCI: MSI quirk detected. pci_msi_quirk set.
> PCI: MSI quirk detected. pci_msi_quirk set.
> PCI: MSI quirk detected. pci_msi_quirk set.
> PCI: MSI quirk detected. pci_msi_quirk set.
> Real Time Clock Driver v1.12
> serio: i8042 AUX port at 0x60,0x64 irq 12
> serio: i8042 KBD port at 0x60,0x64 irq 1
> io scheduler noop registered
> io scheduler anticipatory registered
> io scheduler deadline registered
> io scheduler cfq registered
> floppy0: no floppy controllers found
> RAMDISK driver initialized: 16 RAM disks of 16384K size 1024 blocksize
> loop: loaded (max 8 devices)
> HP CISS Driver (v 2.6.6)
> Intel(R) PRO/1000 Network Driver - version 6.0.54-k2
> Copyright (c) 1999-2004 Intel Corporation.
> pcnet32.c:v1.30j 29.04.2005 tsbogend@alpha.franken.de
> e100: Intel(R) PRO/100 Network Driver, 3.4.8-k2-NAPI
> e100: Copyright(c) 1999-2005 Intel Corporation
> tg3.c:v3.31 (June 8, 2005)
> ACPI: PCI Interrupt 0000:03:02.0[A] -> GSI 28 (level, low) -> IRQ 28
> (XEN) (file=dom_mem_ops.c, line=61) Could not allocate a frame
> ----------- [cut here ] --------- [please bite here ] ---------
> Kernel BUG at "arch/xen/x86_64/mm/hypervisor.c":299
> invalid operand: 0000 [1] 
> CPU 0 
> Modules linked in:
> Pid: 1, comm: swapper Not tainted 2.6.12.2-xen0
> RIP: e030:[<ffffffff8011a32a>] <ffffffff8011a32a>{xen_contig_memory+586}
> RSP: e02b:ffff880002ee1bf8  EFLAGS: 00010296
> RAX: 0000000000000000 RBX: 000000000000000c RCX: ffffffff8011a324
> RDX: 0000000000000001 RSI: ffff880002ee1c28 RDI: 00000000ffffffff
> RBP: 0000000000000001 R08: 0000000000007ff0 R09: 0000000000000001
> R10: 0000000000000001 R11: 0000000000000246 R12: 0000000000000000
> R13: 0000000000000002 R14: 0000000000000001 R15: ffff800000000000
> FS:  0000000000000000(0000) GS:ffffffff80533a80(0000) knlGS:0000000000000000
> CS:  e033 DS: 0000 ES: 0000
> Process swapper (pid: 1, threadinfo ffff880002ee0000, task ffff880002edf410)
> Stack: 000000001002029a ffff880078cc6000 0000000202edf410 0000000052aa4d54 
>        0000000000000000 ffff880078cc4000 0000000000044a89 0000000000000001 
>        ffff880078cc4000 0000000000000000 
> Call Trace:<ffffffff80114920>{dma_alloc_coherent+224} <ffffffff802a8096>{tg3_init_one+7302}
>        <ffffffff803bdafa>{wait_for_completion+170} <ffffffff80126dd0>{default_wake_function+0}
>        <ffffffff802212d6>{pci_device_probe+134} <ffffffff8025ac0d>{driver_probe_device+77}
>        <ffffffff8025ad2b>{driver_attach+75} <ffffffff8025b118>{bus_add_driver+152}
>        <ffffffff80220fe5>{pci_register_driver+117} <ffffffff801090f2>{init+178}
>        <ffffffff8010e567>{child_rip+8} <ffffffff80109040>{init+0}
>        <ffffffff8010e55f>{child_rip+0} 
> 
> Code: 0f 0b 08 91 3d 80 ff ff ff ff 2b 01 b8 01 00 00 00 44 89 f1 
> RIP <ffffffff8011a32a>{xen_contig_memory+586} RSP <ffff880002ee1bf8>
>  <0>Kernel panic - not syncing: Attempted to kill init!
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel
> 
-- 
Jerone Young
IBM Linux Technology Center
jyoung5@us.ibm.com
512-838-1157 (T/L: 678-1157)