Re: pvops dom0: no sound after boot; possibly causedby swiotlb

Re: pvops dom0: no sound after boot; possibly causedby swiotlb

[Ronny.Hegewald]

[-- Attachment #1: Type: text/plain, Size: 2571 bytes --]

>On Tue, Jan 26, 2010 at 01:40:12AM +0100, Ronny.Hegewald@online.de wrote:
>> Software: xen 3.4.1, lastest xen/master (version 2.6.31.6), both 32bit
>> Hardware: Intel Core2Duo System
>>           4GB Ram
>>           Realtek ALC888 soundchip
>
>Motherboard?
MSI P35 Neo.

>
>What I think you are saying is that, when you perform these steps:
>
>1). Boot up machine, have swiotlb=16384 set (or just hand-coded
>swiotlb.c to have a default size of 32MB).
>
>2). Start a DomU with more than 66MB allocated to it.
>
>3). Load the sound modules in Dom0
>
>4). Play music/sound/etc in Dom0, the sound works fine.

Generally right except a little difference.

1.) if i let the swiotlb unchanged at 64 MB and start a DomU with 66 MB or more Ram and do steps 3 and 4 the sound appears; with less RAM there is no sound
2.) if i lower the swiotlb to 32 MB and start a DomU with less then 66 MB Ram and do steps 3 and 4, then the sound is fine.


>What I am curious is your E820 table. That is the first thing Xen
>prints. You can get that by doing 'xm dmesg'
>
>Also include your 'dmesg' output for good measure. 

Attached.

>Lastly, try adding this line to your Xen line: dom0_mem=max:2GB and
>don't do any of the above mentioned hacks. Just start the sound module
>and see if it works.

I tried that too but used as parameter dom0_mem=2GB. But i will retry it anyway because i tried many several things so i might have done something wrong in the process.

>Back to the sound card. The sound card can (I think) only DMA up to 4GB,
>so when it tries to fetch data from above 4GB it ends up truncating the
>physical address down to 32-bit and fetches data from somewhere else. So
>you are probably listening to binary code being played :-)

In that case shouldnt i have that problem on bare metal too? And there is no such problem under a 2.6.31 kernel with the forwardported dom0 patches from gentoo either. 

>Having a DomU start makes Xen shrink Dom0 by a certain size and the DMA
>window for the sound is moved "down" below the 4GB mark (that I think is
>the problem you are encountering). This should NOT happen if the sound
>card driver is using DMA libraries to allocate that buffer (ie,
>dma_alloc_coherent, dma_alloc_free, etc). But it might be using
>'vmalloc_32' which on virtualized environments is not guaranteed to give
>a swatch of memory below 4GB. Here are the steps to investigate:
>look in the sound card drivers and see where and how it allocates the buffer.

Thanks alot for all the explanations. Now i have a starting-point to do some further investigations.

[-- Attachment #2: dmesg.log --]
[-- Type: application/octet-stream, Size: 27830 bytes --]

Reserving virtual address space above 0xf5800000
Initializing cgroup subsys cpuset
Initializing cgroup subsys cpu
Linux version 2.6.31.6-xen-pvops (root@client2-lfs32) (gcc version 3.4.6) #19 SMP Tue Jan 26 02:21:13 CET 2010
KERNEL supported cpus:
  Intel GenuineIntel
  AMD AuthenticAMD
  Cyrix CyrixInstead
  NSC Geode by NSC
  Centaur CentaurHauls
  Transmeta GenuineTMx86
  Transmeta TransmetaCPU
  UMC UMC UMC UMC
xen_release_chunk: looking at area pfn d0000-fee00
xen_release_chunk: looking at area pfn fee01-ffb00
xen_release_chunk: looking at area pfn 0-f5130
released 0 pages of unused memory
BIOS-provided physical RAM map:
 Xen: 0000000000000000 - 000000000009fc00 (usable)
 Xen: 000000000009fc00 - 0000000000100000 (reserved)
 Xen: 0000000000100000 - 00000000cffb0000 (usable)
 Xen: 00000000cffb0000 - 00000000cffbe000 (ACPI data)
 Xen: 00000000cffbe000 - 00000000cfff0000 (ACPI NVS)
 Xen: 00000000cfff0000 - 00000000d0000000 (reserved)
 Xen: 00000000fee00000 - 00000000fee01000 (reserved)
 Xen: 00000000ffb00000 - 0000000100000000 (reserved)
DMI present.
AMI BIOS detected: BIOS may corrupt low RAM, working around it.
e820 update range: 0000000000000000 - 0000000000010000 (usable) ==> (reserved)
last_pfn = 0xcffb0 max_arch_pfn = 0x1000000
Scanning 0 areas for low memory corruption
modified physical RAM map:
 modified: 0000000000000000 - 0000000000010000 (reserved)
 modified: 0000000000010000 - 000000000009fc00 (usable)
 modified: 000000000009fc00 - 0000000000100000 (reserved)
 modified: 0000000000100000 - 00000000cffb0000 (usable)
 modified: 00000000cffb0000 - 00000000cffbe000 (ACPI data)
 modified: 00000000cffbe000 - 00000000cfff0000 (ACPI NVS)
 modified: 00000000cfff0000 - 00000000d0000000 (reserved)
 modified: 00000000fee00000 - 00000000fee01000 (reserved)
 modified: 00000000ffb00000 - 0000000100000000 (reserved)
initial memory mapped : 0 - 02fff000
init_memory_mapping: 0000000000000000-000000002d1fe000
 0000000000 - 002d1fe000 page 4k
kernel direct mapping tables up to 2d1fe000 @ 100000-26d000
RAMDISK: 015a1000 - 022d8c00
ACPI: RSDP 000f95d0 00014 (v00 ACPIAM)
ACPI: RSDT cffb0000 0003C (v01 101107 RSDT1411 20071011 MSFT 00000097)
ACPI: FACP cffb0200 00084 (v01 101107 FACP1411 20071011 MSFT 00000097)
ACPI: DSDT cffb0440 05564 (v01  0AAAA 0AAAA000 00000000 INTL 20051117)
ACPI: FACS cffbe000 00040
ACPI: APIC cffb0390 0006C (v01 101107 APIC1411 20071011 MSFT 00000097)
ACPI: MCFG cffb0400 0003C (v01 101107 OEMMCFG  20071011 MSFT 00000097)
ACPI: OEMB cffbe040 00071 (v01 101107 OEMB1411 20071011 MSFT 00000097)
ACPI: HPET cffb59b0 00038 (v01 101107 OEMHPET  20071011 MSFT 00000097)
ACPI: GSCI cffbe0c0 02024 (v01 101107 GMCHSCI  20071011 MSFT 00000097)
ACPI: Local APIC address 0xfee00000
2605MB HIGHMEM available.
721MB LOWMEM available.
  mapped low ram: 0 - 2d1fe000
  low ram: 0 - 2d1fe000
  node 0 low ram: 00000000 - 2d1fe000
  node 0 bootmap 00010000 - 00015a40
(10 early reservations) ==> bootmem [0000000000 - 002d1fe000]
  #0 [0000000000 - 0000001000]   BIOS data page ==> [0000000000 - 0000001000]
  #1 [00026af000 - 00026c8000]   XEN PAGETABLES ==> [00026af000 - 00026c8000]
  #2 [0000001000 - 0000002000]    EX TRAMPOLINE ==> [0000001000 - 0000002000]
  #3 [0000006000 - 0000007000]       TRAMPOLINE ==> [0000006000 - 0000007000]
  #4 [0001000000 - 000147bca8]    TEXT DATA BSS ==> [0001000000 - 000147bca8]
  #5 [00015a1000 - 00022d8c00]          RAMDISK ==> [00015a1000 - 00022d8c00]
  #6 [00022d9000 - 00026af000]   XEN START INFO ==> [00022d9000 - 00026af000]
  #7 [000147c000 - 000147c15b]              BRK ==> [000147c000 - 000147c15b]
  #8 [0000100000 - 0000252000]          PGTABLE ==> [0000100000 - 0000252000]
  #9 [0000010000 - 0000016000]          BOOTMAP ==> [0000010000 - 0000016000]
found SMP MP-table at [c00ff780] ff780
Zone PFN ranges:
  DMA      0x00000010 -> 0x00001000
  Normal   0x00001000 -> 0x0002d1fe
  HighMem  0x0002d1fe -> 0x000cffb0
Movable zone start PFN for each node
early_node_map[2] active PFN ranges
    0: 0x00000010 -> 0x0000009f
    0: 0x00000100 -> 0x000cffb0
On node 0 totalpages: 851775
free_area_init_node: node 0, pgdat c12f2bc0, node_mem_map c26c8200
  DMA zone: 32 pages used for memmap
  DMA zone: 0 pages reserved
  DMA zone: 3951 pages, LIFO batch:0
  Normal zone: 1412 pages used for memmap
  Normal zone: 179322 pages, LIFO batch:31
  HighMem zone: 5212 pages used for memmap
  HighMem zone: 661846 pages, LIFO batch:31
Using APIC driver default
ACPI: PM-Timer IO Port: 0x808
ACPI: Local APIC address 0xfee00000
ACPI: LAPIC (acpi_id[0x01] lapic_id[0x00] enabled)
ACPI: LAPIC (acpi_id[0x02] lapic_id[0x01] enabled)
ACPI: LAPIC (acpi_id[0x03] lapic_id[0x82] disabled)
ACPI: LAPIC (acpi_id[0x04] lapic_id[0x83] disabled)
ACPI: IOAPIC (id[0x02] address[0xfec00000] gsi_base[0])
IOAPIC[0]: apic_id 2, version 32, address 0xfec00000, GSI 0-23
ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)
ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)
ACPI: IRQ0 used by override.
ACPI: IRQ2 used by override.
ACPI: IRQ9 used by override.
Enabling APIC mode:  Flat.  Using 1 I/O APICs
Using ACPI (MADT) for SMP configuration information
ACPI: HPET id: 0xffffffff base: 0xfed00000
SMP: Allowing 2 CPUs, 0 hotplug CPUs
nr_irqs_gsi: 24
Allocating PCI resources starting at d0000000 (gap: d0000000:2ee00000)
NR_CPUS:4 nr_cpumask_bits:4 nr_cpu_ids:2 nr_node_ids:1
PERCPU: Allocated 8 4k pages, static data 29212 bytes
trying to map vcpu_info 0 at ed9ff020, mfn 12dada, offset 32
cpu 0 using vcpu_info at ed9ff020
trying to map vcpu_info 1 at eda0f020, mfn 12dae2, offset 32
cpu 1 using vcpu_info at eda0f020
Xen: using vcpu_info placement
Built 1 zonelists in Zone order, mobility grouping on.  Total pages: 845119
Kernel command line: root=/dev/ram0 newroot=/dev/sda7 rw chipset=intel sata=ata_piix network=r8169 xen-dom0 
PID hash table entries: 4096 (order: 12, 16384 bytes)
Dentry cache hash table entries: 131072 (order: 7, 524288 bytes)
Inode-cache hash table entries: 65536 (order: 6, 262144 bytes)
Enabling fast FPU save and restore... done.
Enabling unmasked SIMD FPU exception support... done.
Initializing CPU#0
allocated 17037440 bytes of page_cgroup
please try 'cgroup_disable=memory' option if you don't want memory cgroups
PCI-DMA: Using Xen software bounce buffering for IO (Xen-SWIOTLB)
xen_swiotlb_fixup: buf=c51f0000 size=67108864
xen_swiotlb_fixup: buf=c9250000 size=32768
Placing 64MB Xen software IO TLB between c51f0000 - c91f0000
Xen software IO TLB at phys 0x51f0000 - 0x91f0000
Initializing HighMem for node 0 (0002d1fe:000cffb0)
Memory: 3272652k/3407552k available (2194k kernel code, 133660k reserved, 836k data, 348k init, 2668232k highmem)
virtual kernel memory layout:
    fixmap  : 0xf576a000 - 0xf57ff000   ( 596 kB)
    pkmap   : 0xf5200000 - 0xf5400000   (2048 kB)
    vmalloc : 0xed9fe000 - 0xf51fe000   ( 120 MB)
    lowmem  : 0xc0000000 - 0xed1fe000   ( 721 MB)
      .init : 0xc12f6000 - 0xc134d000   ( 348 kB)
      .data : 0xc1224982 - 0xc12f5c28   ( 836 kB)
      .text : 0xc1000000 - 0xc1224982   (2194 kB)
NR_IRQS:384
xen: registering gsi 0 triggering 1 polarity 0
xen: --> irq=0
xen_set_ioapic_routing: irq 0 gsi 0 vector 240 ioapic 0 pin 0 triggering 0 polarity 0
xen: registering gsi 1 triggering 1 polarity 0
xen: --> irq=1
xen_set_ioapic_routing: irq 1 gsi 1 vector 32 ioapic 0 pin 1 triggering 0 polarity 0
xen: registering gsi 3 triggering 1 polarity 0
xen: --> irq=3
xen_set_ioapic_routing: irq 3 gsi 3 vector 40 ioapic 0 pin 3 triggering 0 polarity 0
xen: registering gsi 4 triggering 1 polarity 0
xen: --> irq=4
xen_set_ioapic_routing: irq 4 gsi 4 vector 48 ioapic 0 pin 4 triggering 0 polarity 0
xen: registering gsi 5 triggering 1 polarity 0
xen: --> irq=5
xen_set_ioapic_routing: irq 5 gsi 5 vector 56 ioapic 0 pin 5 triggering 0 polarity 0
xen: registering gsi 6 triggering 1 polarity 0
xen: --> irq=6
xen_set_ioapic_routing: irq 6 gsi 6 vector 64 ioapic 0 pin 6 triggering 0 polarity 0
xen: registering gsi 7 triggering 1 polarity 0
xen: --> irq=7
xen_set_ioapic_routing: irq 7 gsi 7 vector 72 ioapic 0 pin 7 triggering 0 polarity 0
xen: registering gsi 8 triggering 1 polarity 0
xen: --> irq=8
xen_set_ioapic_routing: irq 8 gsi 8 vector 80 ioapic 0 pin 8 triggering 0 polarity 0
xen: registering gsi 9 triggering 0 polarity 0
xen: --> irq=9
xen_set_ioapic_routing: irq 9 gsi 9 vector 88 ioapic 0 pin 9 triggering 1 polarity 0
xen: registering gsi 10 triggering 1 polarity 0
xen: --> irq=10
xen_set_ioapic_routing: irq 10 gsi 10 vector 96 ioapic 0 pin 10 triggering 0 polarity 0
xen: registering gsi 11 triggering 1 polarity 0
xen: --> irq=11
xen_set_ioapic_routing: irq 11 gsi 11 vector 104 ioapic 0 pin 11 triggering 0 polarity 0
xen: registering gsi 12 triggering 1 polarity 0
xen: --> irq=12
xen_set_ioapic_routing: irq 12 gsi 12 vector 112 ioapic 0 pin 12 triggering 0 polarity 0
xen: registering gsi 13 triggering 1 polarity 0
xen: --> irq=13
xen_set_ioapic_routing: irq 13 gsi 13 vector 120 ioapic 0 pin 13 triggering 0 polarity 0
xen: registering gsi 14 triggering 1 polarity 0
xen: --> irq=14
xen_set_ioapic_routing: irq 14 gsi 14 vector 136 ioapic 0 pin 14 triggering 0 polarity 0
xen: registering gsi 15 triggering 1 polarity 0
xen: --> irq=15
xen_set_ioapic_routing: irq 15 gsi 15 vector 144 ioapic 0 pin 15 triggering 0 polarity 0
Detected 2669.794 MHz processor.
Console: colour VGA+ 80x25
console [tty0] enabled
Xen: using vcpuop timer interface
installing Xen timer for CPU 0
Calibrating delay loop (skipped), value calculated using timer frequency.. 5339.58 BogoMIPS (lpj=2669794)
Mount-cache hash table entries: 512
Initializing cgroup subsys ns
Initializing cgroup subsys cpuacct
Initializing cgroup subsys memory
Initializing cgroup subsys devices
Initializing cgroup subsys freezer
CPU: L1 I cache: 32K, L1 D cache: 32K
CPU: L2 cache: 6144K
CPU: Physical Processor ID: 0
CPU: Processor Core ID: 0
mce: CPU supports 6 MCE banks
Performance Counters: unsupported p6 CPU model 23 no PMU driver, software counters only.
SMP alternatives: switching to UP code
ACPI: Core revision 20090521
cpu 0 spinlock event irq 382
installing Xen timer for CPU 1
cpu 1 spinlock event irq 376
SMP alternatives: switching to SMP code
Initializing CPU#1
CPU: L1 I cache: 32K, L1 D cache: 32K
CPU: L2 cache: 6144K
CPU: Physical Processor ID: 0
CPU: Processor Core ID: 1
mce: CPU supports 6 MCE banks
Brought up 2 CPUs
Booting paravirtualized kernel on Xen
Xen version: 3.4.1 (preserve-AD) (dom0)
Grant tables using version 1 layout.
Grant table initialized
NET: Registered protocol family 16
xenbus_probe_init ok
ACPI: bus type pci registered
PCI: MCFG configuration 0: base e0000000 segment 0 buses 0 - 255
PCI: Not using MMCONFIG.
PCI: PCI BIOS revision 3.00 entry at 0xf0031, last bus=4
PCI: Using configuration type 1 for base access
bio: create slab <bio-0> at 0
ACPI: EC: Look up EC in DSDT
ACPI: Interpreter enabled
ACPI: (supports S0 S5)
ACPI: Using IOAPIC for interrupt routing
PCI: MCFG configuration 0: base e0000000 segment 0 buses 0 - 255
PCI: MCFG area at e0000000 reserved in ACPI motherboard resources
PCI: Using MMCONFIG for extended config space
ACPI: No dock devices found.
ACPI: PCI Root Bridge [PCI0] (0000:00)
pci 0000:00:01.0: PME# supported from D0 D3hot D3cold
pci 0000:00:01.0: PME# disabled
pci 0000:00:1a.0: reg 20 io port: [0xcc00-0xcc1f]
pci 0000:00:1a.1: reg 20 io port: [0xc880-0xc89f]
pci 0000:00:1a.7: reg 10 32bit mmio: [0xf9fffc00-0xf9ffffff]
pci 0000:00:1a.7: PME# supported from D0 D3hot D3cold
pci 0000:00:1a.7: PME# disabled
pci 0000:00:1b.0: reg 10 64bit mmio: [0xf9ff8000-0xf9ffbfff]
pci 0000:00:1b.0: PME# supported from D0 D3hot D3cold
pci 0000:00:1b.0: PME# disabled
pci 0000:00:1c.0: PME# supported from D0 D3hot D3cold
pci 0000:00:1c.0: PME# disabled
pci 0000:00:1c.5: PME# supported from D0 D3hot D3cold
pci 0000:00:1c.5: PME# disabled
pci 0000:00:1d.0: reg 20 io port: [0xc800-0xc81f]
pci 0000:00:1d.1: reg 20 io port: [0xc480-0xc49f]
pci 0000:00:1d.2: reg 20 io port: [0xc400-0xc41f]
pci 0000:00:1d.3: reg 20 io port: [0xc080-0xc09f]
pci 0000:00:1d.7: reg 10 32bit mmio: [0xf9fff800-0xf9fffbff]
pci 0000:00:1d.7: PME# supported from D0 D3hot D3cold
pci 0000:00:1d.7: PME# disabled
pci 0000:00:1f.0: quirk: region 0800-087f claimed by ICH6 ACPI/GPIO/TCO
pci 0000:00:1f.0: quirk: region 0480-04bf claimed by ICH6 GPIO
pci 0000:00:1f.0: ICH7 LPC Generic IO decode 1 PIO at 0a00 (mask 00ff)
pci 0000:00:1f.2: reg 10 io port: [0xc000-0xc007]
pci 0000:00:1f.2: reg 14 io port: [0xbc00-0xbc03]
pci 0000:00:1f.2: reg 18 io port: [0xb880-0xb887]
pci 0000:00:1f.2: reg 1c io port: [0xb800-0xb803]
pci 0000:00:1f.2: reg 20 io port: [0xb480-0xb48f]
pci 0000:00:1f.2: reg 24 io port: [0xb400-0xb40f]
pci 0000:00:1f.3: reg 10 64bit mmio: [0xf9fff400-0xf9fff4ff]
pci 0000:00:1f.3: reg 20 io port: [0x400-0x41f]
pci 0000:00:1f.5: reg 10 io port: [0xb000-0xb007]
pci 0000:00:1f.5: reg 14 io port: [0xac00-0xac03]
pci 0000:00:1f.5: reg 18 io port: [0xa880-0xa887]
pci 0000:00:1f.5: reg 1c io port: [0xa800-0xa803]
pci 0000:00:1f.5: reg 20 io port: [0xa480-0xa48f]
pci 0000:00:1f.5: reg 24 io port: [0xa400-0xa40f]
pci 0000:01:00.0: reg 10 32bit mmio: [0xfd000000-0xfdffffff]
pci 0000:01:00.0: reg 14 64bit mmio: [0xd0000000-0xdfffffff]
pci 0000:01:00.0: reg 1c 64bit mmio: [0xfa000000-0xfbffffff]
pci 0000:01:00.0: reg 24 io port: [0xdc00-0xdc7f]
pci 0000:01:00.0: reg 30 32bit mmio: [0xfeae0000-0xfeafffff]
pci 0000:00:01.0: bridge io port: [0xd000-0xdfff]
pci 0000:00:01.0: bridge 32bit mmio: [0xfa000000-0xfeafffff]
pci 0000:00:01.0: bridge 64bit mmio pref: [0xd0000000-0xdfffffff]
pci 0000:03:00.0: reg 10 io port: [0xe800-0xe8ff]
pci 0000:03:00.0: reg 18 64bit mmio: [0xfebff000-0xfebfffff]
pci 0000:03:00.0: reg 30 32bit mmio: [0xfebc0000-0xfebdffff]
pci 0000:03:00.0: supports D1 D2
pci 0000:03:00.0: PME# supported from D1 D2 D3hot D3cold
pci 0000:03:00.0: PME# disabled
pci 0000:00:1c.5: bridge io port: [0xe000-0xefff]
pci 0000:00:1c.5: bridge 32bit mmio: [0xfeb00000-0xfebfffff]
pci 0000:00:1e.0: transparent bridge
pci_bus 0000:00: on NUMA node 0
ACPI: PCI Interrupt Routing Table [\_SB_.PCI0._PRT]
ACPI: PCI Interrupt Routing Table [\_SB_.PCI0.P0P1._PRT]
ACPI: PCI Interrupt Routing Table [\_SB_.PCI0.P0P4._PRT]
ACPI: PCI Interrupt Routing Table [\_SB_.PCI0.P0P9._PRT]
ACPI: PCI Interrupt Link [LNKA] (IRQs 3 4 6 7 *10 11 12 14 15)
ACPI: PCI Interrupt Link [LNKB] (IRQs *5)
ACPI: PCI Interrupt Link [LNKC] (IRQs 3 4 6 7 10 11 12 *14 15)
ACPI: PCI Interrupt Link [LNKD] (IRQs 3 4 6 7 10 *11 12 14 15)
ACPI: PCI Interrupt Link [LNKE] (IRQs 3 4 6 7 10 11 12 14 15) *0, disabled.
ACPI: PCI Interrupt Link [LNKF] (IRQs 3 4 6 7 10 11 12 14 *15)
ACPI: PCI Interrupt Link [LNKG] (IRQs *3 4 6 7 10 11 12 14 15)
ACPI: PCI Interrupt Link [LNKH] (IRQs 3 4 6 *7 10 11 12 14 15)
sync cpu 0 get result ffffffff max_id 0
Failed to sync pcpu 0
xenbus_probe_backend_init bus registered ok
xen_balloon: Initialising balloon driver with page order 0.
SCSI subsystem initialized
PCI: Using ACPI for IRQ routing
IO APIC resources couldn't be allocated.
Switched to high resolution mode on CPU 0
Switched to high resolution mode on CPU 1
pnp: PnP ACPI init
ACPI: bus type pnp registered
xen: registering gsi 8 triggering 1 polarity 0
xen_allocate_pirq: returning irq 8 for gsi 8
xen: --> irq=8
xen_set_ioapic_routing: irq 8 gsi 8 vector 80 ioapic 0 pin 8 triggering 0 polarity 0
xen: registering gsi 13 triggering 1 polarity 0
xen_allocate_pirq: returning irq 13 for gsi 13
xen: --> irq=13
xen_set_ioapic_routing: irq 13 gsi 13 vector 120 ioapic 0 pin 13 triggering 0 polarity 0
xen: registering gsi 1 triggering 1 polarity 0
xen_allocate_pirq: returning irq 1 for gsi 1
xen: --> irq=1
xen_set_ioapic_routing: irq 1 gsi 1 vector 32 ioapic 0 pin 1 triggering 0 polarity 0
pnp: PnP ACPI: found 15 devices
ACPI: ACPI bus type pnp unregistered
system 00:01: iomem range 0xfed14000-0xfed19fff has been reserved
system 00:07: ioport range 0xa00-0xadf has been reserved
system 00:07: ioport range 0xae0-0xaef has been reserved
system 00:08: ioport range 0x4d0-0x4d1 has been reserved
system 00:08: ioport range 0x800-0x87f has been reserved
system 00:08: ioport range 0x480-0x4bf has been reserved
system 00:08: iomem range 0xfed1c000-0xfed1ffff has been reserved
system 00:08: iomem range 0xfed20000-0xfed8ffff has been reserved
system 00:0b: iomem range 0xffc00000-0xffefffff has been reserved
system 00:0c: iomem range 0xfec00000-0xfec00fff has been reserved
system 00:0c: iomem range 0xfee00000-0xfee00fff has been reserved
system 00:0d: iomem range 0xe0000000-0xefffffff has been reserved
system 00:0e: iomem range 0x0-0x9ffff could not be reserved
system 00:0e: iomem range 0xc0000-0xcffff could not be reserved
system 00:0e: iomem range 0xe0000-0xfffff could not be reserved
system 00:0e: iomem range 0x100000-0xcfffffff could not be reserved
pci 0000:00:01.0: PCI bridge, secondary bus 0000:01
pci 0000:00:01.0:   IO window: 0xd000-0xdfff
pci 0000:00:01.0:   MEM window: 0xfa000000-0xfeafffff
pci 0000:00:01.0:   PREFETCH window: 0x000000d0000000-0x000000dfffffff
pci 0000:00:1c.0: PCI bridge, secondary bus 0000:02
pci 0000:00:1c.0:   IO window: disabled
pci 0000:00:1c.0:   MEM window: disabled
pci 0000:00:1c.0:   PREFETCH window: disabled
pci 0000:00:1c.5: PCI bridge, secondary bus 0000:03
pci 0000:00:1c.5:   IO window: 0xe000-0xefff
pci 0000:00:1c.5:   MEM window: 0xfeb00000-0xfebfffff
pci 0000:00:1c.5:   PREFETCH window: disabled
pci 0000:00:1e.0: PCI bridge, secondary bus 0000:04
pci 0000:00:1e.0:   IO window: disabled
pci 0000:00:1e.0:   MEM window: disabled
pci 0000:00:1e.0:   PREFETCH window: disabled
xen: registering gsi 16 triggering 0 polarity 1
xen: --> irq=16
xen_set_ioapic_routing: irq 16 gsi 16 vector 152 ioapic 0 pin 16 triggering 1 polarity 1
pci 0000:00:01.0: PCI INT A -> GSI 16 (level, low) -> IRQ 16
pci 0000:00:01.0: setting latency timer to 64
xen: registering gsi 17 triggering 0 polarity 1
xen: --> irq=17
xen_set_ioapic_routing: irq 17 gsi 17 vector 160 ioapic 0 pin 17 triggering 1 polarity 1
pci 0000:00:1c.0: PCI INT A -> GSI 17 (level, low) -> IRQ 17
pci 0000:00:1c.0: setting latency timer to 64
xen: registering gsi 16 triggering 0 polarity 1
xen_allocate_pirq: returning irq 16 for gsi 16
xen: --> irq=16
xen_set_ioapic_routing: irq 16 gsi 16 vector 152 ioapic 0 pin 16 triggering 1 polarity 1
pci 0000:00:1c.5: PCI INT B -> GSI 16 (level, low) -> IRQ 16
pci 0000:00:1c.5: setting latency timer to 64
pci 0000:00:1e.0: setting latency timer to 64
pci_bus 0000:00: resource 0 io:  [0x00-0xffff]
pci_bus 0000:00: resource 1 mem: [0x000000-0xffffffffffffffff]
pci_bus 0000:01: resource 0 io:  [0xd000-0xdfff]
pci_bus 0000:01: resource 1 mem: [0xfa000000-0xfeafffff]
pci_bus 0000:01: resource 2 pref mem [0xd0000000-0xdfffffff]
pci_bus 0000:03: resource 0 io:  [0xe000-0xefff]
pci_bus 0000:03: resource 1 mem: [0xfeb00000-0xfebfffff]
pci_bus 0000:04: resource 3 io:  [0x00-0xffff]
pci_bus 0000:04: resource 4 mem: [0x000000-0xffffffffffffffff]
NET: Registered protocol family 2
IP route cache hash table entries: 32768 (order: 5, 131072 bytes)
TCP established hash table entries: 131072 (order: 8, 1048576 bytes)
TCP bind hash table entries: 65536 (order: 7, 524288 bytes)
TCP: Hash tables configured (established 131072 bind 65536)
TCP reno registered
Unpacking initramfs...
Freeing initrd memory: 13535k freed
Scanning for low memory corruption every 60 seconds
highmem bounce pool size: 64 pages
msgmni has been set to 1208
io scheduler noop registered
io scheduler cfq registered (default)
pci 0000:00:1a.0: uhci_check_and_reset_hc: cmd = 0x0000
pci 0000:00:1a.0: Performing full reset
pci 0000:00:1a.1: uhci_check_and_reset_hc: cmd = 0x0000
pci 0000:00:1a.1: Performing full reset
pci 0000:00:1d.0: uhci_check_and_reset_hc: cmd = 0x0000
pci 0000:00:1d.0: Performing full reset
pci 0000:00:1d.1: uhci_check_and_reset_hc: cmd = 0x0000
pci 0000:00:1d.1: Performing full reset
pci 0000:00:1d.2: uhci_check_and_reset_hc: cmd = 0x0000
pci 0000:00:1d.2: Performing full reset
pci 0000:00:1d.3: uhci_check_and_reset_hc: cmd = 0x0000
pci 0000:00:1d.3: Performing full reset
pci 0000:01:00.0: Boot video device
pcieport-driver 0000:00:01.0: setting latency timer to 64
pcieport-driver 0000:00:1c.0: setting latency timer to 64
pcieport-driver 0000:00:1c.5: setting latency timer to 64
hpet_acpi_add: no address or irqs in _CRS
PNP: PS/2 Controller [PNP0303:PS2K] at 0x60,0x64 irq 1
PNP: PS/2 appears to have AUX port disabled, if this is incorrect please boot with i8042.nopnp
serio: i8042 KBD port at 0x60,0x64 irq 1
mice: PS/2 mouse device common for all mice
cpuidle: using governor ladder
cpuidle: using governor menu
TCP cubic registered
Using IPI No-Shortcut mode
Freeing unused kernel memory: 348k freed
input: AT Translated Set 2 keyboard as /class/input/input0
NET: Registered protocol family 1
loop: module loaded
NET: Registered protocol family 17
Uniform Multi-Platform E-IDE driver
ide-gd driver 1.18
libata version 3.00 loaded.
ata_piix 0000:00:1f.2: version 2.13
xen: registering gsi 19 triggering 0 polarity 1
xen: --> irq=19
xen_set_ioapic_routing: irq 19 gsi 19 vector 192 ioapic 0 pin 19 triggering 1 polarity 1
ata_piix 0000:00:1f.2: PCI INT B -> GSI 19 (level, low) -> IRQ 19
ata_piix 0000:00:1f.2: MAP [ P0 -- P1 -- ]
ata_piix 0000:00:1f.2: setting latency timer to 64
scsi0 : ata_piix
scsi1 : ata_piix
ata1: SATA max UDMA/133 cmd 0xc000 ctl 0xbc00 bmdma 0xb480 irq 19
ata2: SATA max UDMA/133 cmd 0xb880 ctl 0xb800 bmdma 0xb488 irq 19
xen: registering gsi 19 triggering 0 polarity 1
xen_allocate_pirq: returning irq 19 for gsi 19
xen: --> irq=19
xen_set_ioapic_routing: irq 19 gsi 19 vector 192 ioapic 0 pin 19 triggering 1 polarity 1
ata_piix 0000:00:1f.5: PCI INT B -> GSI 19 (level, low) -> IRQ 19
ata_piix 0000:00:1f.5: MAP [ P0 -- P1 -- ]
ata_piix 0000:00:1f.5: setting latency timer to 64
scsi2 : ata_piix
scsi3 : ata_piix
ata3: SATA max UDMA/133 cmd 0xb000 ctl 0xac00 bmdma 0xa480 irq 19
ata4: SATA max UDMA/133 cmd 0xa880 ctl 0xa800 bmdma 0xa488 irq 19
ata3: SATA link down (SStatus 0 SControl 300)
ata2: SATA link up 3.0 Gbps (SStatus 123 SControl 300)
ata1: SATA link up 1.5 Gbps (SStatus 113 SControl 300)
ata1.00: ATAPI: TSSTcorpDVD-ROM SH-D163B, SB01, max UDMA/33
ata1.00: configured for UDMA/33
scsi 0:0:0:0: CD-ROM            TSSTcorp DVD-ROM SH-D163B SB01 PQ: 0 ANSI: 5
scsi 0:0:0:0: Attached scsi generic sg0 type 5
ata4: SATA link up 3.0 Gbps (SStatus 123 SControl 300)
ata4.00: ATA-7: SAMSUNG HD753LJ, 1AA01117, max UDMA7
ata4.00: 1465149168 sectors, multi 16: LBA48 NCQ (depth 0/32)
ata2.00: ATA-7: ST3400620AS, 3.AAK, max UDMA/133
ata2.00: 781422768 sectors, multi 16: LBA48 NCQ (depth 0/32)
ata4.00: configured for UDMA/133
ata2.00: configured for UDMA/133
scsi 1:0:0:0: Direct-Access     ATA      ST3400620AS      3.AA PQ: 0 ANSI: 5
sd 1:0:0:0: [sda] 781422768 512-byte logical blocks: (400 GB/372 GiB)
sd 1:0:0:0: [sda] Write Protect is off
sd 1:0:0:0: [sda] Mode Sense: 00 3a 00 00
sd 1:0:0:0: [sda] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
 sda:
sd 1:0:0:0: Attached scsi generic sg1 type 0
scsi 3:0:0:0: Direct-Access     ATA      SAMSUNG HD753LJ  1AA0 PQ: 0 ANSI: 5
sd 3:0:0:0: [sdb] 1465149168 512-byte logical blocks: (750 GB/698 GiB)
sd 3:0:0:0: [sdb] Write Protect is off
sd 3:0:0:0: [sdb] Mode Sense: 00 3a 00 00
sd 3:0:0:0: [sdb] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
 sdb:
sd 3:0:0:0: Attached scsi generic sg2 type 0
 sdb1 sdb2 sdb3 sdb4 < sdb5 sdb6 sda1 sda2 sda3 sda4 < sda5 sdb7 >
 sda6
sd 3:0:0:0: [sdb] Attached SCSI disk
 sda7 sda8 sda9 sda10 sda11 sda12 sda13 sda14 >
sd 1:0:0:0: [sda] Attached SCSI disk
r8169 Gigabit Ethernet driver 2.3LK-NAPI loaded
xen: registering gsi 17 triggering 0 polarity 1
xen_allocate_pirq: returning irq 17 for gsi 17
xen: --> irq=17
xen_set_ioapic_routing: irq 17 gsi 17 vector 160 ioapic 0 pin 17 triggering 1 polarity 1
r8169 0000:03:00.0: PCI INT A -> GSI 17 (level, low) -> IRQ 17
r8169 0000:03:00.0: setting latency timer to 64
eth0: RTL8168b/8111b at 0xedae4000, 00:1d:92:32:6a:05, XID 38000000 IRQ 366
Event-channel device installed.
registering netback
kjournald starting.  Commit interval 5 seconds
EXT3-fs warning: mounting unchecked fs, running e2fsck is recommended
EXT3 FS on sda7, internal journal
ext3_orphan_cleanup: deleting unreferenced inode 515107
EXT3-fs: sda7: 1 orphan inode deleted
EXT3-fs: recovery complete.
EXT3-fs: mounted filesystem with ordered data mode.
device-mapper: ioctl: 4.15.0-ioctl (2009-04-01) initialised: dm-devel@redhat.com
Adding 511992k swap on /dev/system/swap.  Priority:-1 extents:1 across:511992k 
xen: registering gsi 22 triggering 0 polarity 1
xen: --> irq=22
xen_set_ioapic_routing: irq 22 gsi 22 vector 208 ioapic 0 pin 22 triggering 1 polarity 1
HDA Intel 0000:00:1b.0: PCI INT A -> GSI 22 (level, low) -> IRQ 22
HDA Intel 0000:00:1b.0: setting latency timer to 64
ALSA sound/pci/hda/hda_intel.c:2379: chipset global capabilities = 0x4401
ALSA sound/pci/hda/hda_intel.c:866: codec_mask = 0x1
ALSA sound/pci/hda/hda_intel.c:1297: codec #0 probed OK
hda_codec: Unknown model for ALC888, trying auto-probe from BIOS...
ALSA sound/pci/hda/hda_codec.c:3857: autoconfig: line_outs=4 (0x14/0x15/0x16/0x17/0x0)
ALSA sound/pci/hda/hda_codec.c:3861:    speaker_outs=0 (0x0/0x0/0x0/0x0/0x0)
ALSA sound/pci/hda/hda_codec.c:3865:    hp_outs=1 (0x1b/0x0/0x0/0x0/0x0)
ALSA sound/pci/hda/hda_codec.c:3866:    mono: mono_out=0x0
ALSA sound/pci/hda/hda_codec.c:3869:    dig-out=0x1e/0x0
ALSA sound/pci/hda/hda_codec.c:3877:    inputs: mic=0x18, fmic=0x19, line=0x1a, fline=0x0, cd=0x0, aux=0x0
ALSA sound/pci/hda/patch_realtek.c:1181: realtek: No valid SSID, checking pincfg 0x4007f603 for NID 0x1d
ALSA sound/pci/hda/patch_realtek.c:1197: realtek: Enabling init ASM_ID=0xf603 CODEC_ID=10ec0888
ALSA sound/pci/hda/hda_codec.c:1638: Cannot find slave Headphone Playback Volume, skipped
ALSA sound/pci/hda/hda_codec.c:1638: Cannot find slave Speaker Playback Volume, skipped
ALSA sound/pci/hda/hda_codec.c:1638: Cannot find slave Mono Playback Volume, skipped
ALSA sound/pci/hda/hda_codec.c:1638: Cannot find slave Line-Out Playback Volume, skipped
ALSA sound/pci/hda/hda_codec.c:1638: Cannot find slave PCM Playback Volume, skipped
ALSA sound/pci/hda/hda_codec.c:1638: Cannot find slave Speaker Playback Switch, skipped
ALSA sound/pci/hda/hda_codec.c:1638: Cannot find slave Mono Playback Switch, skipped
RPC: Registered udp transport module.
RPC: Registered tcp transport module.
r8169: eth0: link up
r8169: eth0: link up
NET: Registered protocol family 10
r8169: peth0: link up
device peth0 entered promiscuous mode
eth0: port 1(peth0) entering forwarding state
xenbus_probe wake_waiting
xenbus_probe wake_waiting
XENBUS: Unable to read cpu state
XENBUS: Unable to read cpu state
xenbus_probe_devices backend
xenbus_probe_devices failed xenbus_directory
backend_probe_and_watch devices probed ok
backend_probe_and_watch watch add ok ok
backend_probe_and_watch all done
device vif1.0 entered promiscuous mode
eth0: port 2(vif1.0) entering forwarding state
ip_tables: (C) 2000-2006 Netfilter Core Team
blkback: ring-ref 770, event-channel 12, protocol 1 (x86_32-abi)
blkback: ring-ref 771, event-channel 13, protocol 1 (x86_32-abi)
blkback: ring-ref 772, event-channel 14, protocol 1 (x86_32-abi)
blkback: ring-ref 773, event-channel 15, protocol 1 (x86_32-abi)
blkback: ring-ref 774, event-channel 16, protocol 1 (x86_32-abi)
blkback: ring-ref 775, event-channel 17, protocol 1 (x86_32-abi)
blkback: ring-ref 776, event-channel 18, protocol 1 (x86_32-abi)
eth0: no IPv6 routers present
peth0: no IPv6 routers present
vif1.0: no IPv6 routers present
peth0: received packet with own address as source address
peth0: received packet with own address as source address
peth0: received packet with own address as source address

[-- Attachment #3: Type: text/plain, Size: 138 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Re: pvops dom0: no sound after boot; possibly causedby swiotlb

[Ronny.Hegewald]

>Well, since you are volunteering. The problem sounds like the sound card
>allocates a buffer from the region above 4GB and tries to DMA to it.
>Keep in mind that on most machines, when you have 4GB, 768 MB of it are 
>offset
>past the 4GB mark. You have these two mega regions: 0-3.3GB, 4GB-4.7GB,
>the 3.3GB to 4GB is called the PCI hole.

I added some log-messages in the sound-driver and indeed, it allocates memory from above 4GB.

On bare metal and with dom0 based on the forward-ported xen-patches from gentoo the dma-memory is allocated in the 4GB range.

When i checked the dma-memory allocation in the driver i noticed that depending if the sound-card reports that it is 64bit-compatible the driver sets the dma_mask to 64, else to 32 (by calling pci_set_consistent_dma_mask).

So i changed the driver that it always sets the dma-mask to 32 - and the sound worked without the need to start a domU. 

Which left the question why it didnt worked out of the box on pvops-dom0.

So i followed the code for the dma-allocation from the sound-driver and found the problem in the xen-swiotlb code in arch/x86/xen/pci-swiotlb.

Bare metal and the forward-ported dom0 both call dma_alloc_coherent_mask if they try to allocate coherent dma-memory - pvops dom0 doesnt. This call sets the DMA-mask to 24 bit when __GFP_DMA is set, else to 32-bit. 

I guess this is the right fix for pvops-dom0 too? I will sent a proper patch for that soon.

Re: pvops dom0: no sound after boot; possibly causedby swiotlb

[Konrad Rzeszutek Wilk]

On Mon, Feb 01, 2010 at 09:54:02PM +0100, Ronny.Hegewald@online.de wrote:
> >Well, since you are volunteering. The problem sounds like the sound card
> >allocates a buffer from the region above 4GB and tries to DMA to it.
> >Keep in mind that on most machines, when you have 4GB, 768 MB of it are 
> >offset
> >past the 4GB mark. You have these two mega regions: 0-3.3GB, 4GB-4.7GB,
> >the 3.3GB to 4GB is called the PCI hole.
> 
> I added some log-messages in the sound-driver and indeed, it allocates memory from above 4GB.
> 
> On bare metal and with dom0 based on the forward-ported xen-patches from gentoo the dma-memory is allocated in the 4GB range.
> 
> When i checked the dma-memory allocation in the driver i noticed that depending if the sound-card reports that it is 64bit-compatible the driver sets the dma_mask to 64, else to 32 (by calling pci_set_consistent_dma_mask).
> 
> So i changed the driver that it always sets the dma-mask to 32 - and the sound worked without the need to start a domU. 

Good detective job. BTW, what is the name of the driver in the source
code?
> 
> Which left the question why it didnt worked out of the box on pvops-dom0.
> 
> So i followed the code for the dma-allocation from the sound-driver and found the problem in the xen-swiotlb code in arch/x86/xen/pci-swiotlb.
> 
> Bare metal and the forward-ported dom0 both call dma_alloc_coherent_mask if they try to allocate coherent dma-memory - pvops dom0 doesnt. This call sets the DMA-mask to 24 bit when __GFP_DMA is set, else to 32-bit. 

What is your sound-driver detecting the card as? As 64-bit or 32-bit or
none of those?

On pvops we can get away from calling dma_alloc_coherent b/c we have
this piece of logic to determine where the driver can DMA from:

604         if (hwdev != NULL && hwdev->coherent_dma_mask)
605                 mask = hwdev->coherent_dma_mask;
606         else
607                 mask = DMA_BIT_MASK(32);

You mentioned that the sound driver calls the pci_set_consistent_dma_mask. That
function is the one that sets the coherent_dma_mask:

2045 int
2046 pci_set_consistent_dma_mask(struct pci_dev *dev, u64 mask)
2047 {
2048         if (!pci_dma_supported(dev, mask))
2049                 return -EIO;
2050
2051         dev->dev.coherent_dma_mask = mask;

So the 'xen_swiotlb_alloc_coherent' checks if you have the coherent DMA
mask and if not, assumes you have a driver that can only access up to
4GB. While the bare-metal assumes that if the driver doesn't have that
mask , it checks the gfp_t flag and if it has __GFP_DMA make the mask
24-bit, otherwise 32-bit.

The only difference here is that under pvops we behave badly with
devices that have GFP_DMA set and don't have the coherent_dma_mask
(which it does not seem to be the case?).

So is your sound-driver not detecting the card properly and not setting
the coherent_dma_mask and/or dma_mask?

Can you print out both of those entries when the sound driver
calls the 'xen_swiotlb_alloc_coherent' (without setting the flags to 32 forcefully?)

Re: pvops dom0: no sound after boot; possibly causedby swiotlb

[Ronny.Hegewald]

> I am curious - if you dom0 is 64-bit, does the sound card work? 

Yes with 64-bit pvops-dom0 the sound works fine without any patches.

>> 
>> Ooooh. I completly failed to notice that your dom0 was 32-bit.
>> 
>> But having that there would make the mask always be below
>> 4GB, irregardless if the dom0 is 32 or 64-bit. Which is 
>> exactly what it does on bare-metal. 
><sigh> That is actually incorrect. Looking at the
>dma_alloc_coherent_mask I missed the '!' and thought it would set it
>irregardless of what the previous value was. That is not the case.

No, your first statement was correct. It would always return the same mask for 32-bit and 64-bit, because of the 32-bitness of the return-variable 'dma_mask'.

So the following patch maybe would be more appropriate?

--- arch/x86/xen/pci-swiotlb.c-org      2010-02-01 23:04:42.000000000 +0100
+++ arch/x86/xen/pci-swiotlb.c  2010-02-04 02:47:17.000000000 +0100
@@ -602,7 +602,11 @@ xen_swiotlb_alloc_coherent(struct device
        ret = (void *)vstart;

        if (hwdev != NULL && hwdev->coherent_dma_mask)
+#ifdef CONFIG_X86_32
+               mask = dma_alloc_coherent_mask(hwdev, flags);
+#else
                mask = hwdev->coherent_dma_mask;
+#endif

        else
                mask = DMA_BIT_MASK(32);

Re: pvops dom0: no sound after boot; possibly causedby swiotlb

[Konrad Rzeszutek Wilk]

> No, your first statement was correct. It would always return the same mask for 32-bit and 64-bit, because of the 32-bitness of the return-variable 'dma_mask'.
> 
> So the following patch maybe would be more appropriate?

I think it isn't neccessary. The 'dmal_alloc_coherent_mask' returns an
unsigned long. On 32-bit 'unsigned long' is a 4-byte value, and
on 64-bit that is 8-byte value. So I think your previous patch:
http://lists.xensource.com/archives/html/xen-devel/2010-02/msg00038.html
is correct as when it is compiled under 64-bit it would return a 64-bit
value and when compiled under 32-bit, it would return a 32-bit value.
This would mimick the behavior of upstream logic.