Just another PCI VGA passthru report

Just another PCI VGA passthru report

[Pavel Matěja]

Hey guys,
I have new toy and I there is something I want to tell you about it.

Hardware:
AMD Phenom(tm) II X6 1075T Processor
16GB of DDR3 ECC
3x ASUS EAH6850
Intel X25-M SSD 160GB G2
2x USB headset Tactic 3D alpha
1x headphones with jack
3x USB keyboard
3x USB mouse
3x LCD monitor with 4-port USB HUB

Software:
dom0:
xen-unstable.hg with original AMD passthru patch
linux-xen 2.6.32.26-g3147caa with backported patch for Marvell 88E8059
Samba, bind, etc.

3x domU:
Win XP SP3 32b
gplpv_XP_0.11.0.238.msi
10-12_xp32_dd_ccc_enu.exe

domU configuration:
Every domU has:
2 dedicated cores
4GB RAM
8GB partition/disk image
passthru to USB and VGA
1GB NIC
USB keyboard
USB mouse
one domU has PCI passthru to onboard soundcard
two have USB headset

Some problems:
Disk performance is poor even when I use gplpv drivers.

I can't read valid CPU frequency from /proc/cpuinfo

I can't power computer off. It's working normaly when running non XEN
kernel.

Onboard sound:
I had to add kernel parameter
xen-pciback.hide=(00:14.2)
I got error 10 in windows driver when I just unbound it from PCI.

USB:
Sometimes I need normal login before XEN start so I unbind USB controller just 
before start of guest.
I was unable to pass EHCI. Just OHCI was working which caused problems when 
detecting USB devices. It took long time and sometimes some keys on keyboard 
were ignored.
I had to:
a) change PCI resource alignment
cd /sys/bus/pci
echo "00:12.2;00:13.2;00:16.2" > /sys/bus/pci/resource_alignment
b) turn power controllers off
echo 0 > slots/0000:00:12.2/power
echo 0 > slots/0000:00:13.2/power
echo 0 > slots/0000:00:16.2/power
c) rescan PCI
echo 1 > rescan
When I tried to use kernel parameter only first PCI device alignment was 
altered.

VGA:
I can see Windows starting just on primary VGA. Booting other instances causes 
image damage to primary VGA Windows instance. I have to start that domU last.

Current status:
I'm successfully running three instances of Windows XP. Every one has fully 
accelerated VGA and it's own keybord, mouse and soudcard.
I took some video by my phone. Check (no comment, poor quality)
http://tinypic.com/player.php?v=34qtx8p&s=7 XEN boot, WXP start
http://tinypic.com/player.php?v=1679kzb&s=7 3x Counter Strike, very short

What is it good for?
Think about offices, schools,.. You can save lot of money when you get three 
computers for price of two or less. My computer is gaming machine so it's more 
expensive than usual one. But you need VGA PCI passthru.
--
Pavel Mateja

Re: Just another PCI VGA passthru report

[Pavel Matěja]

> Hardware:
> AMD Phenom(tm) II X6 1075T Processor
> 16GB of DDR3 ECC
> 3x ASUS EAH6850
> Intel X25-M SSD 160GB G2
> 2x USB headset Tactic 3D alpha
> 1x headphones with jack
> 3x USB keyboard
> 3x USB mouse
> 3x LCD monitor with 4-port USB HUB
I forgot..
MB is ASUS Crosshair IV Formula
-- 
Pavel Mateja

Re: Just another PCI VGA passthru report

[Konrad Rzeszutek Wilk]

On Sun, Jan 09, 2011 at 02:41:16PM +0100, Pavel Matěja wrote:
> Hey guys,
> I have new toy and I there is something I want to tell you about it.
> 
> Hardware:
> AMD Phenom(tm) II X6 1075T Processor
> 16GB of DDR3 ECC
> 3x ASUS EAH6850
> Intel X25-M SSD 160GB G2
> 2x USB headset Tactic 3D alpha
> 1x headphones with jack
> 3x USB keyboard
> 3x USB mouse
> 3x LCD monitor with 4-port USB HUB
> 
> Software:
> dom0:
> xen-unstable.hg with original AMD passthru patch
> linux-xen 2.6.32.26-g3147caa with backported patch for Marvell 88E8059
> Samba, bind, etc.
> 
> 3x domU:
> Win XP SP3 32b
> gplpv_XP_0.11.0.238.msi
> 10-12_xp32_dd_ccc_enu.exe
> 
> domU configuration:
> Every domU has:
> 2 dedicated cores
> 4GB RAM
> 8GB partition/disk image
> passthru to USB and VGA
> 1GB NIC
> USB keyboard
> USB mouse
> one domU has PCI passthru to onboard soundcard
> two have USB headset
> 
> Some problems:
> Disk performance is poor even when I use gplpv drivers.

As in, about the same as HVM drivers?
> 
> I can't read valid CPU frequency from /proc/cpuinfo

This is in Dom0 I presume? 

> 
> I can't power computer off. It's working normaly when running non XEN
> kernel.

Hmm, can you be more specific. Is it working when you boot the  2.6.32.26-g3147caa
as baremetal?

Re: Just another PCI VGA passthru report

[Pavel Matěja]

On Mon 10. of January 2011 15:32:19 you wrote:
> > Some problems:
> > Disk performance is poor even when I use gplpv drivers.
> 
> As in, about the same as HVM drivers?

I'll measure some numbers for you today afternoon.

> > I can't read valid CPU frequency from /proc/cpuinfo
> 
> This is in Dom0 I presume?

Yes, Dom0 reports:
cpu MHz         : 306230.313
Same kernel running on baremetal (no load):
cpu MHz         : 800.000

> > I can't power computer off. It's working normaly when running non XEN
> > kernel.
> 
> Hmm, can you be more specific. Is it working when you boot the 
> 2.6.32.26-g3147caa as baremetal?

Same kernel booted as baremetal turns power off just fine.
Last message from the kernel booted as dom0 is:
ACPI: preparing to enter system sleep state S5
Disabling non-boot CPUs.
-- 
Pavel Mateja

Re: Just another PCI VGA passthru report

[Konrad Rzeszutek Wilk]

On Tue, Jan 11, 2011 at 09:01:47AM +0100, Pavel Matěja wrote:
> On Mon 10. of January 2011 15:32:19 you wrote:
> > > Some problems:
> > > Disk performance is poor even when I use gplpv drivers.
> > 
> > As in, about the same as HVM drivers?
> 
> I'll measure some numbers for you today afternoon.
> 
> > > I can't read valid CPU frequency from /proc/cpuinfo
> > 
> > This is in Dom0 I presume?
> 
> Yes, Dom0 reports:
> cpu MHz         : 306230.313

Whoa. The timer is way off. Did you specify any specific timer clock
on the bootup line? Are you using the 4.0.1 or xen-unstable hypervisor?

> Same kernel running on baremetal (no load):
> cpu MHz         : 800.000
> 
> > > I can't power computer off. It's working normaly when running non XEN
> > > kernel.

Hmm, that ought to work - unless you are using an older hypervisor.
> > 
> > Hmm, can you be more specific. Is it working when you boot the 
> > 2.6.32.26-g3147caa as baremetal?
> 
> Same kernel booted as baremetal turns power off just fine.
> Last message from the kernel booted as dom0 is:
> ACPI: preparing to enter system sleep state S5
> Disabling non-boot CPUs.

Hmmmm.. That does work on my boxes, so I am surprised you see this. What
do you see on the Xen hypervisor serial line if you pass in 'sync_console loglvl=all'
You should see something from the (XEN): .. about ACPI shutdown.

Re: Just another PCI VGA passthru report

[Pavel Matěja]

On Tue 11. of January 2011 18:42:53 Konrad Rzeszutek Wilk wrote:
> On Tue, Jan 11, 2011 at 09:01:47AM +0100, Pavel Matěja wrote:
> > On Mon 10. of January 2011 15:32:19 you wrote:
> > > > Some problems:
> > > > Disk performance is poor even when I use gplpv drivers.
> > > 
> > > As in, about the same as HVM drivers?
> > 
> > I'll measure some numbers for you today afternoon.
> > 
> > > > I can't read valid CPU frequency from /proc/cpuinfo
> > > 
> > > This is in Dom0 I presume?
> > 
> > Yes, Dom0 reports:
> > cpu MHz         : 306230.313
> 
> Whoa. The timer is way off. Did you specify any specific timer clock
> on the bootup line? Are you using the 4.0.1 or xen-unstable hypervisor?

I'm running xen-unstable.

> > Same kernel running on baremetal (no load):
> > cpu MHz         : 800.000
> > 
> > > > I can't power computer off. It's working normaly when running non XEN
> > > > kernel.
> 
> Hmm, that ought to work - unless you are using an older hypervisor.

Hmm, I'm not. I'll update my tree and rebuild.

> > > Hmm, can you be more specific. Is it working when you boot the
> > > 2.6.32.26-g3147caa as baremetal?
> > 
> > Same kernel booted as baremetal turns power off just fine.
> > Last message from the kernel booted as dom0 is:
> > ACPI: preparing to enter system sleep state S5
> > Disabling non-boot CPUs.
> 
> Hmmmm.. That does work on my boxes, so I am surprised you see this. What
> do you see on the Xen hypervisor serial line if you pass in 'sync_console
> loglvl=all' You should see something from the (XEN): .. about ACPI
> shutdown.

Where should I get serial line?
I have nothing special in my GRUB config.

menuentry 'Debian GNU/Linux, with Linux 2.6.32.26-g3147caa-dirty and XEN 4.1-
unstable' --class debian --class gnu-linux --class gnu --class os --class xen 
{
        insmod part_msdos
        insmod ext2
        set root='(hd0,msdos1)'
        search --no-floppy --fs-uuid --set 9c94fbb4-64a7-438c-aa7a-35ca818bc312
        echo    'Loading Linux 2.6.32.26-g3147caa-dirty ...'
        multiboot       /xen-4.1-unstable.gz placeholder
        module  /vmlinuz-2.6.32.26-g3147caa-dirty placeholder 
root=/dev/mapper/lemra-old_root ro  panic=15 xen-pciback.hide=(00:14.2) 
        echo    'Loading initial ramdisk ...'
        module  /initrd.img-2.6.32.26-g3147caa-dirty
}

I'll update kernel to the latest git version and retry.
-- 
Pavel Mateja

Re: Just another PCI VGA passthru report

[Konrad Rzeszutek Wilk]

> Where should I get serial line?


http://wiki.xensource.com/xenwiki/XenSerialConsole
> I have nothing special in my GRUB config.
> 
> menuentry 'Debian GNU/Linux, with Linux 2.6.32.26-g3147caa-dirty and XEN 4.1-
> unstable' --class debian --class gnu-linux --class gnu --class os --class xen 
> {
>         insmod part_msdos
>         insmod ext2
>         set root='(hd0,msdos1)'
>         search --no-floppy --fs-uuid --set 9c94fbb4-64a7-438c-aa7a-35ca818bc312
>         echo    'Loading Linux 2.6.32.26-g3147caa-dirty ...'
>         multiboot       /xen-4.1-unstable.gz placeholder
>         module  /vmlinuz-2.6.32.26-g3147caa-dirty placeholder 
> root=/dev/mapper/lemra-old_root ro  panic=15 xen-pciback.hide=(00:14.2) 
>         echo    'Loading initial ramdisk ...'
>         module  /initrd.img-2.6.32.26-g3147caa-dirty
> }
> 
> I'll update kernel to the latest git version and retry.
> -- 
> Pavel Mateja
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel

Re: Just another PCI VGA passthru report

[Pavel Mateja]

> > Where should I get serial line?
> 
> http://wiki.xensource.com/xenwiki/XenSerialConsole

Sorry, it was rhetorical question. I don't have required hw.
I'll try to get some PCI Serial card..
-- 
Pavel Mateja