A query on frame buffers

A query on frame buffers

[Prasad Joshi]

Hello All,

This is probably my first mail on this mailing list. I am working on a
university project on Nvidia or any GPU Pass-through in KVM. I have a
question related to framebuffer.

I was reading few documents on XEN VGA pass-through, the document says
the first step for pass-through is mapping the framebuffers in the VM
at specific address, specifically speaking 0xA0000 till 0xC0000.

I am not sure how the new framebuffer interface /dev/fbn works. Would
I need to map the host framebuffer at specific address in the Guest
VM?
Or should it be done differently with the help of /dev/fbn interface?

Please share your thoughts. Any pointers will be helpful. Please add
in CC as I am not subscribed to this mailing list.

Thanks and Regards,
Prasad

Re: A query on frame buffers

[Konrad Rzeszutek Wilk]

On Tue, Dec 21, 2010 at 04:51:04AM +0000, Prasad Joshi wrote:
> Hello All,
> 
> This is probably my first mail on this mailing list. I am working on a
> university project on Nvidia or any GPU Pass-through in KVM. I have a

You picking the hard ones first, eh? I would suggest you do Intel
or ATI as they have actually posted patches for this.

> question related to framebuffer.
> 
> I was reading few documents on XEN VGA pass-through, the document says
> the first step for pass-through is mapping the framebuffers in the VM
> at specific address, specifically speaking 0xA0000 till 0xC0000.

I would suggest you look at the code - there have been some patches
posted by AMD engineers on xen-devel for this.

Re: A query on frame buffers

[Prasad Joshi]

On Mon, Dec 27, 2010 at 3:30 PM, Konrad Rzeszutek Wilk
<konrad.wilk@oracle.com> wrote:
> On Tue, Dec 21, 2010 at 04:51:04AM +0000, Prasad Joshi wrote:
>> Hello All,
>>
>> This is probably my first mail on this mailing list. I am working on a
>> university project on Nvidia or any GPU Pass-through in KVM. I have a
>
> You picking the hard ones first, eh? I would suggest you do Intel
> or ATI as they have actually posted patches for this.

Thanks a lot for your reply.

Yes indeed, I started with the ATI Radeon RV370 card. Passing the
Nvidia card is the ultimate aim of the project.

>
>> question related to framebuffer.
>>
>> I was reading few documents on XEN VGA pass-through, the document says
>> the first step for pass-through is mapping the framebuffers in the VM
>> at specific address, specifically speaking 0xA0000 till 0xC0000.
>
> I would suggest you look at the code - there have been some patches
> posted by AMD engineers on xen-devel for this.
>

Yes I am looking at those patches as well, including some research
papers on the same topic.

Here is what I have tried so far.
I have 2 graphics cards connected to my machine
1. Nvidia: This is being used by host machine.
2. ATI: Want to pass-through this card to the VM.

I tried to pass-through the ATI card to VM. First, there was a problem
with ROM BIOS. Then I passed a correct ROM BIOS to QEMU-KVM. Now when
I start the VM with ATI card in pass-through mode.

I see some garbage output on the monitor attached to the ATI card. I
also saw following messages in the system log.

[    2.162294] [drm] Initialized drm 1.1.0 20060810
[    2.459594] [drm] radeon defaulting to kernel modesetting.
[    2.459596] [drm] radeon kernel modesetting enabled.
[    2.766698] radeon 0000:00:04.0: PCI INT A -> Link[LNKD] -> GSI 10
(level, high) -> IRQ 10
[    2.766734] radeon 0000:00:04.0: setting latency timer to 64
[    2.783512] [drm] initializing kernel modesetting (RV380 0x1002:0x5B64).
[    2.792407] [drm] register mmio base: 0x40000000
[    2.792408] [drm] register mmio size: 65536
[    2.797177] [drm] Generation 2 PCI interface, using max accessible memory
[    2.797275] radeon 0000:00:04.0: VRAM: 128M 0x00000000F8000000 -
0x00000000FFFFFFFF (128M used)
[    2.797284] radeon 0000:00:04.0: GTT: 512M 0x00000000D8000000 -
0x00000000F7FFFFFF
[    2.798370] radeon 0000:00:04.0: irq 40 for MSI/MSI-X
[    2.870703] radeon 0000:00:04.0: radeon: using MSI.
[    3.151162] [drm] radeon: irq initialized.
[    3.151539] [drm] Detected VRAM RAM\x128M, BAR\x128M
[    3.151541] [drm] RAM width 128bits DDR
[    3.151610] [drm] radeon: 128M of VRAM memory ready
[    3.151611] [drm] radeon: 512M of GTT memory ready.
[    3.151627] [drm] GART: num cpu pages 131072, num gpu pages 131072
[    3.152701] [drm] radeon: 1 quad pipes, 1 Z pipes initialized.
[    3.544943] [drm] PCIE GART of 512M enabled (table at 0xF8040000).
[    3.548479] radeon 0000:00:04.0: WB enabled
[    3.549020] [drm] Loading R300 Microcode
[    3.554278] [drm] radeon: ring at 0x00000000D8001000
[    3.778476] [drm:r100_ring_test] *ERROR* radeon: ring test failed
(sracth(0x15E4)=0xCAFEDEAD)
[    3.781010] [drm:r100_cp_init] *ERROR* radeon: cp isn't working (-22).
[    3.782542] radeon 0000:00:04.0: failled initializing CP (-22).
[    3.784006] radeon 0000:00:04.0: Disabling GPU acceleration
[    3.793709] [drm] radeon: cp finalized
[    3.834875] radeon 0000:00:04.0: ffff8800320b4c00 unpin not necessary
[    3.841759] [drm] Radeon Display Connectors
[    3.841766] [drm] Connector 0:
[    3.841771] [drm]   VGA
[    3.841777] [drm]   DDC: 0x60 0x60 0x60 0x60 0x60 0x60 0x60 0x60
[    3.841781] [drm]   Encoders:
[    3.841785] [drm]     CRT1: INTERNAL_DAC1
[    3.841788] [drm] Connector 1:
[    3.841791] [drm]   DVI-I
[    3.841794] [drm]   HPD1
[    3.841799] [drm]   DDC: 0x64 0x64 0x64 0x64 0x64 0x64 0x64 0x64
[    3.841803] [drm]   Encoders:
[    3.841806] [drm]     CRT2: INTERNAL_DAC2
[    3.841809] [drm]     DFP1: INTERNAL_TMDS1
[    6.158753] [drm] fb mappable at 0xF8040000
[    6.158756] [drm] vram apper at 0xF8000000
[    6.158757] [drm] size 5242880
[    6.158758] [drm] fb depth is 24
[    6.158759] [drm]    pitch is 5120
[   13.914196] fb0: radeondrmfb frame buffer device
[   13.914198] drm: registered panic notifier
[   13.914683] [drm] Initialized radeon 2.7.0 20080528 for
0000:00:04.0 on minor 0

The errors

[    3.778476] [drm:r100_ring_test] *ERROR* radeon: ring test failed
(sracth(0x15E4)=0xCAFEDEAD)
[    3.781010] [drm:r100_cp_init] *ERROR* radeon: cp isn't working (-22).
[    3.782542] radeon 0000:00:04.0: failled initializing CP (-22).
[    3.784006] radeon 0000:00:04.0: Disabling GPU acceleration

Seem to be disabling GPU functionality. For now, I am interested in
Graphics functionality of the ATI card and not GPGPU functionality.

I will appreciate if you can help me with some information on solving
this problem. Let me know if I should be asking this question on
another forum. Please keep me in CC.

Thanks and Regards,
Prasad

Re: A query on frame buffers

[Michel Dänzer]

On Fre, 2011-01-07 at 02:42 +0000, Prasad Joshi wrote: 
> On Mon, Dec 27, 2010 at 3:30 PM, Konrad Rzeszutek Wilk
> <konrad.wilk@oracle.com> wrote:
> > On Tue, Dec 21, 2010 at 04:51:04AM +0000, Prasad Joshi wrote:
> >> Hello All,
> >>
> >> This is probably my first mail on this mailing list. I am working on a
> >> university project on Nvidia or any GPU Pass-through in KVM. I have a
> >
> > You picking the hard ones first, eh? I would suggest you do Intel
> > or ATI as they have actually posted patches for this.
> 
> Thanks a lot for your reply.
> 
> Yes indeed, I started with the ATI Radeon RV370 card. Passing the
> Nvidia card is the ultimate aim of the project.
> 
> >
> >> question related to framebuffer.
> >>
> >> I was reading few documents on XEN VGA pass-through, the document says
> >> the first step for pass-through is mapping the framebuffers in the VM
> >> at specific address, specifically speaking 0xA0000 till 0xC0000.
> >
> > I would suggest you look at the code - there have been some patches
> > posted by AMD engineers on xen-devel for this.
> >
> 
> Yes I am looking at those patches as well, including some research
> papers on the same topic.
> 
> Here is what I have tried so far.
> I have 2 graphics cards connected to my machine
> 1. Nvidia: This is being used by host machine.
> 2. ATI: Want to pass-through this card to the VM.
> 
> I tried to pass-through the ATI card to VM. First, there was a problem
> with ROM BIOS. Then I passed a correct ROM BIOS to QEMU-KVM. Now when
> I start the VM with ATI card in pass-through mode.
> 
> I see some garbage output on the monitor attached to the ATI card. I
> also saw following messages in the system log.
> 
> [    2.162294] [drm] Initialized drm 1.1.0 20060810
> [    2.459594] [drm] radeon defaulting to kernel modesetting.
> [    2.459596] [drm] radeon kernel modesetting enabled.
> [    2.766698] radeon 0000:00:04.0: PCI INT A -> Link[LNKD] -> GSI 10
> (level, high) -> IRQ 10
> [    2.766734] radeon 0000:00:04.0: setting latency timer to 64
> [    2.783512] [drm] initializing kernel modesetting (RV380 0x1002:0x5B64).
> [    2.792407] [drm] register mmio base: 0x40000000
> [    2.792408] [drm] register mmio size: 65536
> [    2.797177] [drm] Generation 2 PCI interface, using max accessible memory
> [    2.797275] radeon 0000:00:04.0: VRAM: 128M 0x00000000F8000000 -
> 0x00000000FFFFFFFF (128M used)
> [    2.797284] radeon 0000:00:04.0: GTT: 512M 0x00000000D8000000 -
> 0x00000000F7FFFFFF
> [    2.798370] radeon 0000:00:04.0: irq 40 for MSI/MSI-X
> [    2.870703] radeon 0000:00:04.0: radeon: using MSI.
> [    3.151162] [drm] radeon: irq initialized.
> [    3.151539] [drm] Detected VRAM RAM\x128M, BAR\x128M
> [    3.151541] [drm] RAM width 128bits DDR
> [    3.151610] [drm] radeon: 128M of VRAM memory ready
> [    3.151611] [drm] radeon: 512M of GTT memory ready.
> [    3.151627] [drm] GART: num cpu pages 131072, num gpu pages 131072
> [    3.152701] [drm] radeon: 1 quad pipes, 1 Z pipes initialized.
> [    3.544943] [drm] PCIE GART of 512M enabled (table at 0xF8040000).
> [    3.548479] radeon 0000:00:04.0: WB enabled
> [    3.549020] [drm] Loading R300 Microcode
> [    3.554278] [drm] radeon: ring at 0x00000000D8001000
> [    3.778476] [drm:r100_ring_test] *ERROR* radeon: ring test failed
> (sracth(0x15E4)=0xCAFEDEAD)
> [    3.781010] [drm:r100_cp_init] *ERROR* radeon: cp isn't working (-22).
> [    3.782542] radeon 0000:00:04.0: failled initializing CP (-22).
> [    3.784006] radeon 0000:00:04.0: Disabling GPU acceleration
> [    3.793709] [drm] radeon: cp finalized
> [    3.834875] radeon 0000:00:04.0: ffff8800320b4c00 unpin not necessary
> [    3.841759] [drm] Radeon Display Connectors
> [    3.841766] [drm] Connector 0:
> [    3.841771] [drm]   VGA
> [    3.841777] [drm]   DDC: 0x60 0x60 0x60 0x60 0x60 0x60 0x60 0x60
> [    3.841781] [drm]   Encoders:
> [    3.841785] [drm]     CRT1: INTERNAL_DAC1
> [    3.841788] [drm] Connector 1:
> [    3.841791] [drm]   DVI-I
> [    3.841794] [drm]   HPD1
> [    3.841799] [drm]   DDC: 0x64 0x64 0x64 0x64 0x64 0x64 0x64 0x64
> [    3.841803] [drm]   Encoders:
> [    3.841806] [drm]     CRT2: INTERNAL_DAC2
> [    3.841809] [drm]     DFP1: INTERNAL_TMDS1
> [    6.158753] [drm] fb mappable at 0xF8040000
> [    6.158756] [drm] vram apper at 0xF8000000
> [    6.158757] [drm] size 5242880
> [    6.158758] [drm] fb depth is 24
> [    6.158759] [drm]    pitch is 5120
> [   13.914196] fb0: radeondrmfb frame buffer device
> [   13.914198] drm: registered panic notifier
> [   13.914683] [drm] Initialized radeon 2.7.0 20080528 for
> 0000:00:04.0 on minor 0
> 
> The errors
> 
> [    3.778476] [drm:r100_ring_test] *ERROR* radeon: ring test failed
> (sracth(0x15E4)=0xCAFEDEAD)
> [    3.781010] [drm:r100_cp_init] *ERROR* radeon: cp isn't working (-22).
> [    3.782542] radeon 0000:00:04.0: failled initializing CP (-22).
> [    3.784006] radeon 0000:00:04.0: Disabling GPU acceleration
> 
> Seem to be disabling GPU functionality. For now, I am interested in
> Graphics functionality of the ATI card and not GPGPU functionality.
> 
> I will appreciate if you can help me with some information on solving
> this problem. Let me know if I should be asking this question on
> another forum. Please keep me in CC.

Looks like dri-devel material, added to CC.

As a first guess, maybe the system memory pages accessed by the GPU GART
aren't the same ones intended for this by the VM.


-- 
Earthling Michel Dänzer           |                http://www.vmware.com
Libre software enthusiast         |          Debian, X and DRI developer

Re: A query on frame buffers

[Konrad Rzeszutek Wilk]

> > The errors
> > 
> > [    3.778476] [drm:r100_ring_test] *ERROR* radeon: ring test failed
> > (sracth(0x15E4)=0xCAFEDEAD)
> > [    3.781010] [drm:r100_cp_init] *ERROR* radeon: cp isn't working (-22).
> > [    3.782542] radeon 0000:00:04.0: failled initializing CP (-22).
> > [    3.784006] radeon 0000:00:04.0: Disabling GPU acceleration
> > 
> > Seem to be disabling GPU functionality. For now, I am interested in
> > Graphics functionality of the ATI card and not GPGPU functionality.
> > 
> > I will appreciate if you can help me with some information on solving
> > this problem. Let me know if I should be asking this question on
> > another forum. Please keep me in CC.
> 
> Looks like dri-devel material, added to CC.
> 
> As a first guess, maybe the system memory pages accessed by the GPU GART
> aren't the same ones intended for this by the VM.

I actually have patches for that (sorry for the late reply).

Try:
git://git.kernel.org/pub/scm/linux/kernel/git/konrad/xen.git devel/ttm.pci-api.v3

But I am not sure if they would make any difference here. You said
you are using KVM-QEMU - you must also be using an IOMMU? Does the IOMMU
give you any errors?

Re: A query on frame buffers

[Prasad Joshi]

2011/1/7 Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>:
>> > The errors
>> >
>> > [    3.778476] [drm:r100_ring_test] *ERROR* radeon: ring test failed
>> > (sracth(0x15E4)=0xCAFEDEAD)
>> > [    3.781010] [drm:r100_cp_init] *ERROR* radeon: cp isn't working (-22).
>> > [    3.782542] radeon 0000:00:04.0: failled initializing CP (-22).
>> > [    3.784006] radeon 0000:00:04.0: Disabling GPU acceleration
>> >
>> > Seem to be disabling GPU functionality. For now, I am interested in
>> > Graphics functionality of the ATI card and not GPGPU functionality.
>> >
>> > I will appreciate if you can help me with some information on solving
>> > this problem. Let me know if I should be asking this question on
>> > another forum. Please keep me in CC.
>>
>> Looks like dri-devel material, added to CC.
>>
>> As a first guess, maybe the system memory pages accessed by the GPU GART
>> aren't the same ones intended for this by the VM.
>
> I actually have patches for that (sorry for the late reply).
>
> Try:
> git://git.kernel.org/pub/scm/linux/kernel/git/konrad/xen.git devel/ttm.pci-api.v3
>
> But I am not sure if they would make any difference here. You said
> you are using KVM-QEMU - you must also be using an IOMMU? Does the IOMMU
> give you any errors?
>

Yes I want to pass-through the ATI Radeon card to VM in KVM. I have
enabled the IOMMU support and it works perfectly alright. I tried
passing through a network card it worked.

Should I try the patch you suggested?

>

Re: A query on frame buffers

[Konrad Rzeszutek Wilk]

On Fri, Jan 07, 2011 at 05:47:59PM +0000, Prasad Joshi wrote:
> 2011/1/7 Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>:
> >> > The errors
> >> >
> >> > [    3.778476] [drm:r100_ring_test] *ERROR* radeon: ring test failed
> >> > (sracth(0x15E4)=0xCAFEDEAD)
> >> > [    3.781010] [drm:r100_cp_init] *ERROR* radeon: cp isn't working (-22).
> >> > [    3.782542] radeon 0000:00:04.0: failled initializing CP (-22).
> >> > [    3.784006] radeon 0000:00:04.0: Disabling GPU acceleration
> >> >
> >> > Seem to be disabling GPU functionality. For now, I am interested in
> >> > Graphics functionality of the ATI card and not GPGPU functionality.
> >> >
> >> > I will appreciate if you can help me with some information on solving
> >> > this problem. Let me know if I should be asking this question on
> >> > another forum. Please keep me in CC.
> >>
> >> Looks like dri-devel material, added to CC.
> >>
> >> As a first guess, maybe the system memory pages accessed by the GPU GART
> >> aren't the same ones intended for this by the VM.
> >
> > I actually have patches for that (sorry for the late reply).
> >
> > Try:
> > git://git.kernel.org/pub/scm/linux/kernel/git/konrad/xen.git devel/ttm.pci-api.v3
> >
> > But I am not sure if they would make any difference here. You said
> > you are using KVM-QEMU - you must also be using an IOMMU? Does the IOMMU
> > give you any errors?
> >
> 
> Yes I want to pass-through the ATI Radeon card to VM in KVM. I have
> enabled the IOMMU support and it works perfectly alright. I tried
> passing through a network card it worked.
> 
> Should I try the patch you suggested?

Go for it. I am still at lost why you would see those errors unless..
well, what is the DMA code that gets turned on when you run your guest?
What do you see for 'PCI-DMA' ? Is it bounce buffer or swiotlb?
How much memory do you pass to the guest? less than 4GB?

Re: A query on frame buffers

[Prasad Joshi]

On Fri, Jan 7, 2011 at 7:24 PM, Konrad Rzeszutek Wilk
<konrad.wilk@oracle.com> wrote:
> On Fri, Jan 07, 2011 at 05:47:59PM +0000, Prasad Joshi wrote:
>> 2011/1/7 Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>:
>> >> > The errors
>> >> >
>> >> > [    3.778476] [drm:r100_ring_test] *ERROR* radeon: ring test failed
>> >> > (sracth(0x15E4)=0xCAFEDEAD)
>> >> > [    3.781010] [drm:r100_cp_init] *ERROR* radeon: cp isn't working (-22).
>> >> > [    3.782542] radeon 0000:00:04.0: failled initializing CP (-22).
>> >> > [    3.784006] radeon 0000:00:04.0: Disabling GPU acceleration
>> >> >
>> >> > Seem to be disabling GPU functionality. For now, I am interested in
>> >> > Graphics functionality of the ATI card and not GPGPU functionality.
>> >> >
>> >> > I will appreciate if you can help me with some information on solving
>> >> > this problem. Let me know if I should be asking this question on
>> >> > another forum. Please keep me in CC.
>> >>
>> >> Looks like dri-devel material, added to CC.
>> >>
>> >> As a first guess, maybe the system memory pages accessed by the GPU GART
>> >> aren't the same ones intended for this by the VM.
>> >
>> > I actually have patches for that (sorry for the late reply).
>> >
>> > Try:
>> > git://git.kernel.org/pub/scm/linux/kernel/git/konrad/xen.git devel/ttm.pci-api.v3

I could access the git repository, but was not able to get those
patches form the repository. Frankly speaking, I don't know how to get
those patches.

>> >
>> > But I am not sure if they would make any difference here. You said
>> > you are using KVM-QEMU - you must also be using an IOMMU? Does the IOMMU
>> > give you any errors?
>> >
>>
>> Yes I want to pass-through the ATI Radeon card to VM in KVM. I have
>> enabled the IOMMU support and it works perfectly alright. I tried
>> passing through a network card it worked.
>>
>> Should I try the patch you suggested?
>
> Go for it. I am still at lost why you would see those errors unless..
> well, what is the DMA code that gets turned on when you run your guest?
> What do you see for 'PCI-DMA' ? Is it bounce buffer or swiotlb?
> How much memory do you pass to the guest? less than 4GB?
>
I am trying these things on Uni server, I will let you know the
answers by tomorrow.

Thanks and Regards,
Prasad

Re: A query on frame buffers

[Prasad Joshi]

[-- Attachment #1: Type: text/plain, Size: 3499 bytes --]

On Fri, Jan 7, 2011 at 7:24 PM, Konrad Rzeszutek Wilk
<konrad.wilk@oracle.com> wrote:
> On Fri, Jan 07, 2011 at 05:47:59PM +0000, Prasad Joshi wrote:
>> 2011/1/7 Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>:
>> >> > The errors
>> >> >
>> >> > [    3.778476] [drm:r100_ring_test] *ERROR* radeon: ring test failed
>> >> > (sracth(0x15E4)=0xCAFEDEAD)
>> >> > [    3.781010] [drm:r100_cp_init] *ERROR* radeon: cp isn't working (-22).
>> >> > [    3.782542] radeon 0000:00:04.0: failled initializing CP (-22).
>> >> > [    3.784006] radeon 0000:00:04.0: Disabling GPU acceleration
>> >> >
>> >> > Seem to be disabling GPU functionality. For now, I am interested in
>> >> > Graphics functionality of the ATI card and not GPGPU functionality.
>> >> >
>> >> > I will appreciate if you can help me with some information on solving
>> >> > this problem. Let me know if I should be asking this question on
>> >> > another forum. Please keep me in CC.
>> >>
>> >> Looks like dri-devel material, added to CC.
>> >>
>> >> As a first guess, maybe the system memory pages accessed by the GPU GART
>> >> aren't the same ones intended for this by the VM.
>> >
>> > I actually have patches for that (sorry for the late reply).
>> >
>> > Try:
>> > git://git.kernel.org/pub/scm/linux/kernel/git/konrad/xen.git devel/ttm.pci-api.v3
>> >
>> > But I am not sure if they would make any difference here. You said
>> > you are using KVM-QEMU - you must also be using an IOMMU? Does the IOMMU
>> > give you any errors?
>> >
>>
>> Yes I want to pass-through the ATI Radeon card to VM in KVM. I have
>> enabled the IOMMU support and it works perfectly alright. I tried
>> passing through a network card it worked.
>>
>> Should I try the patch you suggested?
>
> Go for it. I am still at lost why you would see those errors unless..

Tried your patches on Guest machine, it did not help. I am attaching
the messages from guest machine.

> well, what is the DMA code that gets turned on when you run your guest?

Looking into the qemu-kvm code to find more information about DMA.

> What do you see for 'PCI-DMA' ?

> Is it bounce buffer or swiotlb?

It is the default configuration. I did not pass swiotlb parameter on
kernel boot line.

The host kernel is booted with iommu=1.

> How much memory do you pass to the guest? less than 4GB?

root@prasad-kvm:~/VMDisks# qemu-system-x86_64 -hda
Ubuntu-10.10-amd64.img -m 1024M -device pci-assign,host=02:00.0

I am passing 1GB memory to the guest machine.

>

I have 2 graphics cards (Nvidia and ATI) attached to my system. The
Nvidia graphics card is used by host machine and I am trying to assign
the ATI graphics card to the Guest machine.

When I unbind the ATI graphics card from the host machine I see these
messages on the host machine
Jan 10 11:59:15 prasad-kvm kernel: [ 3396.912767] pci-stub: invalid id string ""
Jan 10 11:59:59 prasad-kvm kernel: [ 3441.180601] pci-stub
0000:02:00.1: claimed by stub
Jan 10 12:00:08 prasad-kvm kernel: [ 3449.901870] [drm] radeon:
finishing device.
Jan 10 12:00:08 prasad-kvm kernel: [ 3449.902161] [drm] radeon: cp finalized
Jan 10 12:00:08 prasad-kvm kernel: [ 3449.903435] radeon 0000:02:00.0:
ffff88022ae3d200 unpin not necessary
Jan 10 12:00:08 prasad-kvm kernel: [ 3449.903490] [drm] radeon: ttm finalized
Jan 10 12:00:08 prasad-kvm kernel: [ 3449.903551] vga_switcheroo: disabled

Thanks a lot for helping me out.

Thanks and Regards,
Prasad

[-- Attachment #2: guest_msges --]
[-- Type: application/octet-stream, Size: 27341 bytes --]

[    0.000000] Initializing cgroup subsys cpuset
[    0.000000] Initializing cgroup subsys cpu
[    0.000000] Linux version 2.6.37+ (root@prasad-virtual-machine) (gcc version 4.4.5 (Ubuntu/Linaro 4.4.4-14ubuntu5) ) #2 SMP Mon Jan 10 11:32:37 GMT 2011
[    0.000000] Command line: BOOT_IMAGE=/boot/vmlinuz-2.6.37+ root=/dev/sda1 ro
[    0.000000] BIOS-provided physical RAM map:
[    0.000000]  BIOS-e820: 0000000000000000 - 000000000009f400 (usable)
[    0.000000]  BIOS-e820: 000000000009f400 - 00000000000a0000 (reserved)
[    0.000000]  BIOS-e820: 00000000000f0000 - 0000000000100000 (reserved)
[    0.000000]  BIOS-e820: 0000000000100000 - 000000003fffd000 (usable)
[    0.000000]  BIOS-e820: 000000003fffd000 - 0000000040000000 (reserved)
[    0.000000]  BIOS-e820: 00000000feffc000 - 00000000ff000000 (reserved)
[    0.000000]  BIOS-e820: 00000000fffc0000 - 0000000100000000 (reserved)
[    0.000000] NX (Execute Disable) protection: active
[    0.000000] DMI 2.4 present.
[    0.000000] DMI: /Bochs, BIOS Bochs 01/01/2007
[    0.000000] e820 update range: 0000000000000000 - 0000000000010000 (usable) ==> (reserved)
[    0.000000] e820 remove range: 00000000000a0000 - 0000000000100000 (usable)
[    0.000000] No AGP bridge found
[    0.000000] last_pfn = 0x3fffd max_arch_pfn = 0x400000000
[    0.000000] MTRR default type: write-back
[    0.000000] MTRR fixed ranges enabled:
[    0.000000]   00000-9FFFF write-back
[    0.000000]   A0000-BFFFF uncachable
[    0.000000]   C0000-FFFFF write-protect
[    0.000000] MTRR variable ranges enabled:
[    0.000000]   0 base 00E0000000 mask FFE0000000 uncachable
[    0.000000]   1 disabled
[    0.000000]   2 disabled
[    0.000000]   3 disabled
[    0.000000]   4 disabled
[    0.000000]   5 disabled
[    0.000000]   6 disabled
[    0.000000]   7 disabled
[    0.000000] x86 PAT enabled: cpu 0, old 0x70406, new 0x7010600070106
[    0.000000] found SMP MP-table at [ffff8800000fdbf0] fdbf0
[    0.000000] Scanning 0 areas for low memory corruption
[    0.000000] initial memory mapped : 0 - 20000000
[    0.000000] init_memory_mapping: 0000000000000000-000000003fffd000
[    0.000000]  0000000000 - 003fe00000 page 2M
[    0.000000]  003fe00000 - 003fffd000 page 4k
[    0.000000] kernel direct mapping tables up to 3fffd000 @ 1fffd000-20000000
[    0.000000] ACPI: RSDP 00000000000fdba0 00014 (v00 BOCHS )
[    0.000000] ACPI: RSDT 000000003fffdce0 00034 (v01 BOCHS  BXPCRSDT 00000001 BXPC 00000001)
[    0.000000] ACPI: FACP 000000003ffffe70 00074 (v01 BOCHS  BXPCFACP 00000001 BXPC 00000001)
[    0.000000] ACPI: DSDT 000000003fffdee0 01F15 (v01   BXPC   BXDSDT 00000001 INTL 20090123)
[    0.000000] ACPI: FACS 000000003ffffe00 00040
[    0.000000] ACPI: SSDT 000000003fffde40 0009E (v01 BOCHS  BXPCSSDT 00000001 BXPC 00000001)
[    0.000000] ACPI: APIC 000000003fffdd60 00072 (v01 BOCHS  BXPCAPIC 00000001 BXPC 00000001)
[    0.000000] ACPI: HPET 000000003fffdd20 00038 (v01 BOCHS  BXPCHPET 00000001 BXPC 00000001)
[    0.000000] ACPI: Local APIC address 0xfee00000
[    0.000000] No NUMA configuration found
[    0.000000] Faking a node at 0000000000000000-000000003fffd000
[    0.000000] Initmem setup node 0 0000000000000000-000000003fffd000
[    0.000000]   NODE_DATA [000000003fff8000 - 000000003fffcfff]
[    0.000000] kvm-clock: Using msrs 12 and 11
[    0.000000] kvm-clock: cpu 0, msr 0:1ab9941, boot clock
[    0.000000]  [ffffea0000000000-ffffea0000dfffff] PMD -> [ffff88003e600000-ffff88003f3fffff] on node 0
[    0.000000] Zone PFN ranges:
[    0.000000]   DMA      0x00000010 -> 0x00001000
[    0.000000]   DMA32    0x00001000 -> 0x00100000
[    0.000000]   Normal   empty
[    0.000000] Movable zone start PFN for each node
[    0.000000] early_node_map[2] active PFN ranges
[    0.000000]     0: 0x00000010 -> 0x0000009f
[    0.000000]     0: 0x00000100 -> 0x0003fffd
[    0.000000] On node 0 totalpages: 262028
[    0.000000]   DMA zone: 56 pages used for memmap
[    0.000000]   DMA zone: 6 pages reserved
[    0.000000]   DMA zone: 3921 pages, LIFO batch:0
[    0.000000]   DMA32 zone: 3528 pages used for memmap
[    0.000000]   DMA32 zone: 254517 pages, LIFO batch:31
[    0.000000] ACPI: PM-Timer IO Port: 0xb008
[    0.000000] ACPI: Local APIC address 0xfee00000
[    0.000000] ACPI: LAPIC (acpi_id[0x00] lapic_id[0x00] enabled)
[    0.000000] ACPI: IOAPIC (id[0x01] address[0xfec00000] gsi_base[0])
[    0.000000] IOAPIC[0]: apic_id 1, version 17, address 0xfec00000, GSI 0-23
[    0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)
[    0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)
[    0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)
[    0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)
[    0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)
[    0.000000] ACPI: IRQ0 used by override.
[    0.000000] ACPI: IRQ2 used by override.
[    0.000000] ACPI: IRQ5 used by override.
[    0.000000] ACPI: IRQ9 used by override.
[    0.000000] ACPI: IRQ10 used by override.
[    0.000000] ACPI: IRQ11 used by override.
[    0.000000] Using ACPI (MADT) for SMP configuration information
[    0.000000] ACPI: HPET id: 0x8086a201 base: 0xfed00000
[    0.000000] SMP: Allowing 1 CPUs, 0 hotplug CPUs
[    0.000000] nr_irqs_gsi: 40
[    0.000000] PM: Registered nosave memory: 000000000009f000 - 00000000000a0000
[    0.000000] PM: Registered nosave memory: 00000000000a0000 - 00000000000f0000
[    0.000000] PM: Registered nosave memory: 00000000000f0000 - 0000000000100000
[    0.000000] Allocating PCI resources starting at 40000000 (gap: 40000000:beffc000)
[    0.000000] Booting paravirtualized kernel on KVM
[    0.000000] setup_percpu: NR_CPUS:64 nr_cpumask_bits:64 nr_cpu_ids:1 nr_node_ids:1
[    0.000000] PERCPU: Embedded 28 pages/cpu @ffff88003fc00000 s83520 r8192 d22976 u2097152
[    0.000000] pcpu-alloc: s83520 r8192 d22976 u2097152 alloc=1*2097152
[    0.000000] pcpu-alloc: [0] 0 
[    0.000000] kvm-clock: cpu 0, msr 0:3fc13941, primary cpu clock
[    0.000000] Built 1 zonelists in Node order, mobility grouping on.  Total pages: 258438
[    0.000000] Policy zone: DMA32
[    0.000000] Kernel command line: BOOT_IMAGE=/boot/vmlinuz-2.6.37+ root=/dev/sda1 ro
[    0.000000] PID hash table entries: 4096 (order: 3, 32768 bytes)
[    0.000000] Checking aperture...
[    0.000000] No AGP bridge found
[    0.000000] Calgary: detecting Calgary via BIOS EBDA area
[    0.000000] Calgary: Unable to locate Rio Grande table in EBDA - bailing!
[    0.000000] Memory: 1020480k/1048564k available (5734k kernel code, 452k absent, 27632k reserved, 5162k data, 920k init)
[    0.000000] SLUB: Genslabs=15, HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1
[    0.000000] Hierarchical RCU implementation.
[    0.000000] 	RCU dyntick-idle grace-period acceleration is enabled.
[    0.000000] 	RCU-based detection of stalled CPUs is disabled.
[    0.000000] NR_IRQS:4352 nr_irqs:256 16
[    0.000000] Console: colour VGA+ 80x25
[    0.000000] console [tty0] enabled
[    0.000000] allocated 10485760 bytes of page_cgroup
[    0.000000] please try 'cgroup_disable=memory' option if you don't want memory cgroups
[    0.000000] hpet clockevent registered
[    0.000000] Detected 3210.670 MHz processor.
[    0.020000] Calibrating delay loop (skipped) preset value.. 6421.34 BogoMIPS (lpj=32106700)
[    0.020000] pid_max: default: 32768 minimum: 301
[    0.020000] Security Framework initialized
[    0.020000] AppArmor: AppArmor initialized
[    0.020000] Dentry cache hash table entries: 131072 (order: 8, 1048576 bytes)
[    0.020000] Inode-cache hash table entries: 65536 (order: 7, 524288 bytes)
[    0.020000] Mount-cache hash table entries: 256
[    0.020000] Initializing cgroup subsys ns
[    0.020000] ns_cgroup deprecated: consider using the 'clone_children' flag without the ns_cgroup.
[    0.020000] Initializing cgroup subsys cpuacct
[    0.020000] Initializing cgroup subsys memory
[    0.020000] Initializing cgroup subsys devices
[    0.020000] Initializing cgroup subsys freezer
[    0.020000] Initializing cgroup subsys net_cls
[    0.020000] mce: CPU supports 10 MCE banks
[    0.020000] Performance Events: Broken PMU hardware detected, software events only.
[    0.020000] SMP alternatives: switching to UP code
[    0.039991] Freeing SMP alternatives: 24k freed
[    0.040038] ACPI: Core revision 20101013
[    0.041792] ftrace: allocating 21672 entries in 85 pages
[    0.050104] Setting APIC routing to flat
[    0.051354] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1
[    0.051696] CPU0: AMD QEMU Virtual CPU version 0.13.50 stepping 03
[    0.060000] Brought up 1 CPUs
[    0.060000] Total of 1 processors activated (6421.34 BogoMIPS).
[    0.060000] devtmpfs: initialized
[    0.060000] regulator: core version 0.5
[    0.060000] regulator: dummy: 
[    0.060000] Time: 12:01:59  Date: 01/10/11
[    0.060000] NET: Registered protocol family 16
[    0.060000] ACPI: bus type pci registered
[    0.060000] PCI: Using configuration type 1 for base access
[    0.061143] bio: create slab <bio-0> at 0
[    0.064031] ACPI: EC: Look up EC in DSDT
[    0.067649] ACPI: Interpreter enabled
[    0.067934] ACPI: (supports S0 S3 S4 S5)
[    0.068377] ACPI: Using IOAPIC for interrupt routing
[    0.069727] ACPI Exception: AE_NOT_FOUND, Evaluating _PRW (20101013/scan-723)
[    0.070646] ACPI: No dock devices found.
[    0.070936] PCI: Ignoring host bridge windows from ACPI; if necessary, use "pci=use_crs" and report a bug
[    0.071514] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])
[    0.071878] pci_root PNP0A03:00: host bridge window [io  0x0000-0x0cf7] (ignored)
[    0.071880] pci_root PNP0A03:00: host bridge window [io  0x0d00-0xffff] (ignored)
[    0.071882] pci_root PNP0A03:00: host bridge window [mem 0x000a0000-0x000bffff] (ignored)
[    0.071883] pci_root PNP0A03:00: host bridge window [mem 0xe0000000-0xfebfffff] (ignored)
[    0.071907] pci 0000:00:00.0: [8086:1237] type 0 class 0x000600
[    0.072066] pci 0000:00:01.0: [8086:7000] type 0 class 0x000601
[    0.072310] pci 0000:00:01.1: [8086:7010] type 0 class 0x000101
[    0.072464] pci 0000:00:01.1: reg 20: [io  0xc000-0xc00f]
[    0.072572] pci 0000:00:01.3: [8086:7113] type 0 class 0x000680
[    0.072779] pci 0000:00:01.3: quirk: [io  0xb000-0xb03f] claimed by PIIX4 ACPI
[    0.073306] pci 0000:00:01.3: quirk: [io  0xb100-0xb10f] claimed by PIIX4 SMB
[    0.073735] pci 0000:00:02.0: [1013:00b8] type 0 class 0x000300
[    0.078874] pci 0000:00:02.0: reg 10: [mem 0xf0000000-0xf1ffffff pref]
[    0.084927] pci 0000:00:02.0: reg 14: [mem 0xf2000000-0xf2000fff]
[    0.105206] pci 0000:00:02.0: reg 30: [mem 0xf2010000-0xf201ffff pref]
[    0.105340] pci 0000:00:03.0: [10ec:8139] type 0 class 0x000200
[    0.105432] pci 0000:00:03.0: reg 10: [io  0xc100-0xc1ff]
[    0.105498] pci 0000:00:03.0: reg 14: [mem 0xf2020000-0xf20200ff]
[    0.105818] pci 0000:00:03.0: reg 30: [mem 0xf2030000-0xf203ffff pref]
[    0.105885] pci 0000:00:04.0: [1002:5b64] type 0 class 0x000380
[    0.105951] pci 0000:00:04.0: reg 10: [mem 0xf8000000-0xffffffff]
[    0.106006] pci 0000:00:04.0: reg 14: [io  0xc200-0xc2ff]
[    0.106062] pci 0000:00:04.0: reg 18: [mem 0x00000000-0x0000ffff]
[    0.106277] pci 0000:00:04.0: reg 30: [mem 0x00010000-0x0001ffff pref]
[    0.106565] ACPI: PCI Interrupt Routing Table [\_SB_.PCI0._PRT]
[    0.107970] ACPI: PCI Interrupt Link [LNKA] (IRQs 5 *10 11)
[    0.108560] ACPI: PCI Interrupt Link [LNKB] (IRQs 5 *10 11)
[    0.109122] ACPI: PCI Interrupt Link [LNKC] (IRQs 5 10 *11)
[    0.110060] ACPI: PCI Interrupt Link [LNKD] (IRQs 5 10 *11)
[    0.120078] HEST: Table is not found!
[    0.121745] vgaarb: device added: PCI:0000:00:02.0,decodes=io+mem,owns=io+mem,locks=none
[    0.130009] vgaarb: loaded
[    0.131645] SCSI subsystem initialized
[    0.133270] libata version 3.00 loaded.
[    0.133467] usbcore: registered new interface driver usbfs
[    0.134938] usbcore: registered new interface driver hub
[    0.136486] usbcore: registered new device driver usb
[    0.140315] wmi: Mapper loaded
[    0.141561] PCI: Using ACPI for IRQ routing
[    0.142859] PCI: pci_cache_line_size set to 64 bytes
[    0.143195] reserve RAM buffer: 000000000009f400 - 000000000009ffff 
[    0.143214] reserve RAM buffer: 000000003fffd000 - 000000003fffffff 
[    0.143591] NetLabel: Initializing
[    0.144795] NetLabel:  domain hash size = 128
[    0.146078] NetLabel:  protocols = UNLABELED CIPSOv4
[    0.147407] NetLabel:  unlabeled traffic allowed by default
[    0.147739] HPET: 3 timers in total, 0 timers will be used for per-cpu timer
[    0.148102] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0
[    0.148556] hpet0: 3 comparators, 64-bit 100.000000 MHz counter
[    0.160054] Switching to clocksource kvm-clock
[    0.165135] AppArmor: AppArmor Filesystem Enabled
[    0.165456] pnp: PnP ACPI init
[    0.165728] ACPI: bus type pnp registered
[    0.166029] pnp 00:00: [bus 00-ff]
[    0.166031] pnp 00:00: [io  0x0cf8-0x0cff]
[    0.166032] pnp 00:00: [io  0x0000-0x0cf7 window]
[    0.166034] pnp 00:00: [io  0x0d00-0xffff window]
[    0.166035] pnp 00:00: [mem 0x000a0000-0x000bffff window]
[    0.166036] pnp 00:00: [mem 0xe0000000-0xfebfffff window]
[    0.166070] pnp 00:00: Plug and Play ACPI device, IDs PNP0a03 (active)
[    0.166095] pnp 00:01: [io  0x0070-0x0071]
[    0.166120] pnp 00:01: [irq 8]
[    0.166122] pnp 00:01: [io  0x0072-0x0077]
[    0.166137] pnp 00:01: Plug and Play ACPI device, IDs PNP0b00 (active)
[    0.166164] pnp 00:02: [io  0x0060]
[    0.166165] pnp 00:02: [io  0x0064]
[    0.166178] pnp 00:02: [irq 1]
[    0.166192] pnp 00:02: Plug and Play ACPI device, IDs PNP0303 (active)
[    0.166225] pnp 00:03: [irq 12]
[    0.166239] pnp 00:03: Plug and Play ACPI device, IDs PNP0f13 (active)
[    0.166256] pnp 00:04: [io  0x03f2-0x03f5]
[    0.166257] pnp 00:04: [io  0x03f7]
[    0.166268] pnp 00:04: [irq 6]
[    0.166270] pnp 00:04: [dma 2]
[    0.166283] pnp 00:04: Plug and Play ACPI device, IDs PNP0700 (active)
[    0.166325] pnp 00:05: [io  0x0378-0x037f]
[    0.166337] pnp 00:05: [irq 7]
[    0.166354] pnp 00:05: Plug and Play ACPI device, IDs PNP0400 (active)
[    0.166388] pnp 00:06: [io  0x03f8-0x03ff]
[    0.166400] pnp 00:06: [irq 4]
[    0.166414] pnp 00:06: Plug and Play ACPI device, IDs PNP0501 (active)
[    0.166443] pnp 00:07: [mem 0xfed00000-0xfed003ff]
[    0.166460] pnp 00:07: Plug and Play ACPI device, IDs PNP0103 (active)
[    0.166535] pnp: PnP ACPI: found 8 devices
[    0.166820] ACPI: ACPI bus type pnp unregistered
[    0.172648] pci 0000:00:04.0: address space collision: [mem 0x00010000-0x0001ffff pref] conflicts with System RAM [mem 0x00010000-0x0009f3ff]
[    0.173319] pci 0000:00:04.0: BAR 2: assigned [mem 0x40000000-0x4000ffff]
[    0.174167] pci 0000:00:04.0: BAR 2: set to [mem 0x40000000-0x4000ffff] (PCI address [0x40000000-0x4000ffff])
[    0.176704] pci 0000:00:04.0: BAR 6: assigned [mem 0x40010000-0x4001ffff pref]
[    0.179009] pci_bus 0000:00: resource 0 [io  0x0000-0xffff]
[    0.179016] pci_bus 0000:00: resource 1 [mem 0x00000000-0xffffffffff]
[    0.179121] NET: Registered protocol family 2
[    0.180782] IP route cache hash table entries: 32768 (order: 6, 262144 bytes)
[    0.183746] TCP established hash table entries: 131072 (order: 9, 2097152 bytes)
[    0.187692] TCP bind hash table entries: 65536 (order: 8, 1048576 bytes)
[    0.188705] TCP: Hash tables configured (established 131072 bind 65536)
[    0.189650] TCP reno registered
[    0.189934] UDP hash table entries: 512 (order: 2, 16384 bytes)
[    0.190316] UDP-Lite hash table entries: 512 (order: 2, 16384 bytes)
[    0.190734] NET: Registered protocol family 1
[    0.191035] pci 0000:00:00.0: Limiting direct PCI/PCI transfers
[    0.191371] pci 0000:00:01.0: PIIX3: Enabling Passive Release
[    0.191710] pci 0000:00:01.0: Activating ISA DMA hang workarounds
[    0.192057] pci 0000:00:02.0: Boot video device
[    0.192078] PCI: CLS 64 bytes, default 64
[    0.192244] Scanning for low memory corruption every 60 seconds
[    0.192655] audit: initializing netlink socket (disabled)
[    0.192978] type=2000 audit(1294660920.190:1): initialized
[    0.201771] HugeTLB registered 2 MB page size, pre-allocated 0 pages
[    0.203129] VFS: Disk quotas dquot_6.5.2
[    0.203453] Dquot-cache hash table entries: 512 (order 0, 4096 bytes)
[    0.204060] fuse init (API version 7.15)
[    0.204412] msgmni has been set to 1993
[    0.204848] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 253)
[    0.205376] io scheduler noop registered
[    0.205658] io scheduler deadline registered
[    0.205975] io scheduler cfq registered (default)
[    0.206330] pci_hotplug: PCI Hot Plug PCI Core version: 0.5
[    0.206666] pciehp: PCI Express Hot Plug Controller Driver version: 0.4
[    0.207129] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input0
[    0.210352] ACPI: Power Button [PWRF]
[    0.210815] ACPI: acpi_idle registered with cpuidle
[    0.211297] ERST: Table is not found!
[    0.211678] Linux agpgart interface v0.103
[    0.211978] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled
[    0.234763] serial8250: ttyS0 at I/O 0x3f8 (irq = 4) is a 16550A
[    0.262534] 00:06: ttyS0 at I/O 0x3f8 (irq = 4) is a 16550A
[    0.266921] brd: module loaded
[    0.269495] loop: module loaded
[    0.271129] ata_piix 0000:00:01.1: version 2.13
[    0.271399] ata_piix 0000:00:01.1: setting latency timer to 64
[    0.272452] scsi0 : ata_piix
[    0.273855] scsi1 : ata_piix
[    0.275134] ata1: PATA max MWDMA2 cmd 0x1f0 ctl 0x3f6 bmdma 0xc000 irq 14
[    0.276681] ata2: PATA max MWDMA2 cmd 0x170 ctl 0x376 bmdma 0xc008 irq 15
[    0.277734] Fixed MDIO Bus: probed
[    0.278022] PPP generic driver version 2.4.2
[    0.278343] tun: Universal TUN/TAP device driver, 1.6
[    0.278653] tun: (C) 1999-2004 Max Krasnyansky <maxk@qualcomm.com>
[    0.279047] ehci_hcd: USB 2.0 'Enhanced' Host Controller (EHCI) Driver
[    0.279420] ohci_hcd: USB 1.1 'Open' Host Controller (OHCI) Driver
[    0.279790] uhci_hcd: USB Universal Host Controller Interface driver
[    0.280196] PNP: PS/2 Controller [PNP0303:KBD,PNP0f13:MOU] at 0x60,0x64 irq 1,12
[    0.281194] serio: i8042 KBD port at 0x60,0x64 irq 1
[    0.281500] serio: i8042 AUX port at 0x60,0x64 irq 12
[    0.283199] mice: PS/2 mouse device common for all mice
[    0.285392] input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input1
[    0.288543] rtc_cmos 00:01: rtc core: registered rtc_cmos as rtc0
[    0.290371] rtc0: alarms up to one day, 114 bytes nvram, hpet irqs
[    0.292156] device-mapper: uevent: version 1.0.3
[    0.293802] device-mapper: ioctl: 4.18.0-ioctl (2010-06-29) initialised: dm-devel@redhat.com
[    0.296386] device-mapper: multipath: version 1.1.1 loaded
[    0.297148] device-mapper: multipath round-robin: version 1.0.0 loaded
[    0.297579] cpuidle: using governor ladder
[    0.297869] cpuidle: using governor menu
[    0.298290] TCP cubic registered
[    0.298633] NET: Registered protocol family 10
[    0.299127] lo: Disabled Privacy Extensions
[    0.299528] NET: Registered protocol family 17
[    0.299834] Registering the dns_resolver key type
[    0.300176] powernow-k8: Processor cpuid 623 not supported
[    0.300570] PM: Hibernation image not present or could not be loaded.
[    0.300577] registered taskstats version 1
[    0.301040]   Magic number: 11:950:25
[    0.301420] rtc_cmos 00:01: setting system clock to 2011-01-10 12:01:59 UTC (1294660919)
[    0.301962] BIOS EDD facility v0.16 2004-Jun-25, 0 devices found
[    0.302290] EDD information not available.
[    0.430621] ata2.01: NODEV after polling detection
[    0.430836] ata1.01: NODEV after polling detection
[    0.431050] ata2.00: ATAPI: QEMU DVD-ROM, 0.13.50, max UDMA/100
[    0.431527] ata1.00: ATA-7: QEMU HARDDISK, 0.13.50, max UDMA/100
[    0.431854] ata1.00: 20971520 sectors, multi 16: LBA48 
[    0.432482] ata2.00: configured for MWDMA2
[    0.433103] ata1.00: configured for MWDMA2
[    0.433510] scsi 0:0:0:0: Direct-Access     ATA      QEMU HARDDISK    0.13 PQ: 0 ANSI: 5
[    0.435440] sd 0:0:0:0: Attached scsi generic sg0 type 0
[    0.436042] scsi 1:0:0:0: CD-ROM            QEMU     QEMU DVD-ROM     0.13 PQ: 0 ANSI: 5
[    0.439646] sd 0:0:0:0: [sda] 20971520 512-byte logical blocks: (10.7 GB/10.0 GiB)
[    0.442256] sr0: scsi3-mmc drive: 4x/4x xa/form2 tray
[    0.443659] cdrom: Uniform CD-ROM driver Revision: 3.20
[    0.445291] sr 1:0:0:0: Attached scsi CD-ROM sr0
[    0.445412] sr 1:0:0:0: Attached scsi generic sg1 type 5
[    0.446975] sd 0:0:0:0: [sda] Write Protect is off
[    0.447306] sd 0:0:0:0: [sda] Mode Sense: 00 3a 00 00
[    0.447318] sd 0:0:0:0: [sda] Write cache: disabled, read cache: enabled, doesn't support DPO or FUA
[    0.448509]  sda: sda1 sda2 < sda5 >
[    0.448918] sd 0:0:0:0: [sda] Attached SCSI disk
[    0.449229] md: Waiting for all devices to be available before autodetect
[    0.449578] md: If you don't use raid, use raid=noautodetect
[    0.449971] md: Autodetecting RAID arrays.
[    0.450275] md: Scanned 0 and added 0 devices.
[    0.450568] md: autorun ...
[    0.450825] md: ... autorun DONE.
[    0.451430] EXT3-fs (sda1): error: couldn't mount because of unsupported optional features (240)
[    0.452356] EXT2-fs (sda1): error: couldn't mount because of unsupported optional features (240)
[    0.455207] EXT4-fs (sda1): mounted filesystem with ordered data mode. Opts: (null)
[    0.455821] VFS: Mounted root (ext4 filesystem) readonly on device 8:1.
[    0.456427] devtmpfs: mounted
[    0.456765] Freeing unused kernel memory: 920k freed
[    0.457431] Write protecting the kernel read-only data: 10240k
[    0.457867] Freeing unused kernel memory: 392k freed
[    0.458436] Freeing unused kernel memory: 1688k freed
[    1.472567] udev[80]: starting version 163
[    1.605481] piix4_smbus 0000:00:01.3: SMBus Host Controller at 0xb100, revision 0
[    1.694126] 8139cp: 8139cp: 10/100 PCI Ethernet driver v1.3 (Mar 22, 2004)
[    1.694270] ACPI: PCI Interrupt Link [LNKC] enabled at IRQ 11
[    1.694291] 8139cp 0000:00:03.0: PCI INT A -> Link[LNKC] -> GSI 11 (level, high) -> IRQ 11
[    1.755399] parport_pc 00:05: reported by Plug and Play ACPI
[    1.755478] parport0: PC-style at 0x378, irq 7 [PCSPP,TRISTATE]
[    1.757534] 8139cp 0000:00:03.0: eth0: RTL-8139C+ at 0xffffc90000194000, 52:54:00:12:34:56, IRQ 11
[    1.757566] 8139cp 0000:00:03.0: setting latency timer to 64
[    1.770420] FDC 0 is a S82078B
[    1.773167] lp: driver loaded but no devices found
[    1.787444] 8139too: 8139too Fast Ethernet driver 0.9.28
[    1.798762] ppdev: user-space parallel port driver
[    1.830244] lp0: using parport0 (interrupt-driven).
[    1.840582] [drm] Initialized drm 1.1.0 20060810
[    1.970951] Adding 492540k swap on /dev/sda5.  Priority:-1 extents:1 across:492540k 
[    2.122762] [drm] radeon defaulting to kernel modesetting.
[    2.122766] [drm] radeon kernel modesetting enabled.
[    2.123551] ACPI: PCI Interrupt Link [LNKD] enabled at IRQ 10
[    2.123667] radeon 0000:00:04.0: PCI INT A -> Link[LNKD] -> GSI 10 (level, high) -> IRQ 10
[    2.123770] radeon 0000:00:04.0: setting latency timer to 64
[    2.146964] [drm] initializing kernel modesetting (RV380 0x1002:0x5B64).
[    2.150248] [drm] register mmio base: 0x40000000
[    2.150249] [drm] register mmio size: 65536
[    2.153873] [drm] Generation 2 PCI interface, using max accessible memory
[    2.153947] radeon 0000:00:04.0: VRAM: 128M 0x00000000F8000000 - 0x00000000FFFFFFFF (128M used)
[    2.153956] radeon 0000:00:04.0: GTT: 512M 0x00000000D8000000 - 0x00000000F7FFFFFF
[    2.155020] radeon 0000:00:04.0: irq 40 for MSI/MSI-X
[    2.278810] radeon 0000:00:04.0: radeon: using MSI.
[    2.286507] input: ImExPS/2 Generic Explorer Mouse as /devices/platform/i8042/serio1/input/input2
[    2.433702] EXT4-fs (sda1): re-mounted. Opts: errors=remount-ro
[    2.471774] type=1400 audit(1294660921.660:2): apparmor="STATUS" operation="profile_load" name="/sbin/dhclient3" pid=284 comm="apparmor_parser"
[    2.471820] type=1400 audit(1294660921.660:3): apparmor="STATUS" operation="profile_load" name="/usr/lib/NetworkManager/nm-dhcp-client.action" pid=284 comm="apparmor_parser"
[    2.471849] type=1400 audit(1294660921.660:4): apparmor="STATUS" operation="profile_load" name="/usr/lib/connman/scripts/dhclient-script" pid=284 comm="apparmor_parser"
[    2.551973] sshd (409): /proc/409/oom_adj is deprecated, please use /proc/409/oom_score_adj instead.
[    2.562467] apparmor_parser[424]: segfault at 0 ip 000000000040a14e sp 00007fff74e7e0b0 error 4 in apparmor_parser[400000+c9000]
[    2.602643] eth0: link up, 100Mbps, full-duplex, lpa 0x05E1
[    2.617272] apparmor_parser[447]: segfault at 0 ip 000000000040a14e sp 00007fff4f047660 error 4 in apparmor_parser[400000+c9000]
[    2.726183] [drm] radeon: irq initialized.
[    2.727657] [drm] Detected VRAM RAM=128M, BAR=128M
[    2.727665] [drm] RAM width 128bits DDR
[    2.727803] [TTM] Zone  kernel: Available graphics memory: 511752 kiB.
[    2.727817] [TTM] Initializing pool allocator.
[    2.727882] [drm] radeon: 128M of VRAM memory ready
[    2.727887] [drm] radeon: 512M of GTT memory ready.
[    2.727983] [drm] GART: num cpu pages 131072, num gpu pages 131072
[    2.737048] [drm] radeon: 1 quad pipes, 1 Z pipes initialized.
[    3.116938] [drm] PCIE GART of 512M enabled (table at 0xF8040000).
[    3.118391] radeon 0000:00:04.0: WB enabled
[    3.118854] [drm] Loading R300 Microcode
[    3.149456] [drm] radeon: ring at 0x00000000D8001000
[    3.390650] [drm:r100_ring_test] *ERROR* radeon: ring test failed (sracth(0x15E4)=0xCAFEDEAD)
[    3.391855] [drm:r100_cp_init] *ERROR* radeon: cp isn't working (-22).
[    3.392619] radeon 0000:00:04.0: failled initializing CP (-22).
[    3.393335] radeon 0000:00:04.0: Disabling GPU acceleration
[    3.399268] [drm] radeon: cp finalized
[    3.483548] radeon 0000:00:04.0: ffff880033f68400 unpin not necessary
[    3.484413] [drm] Radeon Display Connectors
[    3.484415] [drm] Connector 0:
[    3.484416] [drm]   VGA
[    3.484418] [drm]   DDC: 0x60 0x60 0x60 0x60 0x60 0x60 0x60 0x60
[    3.484418] [drm]   Encoders:
[    3.484419] [drm]     CRT1: INTERNAL_DAC1
[    3.484420] [drm] Connector 1:
[    3.484421] [drm]   DVI-I
[    3.484421] [drm]   HPD1
[    3.484423] [drm]   DDC: 0x64 0x64 0x64 0x64 0x64 0x64 0x64 0x64
[    3.484423] [drm]   Encoders:
[    3.484424] [drm]     CRT2: INTERNAL_DAC2
[    3.484425] [drm]     DFP1: INTERNAL_TMDS1
[    5.759335] [drm] fb mappable at 0xF8040000
[    5.759337] [drm] vram apper at 0xF8000000
[    5.759338] [drm] size 5242880
[    5.759340] [drm] fb depth is 24
[    5.759340] [drm]    pitch is 5120
[    9.541957] Console: switching to colour frame buffer device 160x64
[   12.890011] eth0: no IPv6 routers present
[   13.254355] fb0: radeondrmfb frame buffer device
[   13.254356] drm: registered panic notifier
[   13.255279] [drm] Initialized radeon 2.7.0 20080528 for 0000:00:04.0 on minor 0
[   17.794224] mtrr: no MTRR for f0000000,100000 found
[   18.661036] EXT4-fs (sda1): re-mounted. Opts: errors=remount-ro,commit=0
[   20.535317] EXT4-fs (sda1): re-mounted. Opts: errors=remount-ro,commit=0

Re: A query on frame buffers

[Konrad Rzeszutek Wilk]

> Tried your patches on Guest machine, it did not help. I am attaching
> the messages from guest machine.

Ok, then the issue is not with the guest but the IOMMU.
> 
> > well, what is the DMA code that gets turned on when you run your guest?
> 
> Looking into the qemu-kvm code to find more information about DMA.
> 
> > What do you see for 'PCI-DMA' ?
> 
> > Is it bounce buffer or swiotlb?
> 
> It is the default configuration. I did not pass swiotlb parameter on
> kernel boot line.

That is the nop one. So using virt_to_phys to get DMA address.

> 
> The host kernel is booted with iommu=1.

Is this a AMD or Intel chipset?
> 
> > How much memory do you pass to the guest? less than 4GB?
> 
> root@prasad-kvm:~/VMDisks# qemu-system-x86_64 -hda
> Ubuntu-10.10-amd64.img -m 1024M -device pci-assign,host\x02:00.0
> 
> I am passing 1GB memory to the guest machine.

Since you mentioned that you were able to passthrough other PCI devices
that means the IOMMU is working. This narrows it down.

The big difference between other PCI devices and the graphics card
is that the graphic card has its own VMM and "radeon_gart_bind" ends
up programming the bus address (so virt_to_phys of the pages, and 
you could put in a printk there to see what it is) in the graphic
VMM. This means that when the graphic card is told to fetch the
writeback data it ends up using the address that was programmed
in there to look. Here the IOMMU should step in and see "oh, this
DMA address is for this guest, let me patch it up and actually
look where this would fall in the guest space. Oh OK, let me use
this value real value which correspond the guests real value."

But the IOMMU has a couple of knobs. One of those is the passthrough
code and the GFX mapping code. The first should be easy to spot (should
tell you on bootup whether you got that enabled or not), while the other
looks to be a bunch of workarounds when passing in GFX cards b/c they ..
well, not sure what, but they look to not work correctly with the IOMMU. 

If you see 'identity map for device' where the device is your GFX
then that looks to be the bug. It shouldn't set the identity mapping on it.

Look for 'DMAR GFX' on Google. Might shed some more light.

Re: A query on frame buffers

[Prasad Joshi]

Hello Konard,

Thanks a lot for your efforts and time.

On Mon, Jan 10, 2011 at 6:17 PM, Konrad Rzeszutek Wilk
<konrad.wilk@oracle.com> wrote:
>> Tried your patches on Guest machine, it did not help. I am attaching
>> the messages from guest machine.
>
> Ok, then the issue is not with the guest but the IOMMU.
>>
>> > well, what is the DMA code that gets turned on when you run your guest?
>>
>> Looking into the qemu-kvm code to find more information about DMA.
>>
>> > What do you see for 'PCI-DMA' ?
>>
>> > Is it bounce buffer or swiotlb?
>>
>> It is the default configuration. I did not pass swiotlb parameter on
>> kernel boot line.
>
> That is the nop one. So using virt_to_phys to get DMA address.
>
>>
>> The host kernel is booted with iommu=1.
>
> Is this a AMD or Intel chipset?

AMD chipset.

>>
>> > How much memory do you pass to the guest? less than 4GB?
>>
>> root@prasad-kvm:~/VMDisks# qemu-system-x86_64 -hda
>> Ubuntu-10.10-amd64.img -m 1024M -device pci-assign,host\x02:00.0
>>
>> I am passing 1GB memory to the guest machine.
>
> Since you mentioned that you were able to passthrough other PCI devices
> that means the IOMMU is working. This narrows it down.

Yes I could pass-through a network card to VM. I also tried passing
through a old Sound Card to Windows 7 machine. It did not work as
expected, Windows 7 does not have the old drivers.

>
> The big difference between other PCI devices and the graphics card
> is that the graphic card has its own VMM and "radeon_gart_bind" ends
> up programming the bus address (so virt_to_phys of the pages, and
> you could put in a printk there to see what it is) in the graphic
> VMM. This means that when the graphic card is told to fetch the
> writeback data it ends up using the address that was programmed
> in there to look. Here the IOMMU should step in and see "oh, this
> DMA address is for this guest, let me patch it up and actually
> look where this would fall in the guest space. Oh OK, let me use
> this value real value which correspond the guests real value."

It will take some time for me to understand every thing you have written :)

>
> But the IOMMU has a couple of knobs. One of those is the passthrough
> code and the GFX mapping code. The first should be easy to spot (should
> tell you on bootup whether you got that enabled or not), while the other
> looks to be a bunch of workarounds when passing in GFX cards b/c they ..
> well, not sure what, but they look to not work correctly with the IOMMU.

By GFX Mapping,
Do you mean the code that maps VGA frame-buffers to Guest OS?

One more silly question on the frame-buffer mapping.

I was reading a paper on xen vga pass-though. It says "When applying
PCI pass-through, certain memory areas of the physical machine are
mapped to the VM. When the guest OS writes to one of those memory
addresses, Xen will make sure it is actually written at the
appropriate address. This implicates that no other VM is able to make
use of that device. The frame-buffer is mapped to the machine's main
memory at address 0xA0000 to 0xC0000. This memory range must be mapped
to the VM's memory in order for the OS to address the video adapter."

My question is....
In my case, the host machine is using Nvidia graphics card, so the
host frame-buffer memory 0xA0000 to 0xC0000 is being actively used by
host. Now if the ATI card maps it's framebuffer to this same address
wouldn't it cause any problem.

If the the machine has only 1 graphics card those statement make
sense, but I could not understand what will happen when there are two
graphics cards, each connected to different monitor and assigned to
different VM, where would each machines frame-buffer reside in the
host memory.

Does Xen allow passing-though multiple graphics cards, each to different VM?

I am really sorry if those questions does not make any sense.

>
> If you see 'identity map for device' where the device is your GFX
> then that looks to be the bug. It shouldn't set the identity mapping on it.
>

I guess the identity map is used only on the Intel chipset. Here are
msgs on host machine when addition debugging parameter
(amd_iommu_dump) was added.

prasad@prasad-kvm:~$ dmesg | grep -i -e iommu -e dmar -e amd
[    0.000000] Command line: BOOT_IMAGE=/vmlinuz-2.6.37-rc6+
root=/dev/sda1 ro iommu=1 amd_iommu_dump
[    0.000000] ACPI: SRAT 00000000bfe9f8b0 00108 (v01 AMD    FAM_F_10
00000002 AMD  00000001)
[    0.000000] ACPI: IVRS 00000000bfe9fa00 000D8 (v01  AMD     RD890S
00202031 AMD  00000000)
[    0.000000] ACPI: SSDT 00000000bfe9fae0 00DA4 (v01 A M I  POWERNOW
00000001 AMD  00000001)
[    0.000000] Kernel command line: BOOT_IMAGE=/vmlinuz-2.6.37-rc6+
root=/dev/sda1 ro iommu=1 amd_iommu_dump
[    0.000000] Please enable the IOMMU option in the BIOS setup
[    0.012918] Performance Events: AMD PMU driver.
[    0.137960] CPU0: AMD Phenom(tm) II X6 1090T Processor stepping 00
[    0.300011] System has AMD C1E enabled
[    2.680386] AMD-Vi: device: 00:00.2 cap: 0040 seg: 0 flags: 3e info 1300
[    2.680441] AMD-Vi:        mmio-addr: 00000000f6000000
[    2.680670] AMD-Vi:   DEV_SELECT_RANGE_START	 devid: 00:00.0 flags: 00
[    2.680726] AMD-Vi:   DEV_RANGE_END		 devid: 00:00.2
[    2.680776] AMD-Vi:   DEV_SELECT			 devid: 00:02.0 flags: 00
[    2.680827] AMD-Vi:   DEV_SELECT			 devid: 07:00.0 flags: 00
[    2.680883] AMD-Vi:   DEV_SELECT			 devid: 00:04.0 flags: 00
[    2.680934] AMD-Vi:   DEV_SELECT			 devid: 06:00.0 flags: 00
[    2.680984] AMD-Vi:   DEV_SELECT			 devid: 00:05.0 flags: 00
[    2.681035] AMD-Vi:   DEV_SELECT			 devid: 05:00.0 flags: 00
[    2.681086] AMD-Vi:   DEV_SELECT			 devid: 00:06.0 flags: 00
[    2.681137] AMD-Vi:   DEV_SELECT			 devid: 04:00.0 flags: 00
[    2.681187] AMD-Vi:   DEV_SELECT			 devid: 00:07.0 flags: 00
[    2.681238] AMD-Vi:   DEV_SELECT			 devid: 03:00.0 flags: 00
[    2.681289] AMD-Vi:   DEV_SELECT			 devid: 00:0b.0 flags: 00
[    2.681340] AMD-Vi:   DEV_SELECT_RANGE_START	 devid: 02:00.0 flags: 00
[    2.681391] AMD-Vi:   DEV_RANGE_END		 devid: 02:00.1
[    2.681441] AMD-Vi:   DEV_SELECT			 devid: 00:11.0 flags: 00
[    2.681492] AMD-Vi:   DEV_SELECT_RANGE_START	 devid: 00:12.0 flags: 00
[    2.681543] AMD-Vi:   DEV_RANGE_END		 devid: 00:12.2
[    2.681594] AMD-Vi:   DEV_SELECT_RANGE_START	 devid: 00:13.0 flags: 00
[    2.681645] AMD-Vi:   DEV_RANGE_END		 devid: 00:13.2
[    2.681695] AMD-Vi:   DEV_SELECT			 devid: 00:14.0 flags: d7
[    2.681746] AMD-Vi:   DEV_SELECT			 devid: 00:14.1 flags: 00
[    2.681797] AMD-Vi:   DEV_SELECT			 devid: 00:14.2 flags: 00
[    2.681847] AMD-Vi:   DEV_SELECT			 devid: 00:14.3 flags: 00
[    2.681898] AMD-Vi:   DEV_SELECT			 devid: 00:14.4 flags: 00
[    2.681949] AMD-Vi:   DEV_ALIAS_RANGE		 devid: 01:00.0 flags: 00
devid_to: 00:14.4
[    2.682010] AMD-Vi:   DEV_RANGE_END		 devid: 01:1f.7
[    2.682855] AMD-Vi:   DEV_SELECT			 devid: 00:14.5 flags: 00
[    2.682906] AMD-Vi:   DEV_SELECT_RANGE_START	 devid: 00:16.0 flags: 00
[    2.682957] AMD-Vi:   DEV_RANGE_END		 devid: 00:16.2
[    2.683234] AMD-Vi: Enabling IOMMU at 0000:00:00.2 cap 0x40
[    2.692043] AMD-Vi: Lazy IO/TLB flushing enabled
[    4.360112] ehci_hcd 0000:00:12.2: applying AMD
SB700/SB800/Hudson-2/3 EHCI dummy qh workaround
[    4.380635] ehci_hcd 0000:00:13.2: applying AMD
SB700/SB800/Hudson-2/3 EHCI dummy qh workaround
[    4.400658] ehci_hcd 0000:00:16.2: applying AMD
SB700/SB800/Hudson-2/3 EHCI dummy qh workaround
[    4.811320] powernow-k8: Found 1 AMD Phenom(tm) II X6 1090T
Processor (6 cpu cores) (version 2.20.00)
[   12.349919] EDAC amd64_edac:  Ver: 3.3.0 Dec 21 2010
[   12.350018] EDAC amd64: This node reports that Memory ECC is
currently disabled, set F3x44[22] (0000:00:18.3).
[   12.350030] EDAC amd64: ECC disabled in the BIOS or no ECC
capability, module will not load.
[   12.350044] amd64_edac: probe of 0000:00:18.2 failed with error -22

prasad@prasad-kvm:~$ lspci | grep -i RV
02:00.0 VGA compatible controller: ATI Technologies Inc RV370 5B64
[FireGL V3100 (PCIE)] (rev 80)
02:00.1 Display controller: ATI Technologies Inc RV370 5B64 [FireGL
V3100 (PCIE)] (Secondary) (rev 80)

> Look for 'DMAR GFX' on Google. Might shed some more light.
>

No DMAR for AMD. I think I should have a look at DMAR GFX.

I am really thankful to you for your support.

Thanks and Regards,
Prasad

Re: A query on frame buffers

[Konrad Rzeszutek Wilk]

> AMD chipset.

They don't seem to have have any errata's for this GFX business.
But they do have a passthrough mode - hopefull that is not what you have
passed in?

> > Since you mentioned that you were able to passthrough other PCI devices
> > that means the IOMMU is working. This narrows it down.
> 
> Yes I could pass-through a network card to VM. I also tried passing
> through a old Sound Card to Windows 7 machine. It did not work as
> expected, Windows 7 does not have the old drivers.

Heh.
> 
> >
> > The big difference between other PCI devices and the graphics card
> > is that the graphic card has its own VMM and "radeon_gart_bind" ends
> > up programming the bus address (so virt_to_phys of the pages, and
> > you could put in a printk there to see what it is) in the graphic
> > VMM. This means that when the graphic card is told to fetch the
> > writeback data it ends up using the address that was programmed
> > in there to look. Here the IOMMU should step in and see "oh, this
> > DMA address is for this guest, let me patch it up and actually
> > look where this would fall in the guest space. Oh OK, let me use
> > this value real value which correspond the guests real value."
> 
> It will take some time for me to understand every thing you have written :)

Ok. Ping me if you have some questions. Reading this might
help (or confuse you more): http://wiki.xensource.com/xenwiki/XenPVOPSDRM

> 
> >
> > But the IOMMU has a couple of knobs. One of those is the passthrough
> > code and the GFX mapping code. The first should be easy to spot (should
> > tell you on bootup whether you got that enabled or not), while the other
> > looks to be a bunch of workarounds when passing in GFX cards b/c they ..
> > well, not sure what, but they look to not work correctly with the IOMMU.
> 
> By GFX Mapping,
> Do you mean the code that maps VGA frame-buffers to Guest OS?

No. It is the IOMMU kernel code that checks to see if this is a GFX
card and if so does something fancy. But the workarounds for this appear
to be exclusive to Intel chipset so you shouldn't hit any.

> 
> One more silly question on the frame-buffer mapping.
> 
> I was reading a paper on xen vga pass-though. It says "When applying
> PCI pass-through, certain memory areas of the physical machine are
> mapped to the VM. When the guest OS writes to one of those memory
> addresses, Xen will make sure it is actually written at the
> appropriate address. This implicates that no other VM is able to make
> use of that device. The frame-buffer is mapped to the machine's main
> memory at address 0xA0000 to 0xC0000. This memory range must be mapped
> to the VM's memory in order for the OS to address the video adapter."
<nods>
> 
> My question is....
> In my case, the host machine is using Nvidia graphics card, so the
> host frame-buffer memory 0xA0000 to 0xC0000 is being actively used by
> host. Now if the ATI card maps it's framebuffer to this same address
> wouldn't it cause any problem.

Ah, but it does not. The first card that is called by the BIOS (so your
motherboard BIOS) sets itself up to use that memory. The other card has
to wait. Keep in mind that the A0000 to C0000 is the old style VGA
framebuffer (80x25) or if you program the card properly it can do
VESA graphics, but nothing else. Usually the guest gets a Cirrus Logic
video card passed in which sets this device up (and touching that
area in the guest ends up in VNC window or SDL window).

Anyhow back to your host..
If you grep in your kernel log you should see something like:
[drm] fb mappable at 0xE0142000

This is the framebuffer address.
> 
> If the the machine has only 1 graphics card those statement make
> sense, but I could not understand what will happen when there are two
> graphics cards, each connected to different monitor and assigned to
> different VM, where would each machines frame-buffer reside in the
> host memory.

Check the kernel log. Usually it is also the first BAR in the pci device.

> 
> Does Xen allow passing-though multiple graphics cards, each to different VM?

Yes (with some patches posted by the AMD and Intel folks)..
(look for threads that have some GFX or Radeon or something like that in its title).
> 
> >
> > If you see 'identity map for device' where the device is your GFX
> > then that looks to be the bug. It shouldn't set the identity mapping on it.
> >
> 
> I guess the identity map is used only on the Intel chipset. Here are
> msgs on host machine when addition debugging parameter
> (amd_iommu_dump) was added.
> 
> prasad@prasad-kvm:~$ dmesg | grep -i -e iommu -e dmar -e amd
> [    0.000000] Command line: BOOT_IMAGE=/vmlinuz-2.6.37-rc6+
> root=/dev/sda1 ro iommu=1 amd_iommu_dump
> [    0.000000] ACPI: SRAT 00000000bfe9f8b0 00108 (v01 AMD    FAM_F_10
> 00000002 AMD  00000001)
> [    0.000000] ACPI: IVRS 00000000bfe9fa00 000D8 (v01  AMD     RD890S
> 00202031 AMD  00000000)
> [    0.000000] ACPI: SSDT 00000000bfe9fae0 00DA4 (v01 A M I  POWERNOW
> 00000001 AMD  00000001)
> [    0.000000] Kernel command line: BOOT_IMAGE=/vmlinuz-2.6.37-rc6+
> root=/dev/sda1 ro iommu=1 amd_iommu_dump
> [    0.000000] Please enable the IOMMU option in the BIOS setup
> [    0.012918] Performance Events: AMD PMU driver.
> [    0.137960] CPU0: AMD Phenom(tm) II X6 1090T Processor stepping 00
> [    0.300011] System has AMD C1E enabled
> [    2.680386] AMD-Vi: device: 00:00.2 cap: 0040 seg: 0 flags: 3e info 1300
> [    2.680441] AMD-Vi:        mmio-addr: 00000000f6000000
> [    2.680670] AMD-Vi:   DEV_SELECT_RANGE_START	 devid: 00:00.0 flags: 00
> [    2.680726] AMD-Vi:   DEV_RANGE_END		 devid: 00:00.2
> [    2.680776] AMD-Vi:   DEV_SELECT			 devid: 00:02.0 flags: 00
> [    2.680827] AMD-Vi:   DEV_SELECT			 devid: 07:00.0 flags: 00
> [    2.680883] AMD-Vi:   DEV_SELECT			 devid: 00:04.0 flags: 00
> [    2.680934] AMD-Vi:   DEV_SELECT			 devid: 06:00.0 flags: 00
> [    2.680984] AMD-Vi:   DEV_SELECT			 devid: 00:05.0 flags: 00
> [    2.681035] AMD-Vi:   DEV_SELECT			 devid: 05:00.0 flags: 00
> [    2.681086] AMD-Vi:   DEV_SELECT			 devid: 00:06.0 flags: 00
> [    2.681137] AMD-Vi:   DEV_SELECT			 devid: 04:00.0 flags: 00
> [    2.681187] AMD-Vi:   DEV_SELECT			 devid: 00:07.0 flags: 00
> [    2.681238] AMD-Vi:   DEV_SELECT			 devid: 03:00.0 flags: 00
> [    2.681289] AMD-Vi:   DEV_SELECT			 devid: 00:0b.0 flags: 00
> [    2.681340] AMD-Vi:   DEV_SELECT_RANGE_START	 devid: 02:00.0 flags: 00
> [    2.681391] AMD-Vi:   DEV_RANGE_END		 devid: 02:00.1
> [    2.681441] AMD-Vi:   DEV_SELECT			 devid: 00:11.0 flags: 00
> [    2.681492] AMD-Vi:   DEV_SELECT_RANGE_START	 devid: 00:12.0 flags: 00
> [    2.681543] AMD-Vi:   DEV_RANGE_END		 devid: 00:12.2
> [    2.681594] AMD-Vi:   DEV_SELECT_RANGE_START	 devid: 00:13.0 flags: 00
> [    2.681645] AMD-Vi:   DEV_RANGE_END		 devid: 00:13.2
> [    2.681695] AMD-Vi:   DEV_SELECT			 devid: 00:14.0 flags: d7
> [    2.681746] AMD-Vi:   DEV_SELECT			 devid: 00:14.1 flags: 00
> [    2.681797] AMD-Vi:   DEV_SELECT			 devid: 00:14.2 flags: 00
> [    2.681847] AMD-Vi:   DEV_SELECT			 devid: 00:14.3 flags: 00
> [    2.681898] AMD-Vi:   DEV_SELECT			 devid: 00:14.4 flags: 00
> [    2.681949] AMD-Vi:   DEV_ALIAS_RANGE		 devid: 01:00.0 flags: 00
> devid_to: 00:14.4
> [    2.682010] AMD-Vi:   DEV_RANGE_END		 devid: 01:1f.7
> [    2.682855] AMD-Vi:   DEV_SELECT			 devid: 00:14.5 flags: 00
> [    2.682906] AMD-Vi:   DEV_SELECT_RANGE_START	 devid: 00:16.0 flags: 00
> [    2.682957] AMD-Vi:   DEV_RANGE_END		 devid: 00:16.2
> [    2.683234] AMD-Vi: Enabling IOMMU at 0000:00:00.2 cap 0x40
> [    2.692043] AMD-Vi: Lazy IO/TLB flushing enabled
> [    4.360112] ehci_hcd 0000:00:12.2: applying AMD
> SB700/SB800/Hudson-2/3 EHCI dummy qh workaround
> [    4.380635] ehci_hcd 0000:00:13.2: applying AMD
> SB700/SB800/Hudson-2/3 EHCI dummy qh workaround
> [    4.400658] ehci_hcd 0000:00:16.2: applying AMD
> SB700/SB800/Hudson-2/3 EHCI dummy qh workaround
> [    4.811320] powernow-k8: Found 1 AMD Phenom(tm) II X6 1090T
> Processor (6 cpu cores) (version 2.20.00)
> [   12.349919] EDAC amd64_edac:  Ver: 3.3.0 Dec 21 2010
> [   12.350018] EDAC amd64: This node reports that Memory ECC is
> currently disabled, set F3x44[22] (0000:00:18.3).
> [   12.350030] EDAC amd64: ECC disabled in the BIOS or no ECC
> capability, module will not load.
> [   12.350044] amd64_edac: probe of 0000:00:18.2 failed with error -22

What motherboard is this?
> 
> prasad@prasad-kvm:~$ lspci | grep -i RV
> 02:00.0 VGA compatible controller: ATI Technologies Inc RV370 5B64
> [FireGL V3100 (PCIE)] (rev 80)
> 02:00.1 Display controller: ATI Technologies Inc RV370 5B64 [FireGL
> V3100 (PCIE)] (Secondary) (rev 80)

So, how come you aren't passing in 02:00.1?
> 
> > Look for 'DMAR GFX' on Google. Might shed some more light.
> >
> 
> No DMAR for AMD. I think I should have a look at DMAR GFX.
> 
> I am really thankful to you for your support.

Hopefully I haven't confused you any further :-)
> 
> Thanks and Regards,
> Prasad

Re: A query on frame buffers

[Prasad Joshi]

On Mon, Jan 10, 2011 at 8:24 PM, Konrad Rzeszutek Wilk
<konrad.wilk@oracle.com> wrote:
>> AMD chipset.
>
> They don't seem to have have any errata's for this GFX business.
> But they do have a passthrough mode - hopefull that is not what you have
> passed in?
>
>> > Since you mentioned that you were able to passthrough other PCI devices
>> > that means the IOMMU is working. This narrows it down.
>>
>> Yes I could pass-through a network card to VM. I also tried passing
>> through a old Sound Card to Windows 7 machine. It did not work as
>> expected, Windows 7 does not have the old drivers.
>
> Heh.
>>
>> >
>> > The big difference between other PCI devices and the graphics card
>> > is that the graphic card has its own VMM and "radeon_gart_bind" ends
>> > up programming the bus address (so virt_to_phys of the pages, and
>> > you could put in a printk there to see what it is) in the graphic
>> > VMM. This means that when the graphic card is told to fetch the
>> > writeback data it ends up using the address that was programmed
>> > in there to look. Here the IOMMU should step in and see "oh, this
>> > DMA address is for this guest, let me patch it up and actually
>> > look where this would fall in the guest space. Oh OK, let me use
>> > this value real value which correspond the guests real value."
>>
>> It will take some time for me to understand every thing you have written :)
>
> Ok. Ping me if you have some questions. Reading this might
> help (or confuse you more): http://wiki.xensource.com/xenwiki/XenPVOPSDRM
>
>>
>> >
>> > But the IOMMU has a couple of knobs. One of those is the passthrough
>> > code and the GFX mapping code. The first should be easy to spot (should
>> > tell you on bootup whether you got that enabled or not), while the other
>> > looks to be a bunch of workarounds when passing in GFX cards b/c they ..
>> > well, not sure what, but they look to not work correctly with the IOMMU.
>>
>> By GFX Mapping,
>> Do you mean the code that maps VGA frame-buffers to Guest OS?
>
> No. It is the IOMMU kernel code that checks to see if this is a GFX
> card and if so does something fancy. But the workarounds for this appear
> to be exclusive to Intel chipset so you shouldn't hit any.
>
>>
>> One more silly question on the frame-buffer mapping.
>>
>> I was reading a paper on xen vga pass-though. It says "When applying
>> PCI pass-through, certain memory areas of the physical machine are
>> mapped to the VM. When the guest OS writes to one of those memory
>> addresses, Xen will make sure it is actually written at the
>> appropriate address. This implicates that no other VM is able to make
>> use of that device. The frame-buffer is mapped to the machine's main
>> memory at address 0xA0000 to 0xC0000. This memory range must be mapped
>> to the VM's memory in order for the OS to address the video adapter."
> <nods>
>>
>> My question is....
>> In my case, the host machine is using Nvidia graphics card, so the
>> host frame-buffer memory 0xA0000 to 0xC0000 is being actively used by
>> host. Now if the ATI card maps it's framebuffer to this same address
>> wouldn't it cause any problem.
>
> Ah, but it does not. The first card that is called by the BIOS (so your
> motherboard BIOS) sets itself up to use that memory. The other card has
> to wait. Keep in mind that the A0000 to C0000 is the old style VGA
> framebuffer (80x25) or if you program the card properly it can do
> VESA graphics, but nothing else. Usually the guest gets a Cirrus Logic
> video card passed in which sets this device up (and touching that
> area in the guest ends up in VNC window or SDL window).
>
> Anyhow back to your host..
> If you grep in your kernel log you should see something like:
> [drm] fb mappable at 0xE0142000
>
> This is the framebuffer address.
>>
>> If the the machine has only 1 graphics card those statement make
>> sense, but I could not understand what will happen when there are two
>> graphics cards, each connected to different monitor and assigned to
>> different VM, where would each machines frame-buffer reside in the
>> host memory.
>
> Check the kernel log. Usually it is also the first BAR in the pci device.
>
>>
>> Does Xen allow passing-though multiple graphics cards, each to different VM?
>
> Yes (with some patches posted by the AMD and Intel folks)..
> (look for threads that have some GFX or Radeon or something like that in its title).
>>
>> >
>> > If you see 'identity map for device' where the device is your GFX
>> > then that looks to be the bug. It shouldn't set the identity mapping on it.
>> >
>>
>> I guess the identity map is used only on the Intel chipset. Here are
>> msgs on host machine when addition debugging parameter
>> (amd_iommu_dump) was added.
>>
>> prasad@prasad-kvm:~$ dmesg | grep -i -e iommu -e dmar -e amd
>> [    0.000000] Command line: BOOT_IMAGE=/vmlinuz-2.6.37-rc6+
>> root=/dev/sda1 ro iommu=1 amd_iommu_dump
>> [    0.000000] ACPI: SRAT 00000000bfe9f8b0 00108 (v01 AMD    FAM_F_10
>> 00000002 AMD  00000001)
>> [    0.000000] ACPI: IVRS 00000000bfe9fa00 000D8 (v01  AMD     RD890S
>> 00202031 AMD  00000000)
>> [    0.000000] ACPI: SSDT 00000000bfe9fae0 00DA4 (v01 A M I  POWERNOW
>> 00000001 AMD  00000001)
>> [    0.000000] Kernel command line: BOOT_IMAGE=/vmlinuz-2.6.37-rc6+
>> root=/dev/sda1 ro iommu=1 amd_iommu_dump
>> [    0.000000] Please enable the IOMMU option in the BIOS setup
>> [    0.012918] Performance Events: AMD PMU driver.
>> [    0.137960] CPU0: AMD Phenom(tm) II X6 1090T Processor stepping 00
>> [    0.300011] System has AMD C1E enabled
>> [    2.680386] AMD-Vi: device: 00:00.2 cap: 0040 seg: 0 flags: 3e info 1300
>> [    2.680441] AMD-Vi:        mmio-addr: 00000000f6000000
>> [    2.680670] AMD-Vi:   DEV_SELECT_RANGE_START        devid: 00:00.0 flags: 00
>> [    2.680726] AMD-Vi:   DEV_RANGE_END                 devid: 00:00.2
>> [    2.680776] AMD-Vi:   DEV_SELECT                    devid: 00:02.0 flags: 00
>> [    2.680827] AMD-Vi:   DEV_SELECT                    devid: 07:00.0 flags: 00
>> [    2.680883] AMD-Vi:   DEV_SELECT                    devid: 00:04.0 flags: 00
>> [    2.680934] AMD-Vi:   DEV_SELECT                    devid: 06:00.0 flags: 00
>> [    2.680984] AMD-Vi:   DEV_SELECT                    devid: 00:05.0 flags: 00
>> [    2.681035] AMD-Vi:   DEV_SELECT                    devid: 05:00.0 flags: 00
>> [    2.681086] AMD-Vi:   DEV_SELECT                    devid: 00:06.0 flags: 00
>> [    2.681137] AMD-Vi:   DEV_SELECT                    devid: 04:00.0 flags: 00
>> [    2.681187] AMD-Vi:   DEV_SELECT                    devid: 00:07.0 flags: 00
>> [    2.681238] AMD-Vi:   DEV_SELECT                    devid: 03:00.0 flags: 00
>> [    2.681289] AMD-Vi:   DEV_SELECT                    devid: 00:0b.0 flags: 00
>> [    2.681340] AMD-Vi:   DEV_SELECT_RANGE_START        devid: 02:00.0 flags: 00
>> [    2.681391] AMD-Vi:   DEV_RANGE_END                 devid: 02:00.1
>> [    2.681441] AMD-Vi:   DEV_SELECT                    devid: 00:11.0 flags: 00
>> [    2.681492] AMD-Vi:   DEV_SELECT_RANGE_START        devid: 00:12.0 flags: 00
>> [    2.681543] AMD-Vi:   DEV_RANGE_END                 devid: 00:12.2
>> [    2.681594] AMD-Vi:   DEV_SELECT_RANGE_START        devid: 00:13.0 flags: 00
>> [    2.681645] AMD-Vi:   DEV_RANGE_END                 devid: 00:13.2
>> [    2.681695] AMD-Vi:   DEV_SELECT                    devid: 00:14.0 flags: d7
>> [    2.681746] AMD-Vi:   DEV_SELECT                    devid: 00:14.1 flags: 00
>> [    2.681797] AMD-Vi:   DEV_SELECT                    devid: 00:14.2 flags: 00
>> [    2.681847] AMD-Vi:   DEV_SELECT                    devid: 00:14.3 flags: 00
>> [    2.681898] AMD-Vi:   DEV_SELECT                    devid: 00:14.4 flags: 00
>> [    2.681949] AMD-Vi:   DEV_ALIAS_RANGE               devid: 01:00.0 flags: 00
>> devid_to: 00:14.4
>> [    2.682010] AMD-Vi:   DEV_RANGE_END                 devid: 01:1f.7
>> [    2.682855] AMD-Vi:   DEV_SELECT                    devid: 00:14.5 flags: 00
>> [    2.682906] AMD-Vi:   DEV_SELECT_RANGE_START        devid: 00:16.0 flags: 00
>> [    2.682957] AMD-Vi:   DEV_RANGE_END                 devid: 00:16.2
>> [    2.683234] AMD-Vi: Enabling IOMMU at 0000:00:00.2 cap 0x40
>> [    2.692043] AMD-Vi: Lazy IO/TLB flushing enabled
>> [    4.360112] ehci_hcd 0000:00:12.2: applying AMD
>> SB700/SB800/Hudson-2/3 EHCI dummy qh workaround
>> [    4.380635] ehci_hcd 0000:00:13.2: applying AMD
>> SB700/SB800/Hudson-2/3 EHCI dummy qh workaround
>> [    4.400658] ehci_hcd 0000:00:16.2: applying AMD
>> SB700/SB800/Hudson-2/3 EHCI dummy qh workaround
>> [    4.811320] powernow-k8: Found 1 AMD Phenom(tm) II X6 1090T
>> Processor (6 cpu cores) (version 2.20.00)
>> [   12.349919] EDAC amd64_edac:  Ver: 3.3.0 Dec 21 2010
>> [   12.350018] EDAC amd64: This node reports that Memory ECC is
>> currently disabled, set F3x44[22] (0000:00:18.3).
>> [   12.350030] EDAC amd64: ECC disabled in the BIOS or no ECC
>> capability, module will not load.
>> [   12.350044] amd64_edac: probe of 0000:00:18.2 failed with error -22
>
> What motherboard is this?

ASUS M4A89TD Pro/USB3, it is mentioned on the link
http://wiki.xensource.com/xenwiki/VTdHowTo

>>
>> prasad@prasad-kvm:~$ lspci | grep -i RV
>> 02:00.0 VGA compatible controller: ATI Technologies Inc RV370 5B64
>> [FireGL V3100 (PCIE)] (rev 80)
>> 02:00.1 Display controller: ATI Technologies Inc RV370 5B64 [FireGL
>> V3100 (PCIE)] (Secondary) (rev 80)
>
> So, how come you aren't passing in 02:00.1?

I tried passing through 02.00.1 as well. When I pass it to a Linux
machine I do not see the device bind to a driver, I don't see the drm
logs in the dmsg. So I thought I am not supposed to pass the 02:00:1.

Should I be passing 02.00.1 instead of 02.00.0?

>>
>> > Look for 'DMAR GFX' on Google. Might shed some more light.
>> >
>>
>> No DMAR for AMD. I think I should have a look at DMAR GFX.
>>
>> I am really thankful to you for your support.
>
> Hopefully I haven't confused you any further :-)
>>
>> Thanks and Regards,
>> Prasad
>

Re: A query on frame buffers

[Alex Deucher]

On Tue, Jan 11, 2011 at 11:58 AM, Prasad Joshi <prasadjoshi124@gmail.com> wrote:
> On Mon, Jan 10, 2011 at 8:24 PM, Konrad Rzeszutek Wilk
> <konrad.wilk@oracle.com> wrote:
>>> AMD chipset.
>>
>> They don't seem to have have any errata's for this GFX business.
>> But they do have a passthrough mode - hopefull that is not what you have
>> passed in?
>>
>>> > Since you mentioned that you were able to passthrough other PCI devices
>>> > that means the IOMMU is working. This narrows it down.
>>>
>>> Yes I could pass-through a network card to VM. I also tried passing
>>> through a old Sound Card to Windows 7 machine. It did not work as
>>> expected, Windows 7 does not have the old drivers.
>>
>> Heh.
>>>
>>> >
>>> > The big difference between other PCI devices and the graphics card
>>> > is that the graphic card has its own VMM and "radeon_gart_bind" ends
>>> > up programming the bus address (so virt_to_phys of the pages, and
>>> > you could put in a printk there to see what it is) in the graphic
>>> > VMM. This means that when the graphic card is told to fetch the
>>> > writeback data it ends up using the address that was programmed
>>> > in there to look. Here the IOMMU should step in and see "oh, this
>>> > DMA address is for this guest, let me patch it up and actually
>>> > look where this would fall in the guest space. Oh OK, let me use
>>> > this value real value which correspond the guests real value."
>>>
>>> It will take some time for me to understand every thing you have written :)
>>
>> Ok. Ping me if you have some questions. Reading this might
>> help (or confuse you more): http://wiki.xensource.com/xenwiki/XenPVOPSDRM
>>
>>>
>>> >
>>> > But the IOMMU has a couple of knobs. One of those is the passthrough
>>> > code and the GFX mapping code. The first should be easy to spot (should
>>> > tell you on bootup whether you got that enabled or not), while the other
>>> > looks to be a bunch of workarounds when passing in GFX cards b/c they ..
>>> > well, not sure what, but they look to not work correctly with the IOMMU.
>>>
>>> By GFX Mapping,
>>> Do you mean the code that maps VGA frame-buffers to Guest OS?
>>
>> No. It is the IOMMU kernel code that checks to see if this is a GFX
>> card and if so does something fancy. But the workarounds for this appear
>> to be exclusive to Intel chipset so you shouldn't hit any.
>>
>>>
>>> One more silly question on the frame-buffer mapping.
>>>
>>> I was reading a paper on xen vga pass-though. It says "When applying
>>> PCI pass-through, certain memory areas of the physical machine are
>>> mapped to the VM. When the guest OS writes to one of those memory
>>> addresses, Xen will make sure it is actually written at the
>>> appropriate address. This implicates that no other VM is able to make
>>> use of that device. The frame-buffer is mapped to the machine's main
>>> memory at address 0xA0000 to 0xC0000. This memory range must be mapped
>>> to the VM's memory in order for the OS to address the video adapter."
>> <nods>
>>>
>>> My question is....
>>> In my case, the host machine is using Nvidia graphics card, so the
>>> host frame-buffer memory 0xA0000 to 0xC0000 is being actively used by
>>> host. Now if the ATI card maps it's framebuffer to this same address
>>> wouldn't it cause any problem.
>>
>> Ah, but it does not. The first card that is called by the BIOS (so your
>> motherboard BIOS) sets itself up to use that memory. The other card has
>> to wait. Keep in mind that the A0000 to C0000 is the old style VGA
>> framebuffer (80x25) or if you program the card properly it can do
>> VESA graphics, but nothing else. Usually the guest gets a Cirrus Logic
>> video card passed in which sets this device up (and touching that
>> area in the guest ends up in VNC window or SDL window).
>>
>> Anyhow back to your host..
>> If you grep in your kernel log you should see something like:
>> [drm] fb mappable at 0xE0142000
>>
>> This is the framebuffer address.
>>>
>>> If the the machine has only 1 graphics card those statement make
>>> sense, but I could not understand what will happen when there are two
>>> graphics cards, each connected to different monitor and assigned to
>>> different VM, where would each machines frame-buffer reside in the
>>> host memory.
>>
>> Check the kernel log. Usually it is also the first BAR in the pci device.
>>
>>>
>>> Does Xen allow passing-though multiple graphics cards, each to different VM?
>>
>> Yes (with some patches posted by the AMD and Intel folks)..
>> (look for threads that have some GFX or Radeon or something like that in its title).
>>>
>>> >
>>> > If you see 'identity map for device' where the device is your GFX
>>> > then that looks to be the bug. It shouldn't set the identity mapping on it.
>>> >
>>>
>>> I guess the identity map is used only on the Intel chipset. Here are
>>> msgs on host machine when addition debugging parameter
>>> (amd_iommu_dump) was added.
>>>
>>> prasad@prasad-kvm:~$ dmesg | grep -i -e iommu -e dmar -e amd
>>> [    0.000000] Command line: BOOT_IMAGE=/vmlinuz-2.6.37-rc6+
>>> root=/dev/sda1 ro iommu=1 amd_iommu_dump
>>> [    0.000000] ACPI: SRAT 00000000bfe9f8b0 00108 (v01 AMD    FAM_F_10
>>> 00000002 AMD  00000001)
>>> [    0.000000] ACPI: IVRS 00000000bfe9fa00 000D8 (v01  AMD     RD890S
>>> 00202031 AMD  00000000)
>>> [    0.000000] ACPI: SSDT 00000000bfe9fae0 00DA4 (v01 A M I  POWERNOW
>>> 00000001 AMD  00000001)
>>> [    0.000000] Kernel command line: BOOT_IMAGE=/vmlinuz-2.6.37-rc6+
>>> root=/dev/sda1 ro iommu=1 amd_iommu_dump
>>> [    0.000000] Please enable the IOMMU option in the BIOS setup
>>> [    0.012918] Performance Events: AMD PMU driver.
>>> [    0.137960] CPU0: AMD Phenom(tm) II X6 1090T Processor stepping 00
>>> [    0.300011] System has AMD C1E enabled
>>> [    2.680386] AMD-Vi: device: 00:00.2 cap: 0040 seg: 0 flags: 3e info 1300
>>> [    2.680441] AMD-Vi:        mmio-addr: 00000000f6000000
>>> [    2.680670] AMD-Vi:   DEV_SELECT_RANGE_START        devid: 00:00.0 flags: 00
>>> [    2.680726] AMD-Vi:   DEV_RANGE_END                 devid: 00:00.2
>>> [    2.680776] AMD-Vi:   DEV_SELECT                    devid: 00:02.0 flags: 00
>>> [    2.680827] AMD-Vi:   DEV_SELECT                    devid: 07:00.0 flags: 00
>>> [    2.680883] AMD-Vi:   DEV_SELECT                    devid: 00:04.0 flags: 00
>>> [    2.680934] AMD-Vi:   DEV_SELECT                    devid: 06:00.0 flags: 00
>>> [    2.680984] AMD-Vi:   DEV_SELECT                    devid: 00:05.0 flags: 00
>>> [    2.681035] AMD-Vi:   DEV_SELECT                    devid: 05:00.0 flags: 00
>>> [    2.681086] AMD-Vi:   DEV_SELECT                    devid: 00:06.0 flags: 00
>>> [    2.681137] AMD-Vi:   DEV_SELECT                    devid: 04:00.0 flags: 00
>>> [    2.681187] AMD-Vi:   DEV_SELECT                    devid: 00:07.0 flags: 00
>>> [    2.681238] AMD-Vi:   DEV_SELECT                    devid: 03:00.0 flags: 00
>>> [    2.681289] AMD-Vi:   DEV_SELECT                    devid: 00:0b.0 flags: 00
>>> [    2.681340] AMD-Vi:   DEV_SELECT_RANGE_START        devid: 02:00.0 flags: 00
>>> [    2.681391] AMD-Vi:   DEV_RANGE_END                 devid: 02:00.1
>>> [    2.681441] AMD-Vi:   DEV_SELECT                    devid: 00:11.0 flags: 00
>>> [    2.681492] AMD-Vi:   DEV_SELECT_RANGE_START        devid: 00:12.0 flags: 00
>>> [    2.681543] AMD-Vi:   DEV_RANGE_END                 devid: 00:12.2
>>> [    2.681594] AMD-Vi:   DEV_SELECT_RANGE_START        devid: 00:13.0 flags: 00
>>> [    2.681645] AMD-Vi:   DEV_RANGE_END                 devid: 00:13.2
>>> [    2.681695] AMD-Vi:   DEV_SELECT                    devid: 00:14.0 flags: d7
>>> [    2.681746] AMD-Vi:   DEV_SELECT                    devid: 00:14.1 flags: 00
>>> [    2.681797] AMD-Vi:   DEV_SELECT                    devid: 00:14.2 flags: 00
>>> [    2.681847] AMD-Vi:   DEV_SELECT                    devid: 00:14.3 flags: 00
>>> [    2.681898] AMD-Vi:   DEV_SELECT                    devid: 00:14.4 flags: 00
>>> [    2.681949] AMD-Vi:   DEV_ALIAS_RANGE               devid: 01:00.0 flags: 00
>>> devid_to: 00:14.4
>>> [    2.682010] AMD-Vi:   DEV_RANGE_END                 devid: 01:1f.7
>>> [    2.682855] AMD-Vi:   DEV_SELECT                    devid: 00:14.5 flags: 00
>>> [    2.682906] AMD-Vi:   DEV_SELECT_RANGE_START        devid: 00:16.0 flags: 00
>>> [    2.682957] AMD-Vi:   DEV_RANGE_END                 devid: 00:16.2
>>> [    2.683234] AMD-Vi: Enabling IOMMU at 0000:00:00.2 cap 0x40
>>> [    2.692043] AMD-Vi: Lazy IO/TLB flushing enabled
>>> [    4.360112] ehci_hcd 0000:00:12.2: applying AMD
>>> SB700/SB800/Hudson-2/3 EHCI dummy qh workaround
>>> [    4.380635] ehci_hcd 0000:00:13.2: applying AMD
>>> SB700/SB800/Hudson-2/3 EHCI dummy qh workaround
>>> [    4.400658] ehci_hcd 0000:00:16.2: applying AMD
>>> SB700/SB800/Hudson-2/3 EHCI dummy qh workaround
>>> [    4.811320] powernow-k8: Found 1 AMD Phenom(tm) II X6 1090T
>>> Processor (6 cpu cores) (version 2.20.00)
>>> [   12.349919] EDAC amd64_edac:  Ver: 3.3.0 Dec 21 2010
>>> [   12.350018] EDAC amd64: This node reports that Memory ECC is
>>> currently disabled, set F3x44[22] (0000:00:18.3).
>>> [   12.350030] EDAC amd64: ECC disabled in the BIOS or no ECC
>>> capability, module will not load.
>>> [   12.350044] amd64_edac: probe of 0000:00:18.2 failed with error -22
>>
>> What motherboard is this?
>
> ASUS M4A89TD Pro/USB3, it is mentioned on the link
> http://wiki.xensource.com/xenwiki/VTdHowTo
>
>>>
>>> prasad@prasad-kvm:~$ lspci | grep -i RV
>>> 02:00.0 VGA compatible controller: ATI Technologies Inc RV370 5B64
>>> [FireGL V3100 (PCIE)] (rev 80)
>>> 02:00.1 Display controller: ATI Technologies Inc RV370 5B64 [FireGL
>>> V3100 (PCIE)] (Secondary) (rev 80)
>>
>> So, how come you aren't passing in 02:00.1?
>
> I tried passing through 02.00.1 as well. When I pass it to a Linux
> machine I do not see the device bind to a driver, I don't see the drm
> logs in the dmsg. So I thought I am not supposed to pass the 02:00:1.
>
> Should I be passing 02.00.1 instead of 02.00.0?

02:00.0 is the actual device.  02:00.1 is just a placeholder for
supporting dualhead certain other OSes.

Alex

>
>>>
>>> > Look for 'DMAR GFX' on Google. Might shed some more light.
>>> >
>>>
>>> No DMAR for AMD. I think I should have a look at DMAR GFX.
>>>
>>> I am really thankful to you for your support.
>>
>> Hopefully I haven't confused you any further :-)
>>>
>>> Thanks and Regards,
>>> Prasad
>>
> _______________________________________________
> dri-devel mailing list
> dri-devel@lists.freedesktop.org
> http://lists.freedesktop.org/mailman/listinfo/dri-devel
>

Re: A query on frame buffers

[Konrad Rzeszutek Wilk]

> > What motherboard is this?
> 
> ASUS M4A89TD Pro/USB3, it is mentioned on the link
> http://wiki.xensource.com/xenwiki/VTdHowTo

I am not sure how the implementation of the IOMMU
in the Xen hypervisor is different from the Linux implementation.
Usually it is lock-step, but it might be different.

Also, Xen's QEMU has different mechanism for handling PCI passthrough
so ... 

Lots of variables to figure out why it does not work for you.
It might make sense to start first with a working case and from there
start switching over to KVM.