Resetting a PCI device's BAR registers in HVM guest

Resetting a PCI device's BAR registers in HVM guest

[Dan Gora]

Hi,

I have a technical question that I thought would probably be better
here than in xen-users.  Excuse me if I'm wrong...

I'm have a driver for a PCI card of ours which when reset will clear
the card's BAR registers (along with the rest of of the PCI
configuration space).  We need to be able to reset the board while the
machine is running when the driver is being loaded or being unloaded
or we need to update the board level software.  What we typically do
is to suck the BAR registers (and the rest of config space) out of PCI
configuration space in the driver attach routine, then whenever we
reset the board we just dump the same values back in.  As you might
have guessed by now, the problem with running on a HVM guest is that
the BAR registers that are exposed to the guest are not the same as
the ones actually _in_ the hardware (these we can see from the dom0,
but not the domU), so the question is "how can I get access to the
_real_ BAR registers, so that I can get the proper values (the ones
seen from dom0) and put them back"?  pci_read_config_XX just reads the
domU versions.  Is there a xen hypercall for this?  Will the
hypervisor not only trap the reads to config space, but the writes
back as well?

thanks in advance for any info..
dan

Re: Resetting a PCI device's BAR registers in HVM guest

[Keir Fraser]

On 09/02/2010 00:07, "Dan Gora" <dan.gora@gmail.com> wrote:

> I'm have a driver for a PCI card of ours which when reset will clear
> the card's BAR registers (along with the rest of of the PCI
> configuration space).  We need to be able to reset the board while the
> machine is running when the driver is being loaded or being unloaded
> or we need to update the board level software.  What we typically do
> is to suck the BAR registers (and the rest of config space) out of PCI
> configuration space in the driver attach routine, then whenever we
> reset the board we just dump the same values back in.  As you might
> have guessed by now, the problem with running on a HVM guest is that
> the BAR registers that are exposed to the guest are not the same as
> the ones actually _in_ the hardware (these we can see from the dom0,
> but not the domU), so the question is "how can I get access to the
> _real_ BAR registers, so that I can get the proper values (the ones
> seen from dom0) and put them back"?  pci_read_config_XX just reads the
> domU versions.  Is there a xen hypercall for this?  Will the
> hypervisor not only trap the reads to config space, but the writes
> back as well?

All PCI config space accesses will trap to qemu-dm (our qemu-based device
model running as a per-guest process in dom0). Look in our
qemu-xen-unstable.git:hw/passthrough.c for what happens there. I think BARs
are totally virtualised and you cannot get through-access currently. You
could I suppose hack it in by shadowing the real BARs at a usually-unused
place in the PCI config space. The problem more generally will be that this
can break VM sandboxing (badness like BARs overlapping other devices, etc)
so questionable whether this could be made a default addition, although
perhaps a per-device domain config option...

 -- Keir