[Qemu-devel] Re: [PATCH 12/25] pci: 64bit bar support.

["Michael S. Tsirkin" <mst@redhat.com>]

On Mon, Oct 05, 2009 at 07:26:47PM +0900, Isaku Yamahata wrote:
> On Mon, Oct 05, 2009 at 12:08:42PM +0200, Michael S. Tsirkin wrote:
> > On Mon, Oct 05, 2009 at 06:45:06PM +0900, Isaku Yamahata wrote:
> > > On Sun, Oct 04, 2009 at 12:26:46PM +0200, Michael S. Tsirkin wrote:
> > > > On Sat, Oct 03, 2009 at 05:16:04AM +0900, Isaku Yamahata wrote:
> > > > > implemented pci 64bit bar support.
> > > > > 
> > > > > Signed-off-by: Isaku Yamahata <yamahata@valinux.co.jp>
> > > > > ---
> > > > >  hw/pci.c |   41 ++++++++++++++++++++++++++++++++++++-----
> > > > >  hw/pci.h |    9 +++++++++
> > > > >  2 files changed, 45 insertions(+), 5 deletions(-)
> > > > > 
> > > > > diff --git a/hw/pci.c b/hw/pci.c
> > > > > index 510fad2..75af2cd 100644
> > > > > --- a/hw/pci.c
> > > > > +++ b/hw/pci.c
> > > > > @@ -427,8 +427,13 @@ void pci_register_bar(PCIDevice *pci_dev, int region_num,
> > > > >          addr = 0x10 + region_num * 4;
> > > > >      }
> > > > >      pci_set_long(pci_dev->config + addr, type);
> > > > > -    pci_set_long(pci_dev->wmask + addr, wmask & 0xffffffff);
> > > > > -    pci_set_long(pci_dev->cmask + addr, 0xffffffff);
> > > > > +    if (pci_bar_is_64bit(r)) {
> > > > > +        pci_set_quad(pci_dev->wmask + addr, wmask);
> > > > > +        pci_set_quad(pci_dev->cmask + addr, ~0ULL);
> > > > > +    } else {
> > > > > +        pci_set_long(pci_dev->wmask + addr, wmask & 0xffffffff);
> > > > > +        pci_set_long(pci_dev->cmask + addr, 0xffffffff);
> > > > > +    }
> > > > >  }
> > > > >  
> > > > >  static void pci_update_mappings(PCIDevice *d)
> > > > > @@ -462,7 +467,11 @@ static void pci_update_mappings(PCIDevice *d)
> > > > >                  }
> > > > >              } else {
> > > > >                  if (cmd & PCI_COMMAND_MEMORY) {
> > > > > -                    new_addr = pci_get_long(d->config + config_ofs);
> > > > > +                    if (pci_bar_is_64bit(r)) {
> > > > > +                        new_addr = pci_get_quad(d->config + config_ofs);
> > > > > +                    } else {
> > > > > +                        new_addr = pci_get_long(d->config + config_ofs);
> > > > > +                    }
> > > > >                      /* the ROM slot has a specific enable bit */
> > > > >                      if (i == PCI_ROM_SLOT && !(new_addr & 1))
> > > > >                          goto no_mem_map;
> > > > > @@ -477,7 +486,7 @@ static void pci_update_mappings(PCIDevice *d)
> > > > >  
> > > > >                          /* keep old behaviour
> > > > >                           * without this, PC ide doesn't work well. */
> > > > > -                        last_addr >= UINT32_MAX) {
> > > > > +                        (!pci_bar_is_64bit(r) && last_addr >= UINT32_MAX)) {
> > > > >                          new_addr = PCI_BAR_UNMAPPED;
> > > > 
> > > > I don't really understand what is this doing
> > > 
> > > I'll add a comment above the line in the patch.
> > > 
> > > In a word, it is a work around of qemu implementation limitation.
> > > So far bar was only 32bit, just wrapping around check worked.
> > > But now it's 64bit, explicit 4G check becomes needed.
> > > OS writes all one bits to BAR register to detect its size
> > > meaning that memory 32 bit bar is once set right below 4GB,
> > > and then maps the area to where OS really want it to exist.
> > 
> > PCI spec actually says:
> > 	Decode (I/O or memory) of a register is disabled
> > 	via the command register before sizing a Base Address register.
> > Is the OS in question out of spec then?
> > 
> > > Since mapping memory space at 4G causes troubles,
> > 
> > What kind of trouble? Does it happen to clash with the -1 value we use for
> > invalid regions? Would changing invalid region to some other value help?
> 
> PCI ide didn't work well on Linux. Linux failed to find any disks.
> Changing type from uint32_t to uint64_t causes the logic of
> address wrapping around of 32bit bar.
> So I had to keep 32bit bar wrap around logic explicitly
> with uint64_t to boot Linux.

I'm not saying it's wrong.
I'm just asking for clarification on how this helps.

> 
> > > pci_update_mapping() have to work around not to map right below 4G.
> > 
> > Thats out of spec, isn't it? Please add a TODO so we'll remember to fix it.
> 
> Right. Will do.
> 
> -- 
> yamahata