Re: [Qemu-devel] [PATCH v2] x86: gigabyte alignment for ram

["Michael S. Tsirkin" <mst@redhat.com>]

On Mon, Dec 16, 2013 at 02:46:17PM +0100, Gerd Hoffmann wrote:
> On Mo, 2013-12-16 at 13:54 +0200, Michael S. Tsirkin wrote:
> > On Mon, Dec 16, 2013 at 10:11:28AM +0100, Gerd Hoffmann wrote:
> > > Map 3G (i440fx) or 2G (q35) of memory below 4G, so the RAM pieces
> > > are nicely aligned to gigabyte borders.
> > > 
> > > Keep old memory layout for (a) old machine types and (b) in case all
> > > memory fits below 4G and thus we don't have to split RAM into pieces
> > > in the first place.  The later makes sure this change doesn't take
> > > away memory from 32bit guests.
> > > 
> > > So, with i440fx and up to 3.5 GB of memory, all of it will be mapped
> > > below 4G.  With more than 3.5 GB of memory 3 GB will be mapped below
> > > 4G and the remaining amount will be mapped above 4G.
> > > 
> > > Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
> > 
> > 
> > OK the piix part looks ok to me.  For now I split this
> > int two patches: for q35 and piix and parked this on my PCI tree.
> > pushed, so pls check it out.
> > I also added some comments - see patches I've sent on list.
> > 
> > However, I'm not sure why do we reserve so much memory for q35.
> > 
> > I re-checked the pci express spec, it explicitly says
> > (Table 7-1: Enhanced Configuration Address Mapping)
> > that address bits used for ECAM (aka MMCFG) are 20 + n - 1 to 0,
> > wheren n is bits in the bus number field, so up to 8.
> > Doing the math we need up to 28 bits that is 256 Megabytes
> > of memory.
> 
> Correct.
> 
> > So what's the issue with using up to 3G for RAM?
> > This makes me think the only issue is that it seems to conflict
> > with MCH_HOST_BRIDGE_PCIEXBAR_DEFAULT, which we should just get rid of -
> > it's never actually used.
> 
> Indeed, that has no use because the firmware initializes the xbar
> anyway.
> 
> Problem is that the firmware places the xbar @ 0xb000000.
> Hardcoded, assuming qemu will not map ram above 0xb0000000.

Can't bios figure out the size of memory below 4G from fwcfg?
I refer to 7db16f2480db5e246d34d0c453cff4f58549df0e specifically.


> So, we must (a) fix firmware first and (b) get a ugly dependency
> that older firmware will not run on latest qemu.

That's only important for old machine types though, right?

> We also need to figure how we wanna fixup things.  So, current memory
> layout looks like this:
> 
>    0x00000000 - 0xafffffff  --  RAM / unused
>    0xb0000000 - 0xbfffffff  --  mmconfig xbar [256 pci busses]
>    0xc0000000 - 0xfec00000  --  space for pci bars, almost 1g
> 
> Largest pci bar we can map below 4g is 512m, @ 0xc0000000.
> 
> If we wanna map 3G RAM we need to move the xbar somewhere else.  Big
> question is where?
> 
> We can move it to 0xc0000000.  Then we can't map 512m pci bars any more.

I would go for this when we have 3G of RAM.
I think that ability to support a single 512m BAR is not all that important.

> 
> We can move it to 0xe0000000.  Then we have to split the pci bar space,
> mapping large bars below 0xe0000000 and small ones above 0xf0000000.
> SeaBIOS pci init code isn't really up to it.
> Could also become tricky
> to declare it correctly in acpi / e820 due to the split.

My laptop's ACPI has this space all fragmented up, seems to boot fine ...

> We can move it to 0xf0000000 or 0xf8000000, then map all pci bars below
> the xbar.  We have to make the xbar smaller then, reducing the number of
> pci busses the mmconf area can handle.  My laptop looks like this.  Not
> sure this a good move for qemu though, do we really want to limit the
> number of busses we can handle, with everything moving to pci express?
> 
> cheers,
>   Gerd

I don't think it's good. With bus per device we are limited to 256
devices as it is.