Re: [PATCH 00/23] Add KVM support for PPC64 (970) hosts

[Benjamin Herrenschmidt <benh@kernel.crashing.org>]

> > The cache line sizes vary between processors, that is, the problem is
> > not only classic 32 "32 bytes" vs. classic 64 "128 bytes", some CPUs
> > have 64 bytes cache line size for example too.
> 
> Which non-booke CPUs have 64 byte cache line size? I thought all BookS  
> PPC64 ones are on 128 bytes.

PA-Semi, maybe rs64 (not sure, I have some vague memories here though
the later is only supported on some legacy iSeries which we may not care
about... the kernel cputable still says 128 though, but maybe the
device-tree says otherwise).

Also, for 32-bit, there's some e500's that have 64 byte cache line size
and I know of at least one thing not released yet that will be 32-bits
and will have 128 d-side and 32 i-side :-) (so it can be tricked into
making look like i side is 128 too).

> Well, we only need to tell our guest firmware what the host cache line  
> size is, which qemu knows already, as it's passed to userspace.

Right. As long as the guest OS does the right thing, which afaik only
linux 64-bit does at the moment, though we plan to fix linux 32-bit.

> MacOS always sets the HID5 dcbz32 bit and assumes 128 bytes for dcbzl.

Which will be a problem on processor that don't support that bit.. your
patching technique is interesting though. It should probably remain an
option in case it causes trouble.

For MacOS X, it's reasonably easy to fix things up with paravirt
extensions to a certain extent. For MOL, we have some optional add-ons
you can install inside OS X that patch its kernel in various ways to
make it work better/faster in MOL by avoiding some nasty constructs,
such as abuse of split MMU mode (IR clear DR set or the other way
around) (thanks for Apple publishing their source code here :-)

> It'd be really nice to have 32-bit Linux be a bit more clever about  
> this. Right now I have a dcbz hack in that makes all dcbz basically be  
> dcbz32 when running in book3s_32 mode or HID5.dcbz32 = 1 for book3s_64.
>
> That hack is either achieved by setting HID5.dcbz32 = 1 on the host  
> (if possible) or by runtime binary patching of the guest.

Yup, I saw.

> > The patching technique Alexander does on executable pages is fun :-)
> > Though it's also both a bit slow and potentially dangerous (you don't
> > know for sure that what you are patching is actually an instruction).
> 
> Yeah, we also lose nx capabilites in the guest atm. We're rather safe  
> on not patching data by only patching on execution, but you're right -  
> there could still be data in executed pages.

Well, most 32-bit "S" processors don't support NX at the PTE level
anyway, only at the segment level.

> > Maybe it should be a configuration option to only be enabled when
> > needed.
> 
> It _is_ needed for every possible configuration atm :-). PPC32 Linux  
> doesn't boot without. PPC64 Linux doesn't set dcbz32.

Right. We do need to fix ppc32 linux :-) dcbz32 only exists on 970, not
on other processors so we can't really make that more generic.

Cheers,
Ben.