PVR hell

PVR hell

[Benjamin Herrenschmidt]

Hi !

We have a problem..

Currently we have bloat of code testing against PVR, in different ways &
places, sometimes using constants, sometimes, not, and possibly missing
some CPUs (in fact, we do miss some: with the 7410 used in the TiPB, we
will miss the L2 cache and Altivec).

I'd like to get this all cleaned up. The best way to acheive that is I
beleive a bit similar to what Apple does in Darwin: a table with mask/
value pairs used to "match' the PVR. The table will then give you the CPU
name, eventually a pointer to some early init code, etc... and a bitmask
of CPU "features". This bitmask can then be used for things like "has
L2CR", "has Altivec", "has L3", "has reliable TAU", etc...

The mask/value also allow, if we want it, to match against specific revs
and if not, fallback to the generic case for a given CPU if we want a
"feature" disabled or enabled on a given CPU rev.

I want to fix that now, as I'm already biten by this problem on the
Titanium PowerBook, will be more with the 7450 used in new macs, and
whatever new PPC may come out.

The only thing for which I'm not too sure is where should I store that
"feature" bitmask once the CPU has been matched during early boot. Darwin
stores the CPU infos (including the CPU number) stuffed in an SPRG
reserved to that for the entire kernel life. This looks like a good idea,
but I'm not sure we have an SPRG free in our kernel. In most case,
accessing the current CPU feature mask in memory would be harmless as
it's really useful for one-shot operations, like tapping the L2CR.
However, there is at least one critical place (and maybe more) which is
testing for an altivec in the context switch code.

If we store it in memory, we can hard-code the location, meaning that we
only support configs where all CPUs share the same "feature" mask and
potentially causing cache ping-pong. Or we store it in a per-CPU
location, effectively doubling the memory overhead (retreive CPU number
via r2, then figure out the feature bit mask for the current CPU).

For now, I'm going toward the single location in memory shared by all
CPUs, as this is only read (never written), lowering the possibility of
cache ping-pong, with the address of this location resolved at compile/
link time.

Any comment ?

Note: This _is_ a _2_4 thing, I'm really tired of the PVR mess in the
current code. We have bits in head.S, misc.S, ppc_htab.c, entry.S,
setup.c, pmac_setup.c, maybe more, with not always complete tables, some
of them only recently updated for G4, all of them missing the 7410 (I
don't even think about the 7450), etc...

Ben.


** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

Re: PVR hell

[Benjamin Herrenschmidt]

>
>I'd like to get this all cleaned up. The best way to acheive that is I
>beleive a bit similar to what Apple does in Darwin: a table with mask/
>value pairs used to "match' the PVR. The table will then give you the CPU
>name, eventually a pointer to some early init code, etc... and a bitmask
>of CPU "features". This bitmask can then be used for things like "has
>L2CR", "has Altivec", "has L3", "has reliable TAU", etc...

Obviously, my intend is to have the table lookup done only once during
kernel boot (head.S), and all further code relying on the table index & CPU
features bit mask retreived at this point to display CPU name, get specific
CPU features, call CPU init code (for cache init, etc...).

I'm pretty sure some bits of head.S would benefit from it too ;)

Ben.


** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

Re: PVR hell

[Gabriel Paubert]

On Tue, 27 Mar 2001, Benjamin Herrenschmidt wrote:

> Hi !
>
> We have a problem..
>
> Currently we have bloat of code testing against PVR, in different ways &
> places, sometimes using constants, sometimes, not, and possibly missing
> some CPUs (in fact, we do miss some: with the 7410 used in the TiPB, we
> will miss the L2 cache and Altivec).
>
> I'd like to get this all cleaned up. The best way to acheive that is I
> beleive a bit similar to what Apple does in Darwin: a table with mask/
> value pairs used to "match' the PVR. The table will then give you the CPU
> name, eventually a pointer to some early init code, etc... and a bitmask
> of CPU "features". This bitmask can then be used for things like "has
> L2CR", "has Altivec", "has L3", "has reliable TAU", etc...

That's exactly why I did not like the solution of emulting mfpvr in user
space: applications or libraries using Altivec on a 7400 will have to be
at least recompiled when run on later processors.

For this we should define additional values in the auxiliary table which
is passed to applications and forget about emulating mfpvr (a trick I have
systematically patched out in my kernels BTW).

> The mask/value also allow, if we want it, to match against specific revs
> and if not, fallback to the generic case for a given CPU if we want a
> "feature" disabled or enabled on a given CPU rev.

BTW, there is also another feature that you forget about and for which
mfpvr is currently used (hidden in USE_RTC macro): rtc or timebase for
gettimeofday() and friends.

> I want to fix that now, as I'm already biten by this problem on the
> Titanium PowerBook, will be more with the 7450 used in new macs, and
> whatever new PPC may come out.
>
> The only thing for which I'm not too sure is where should I store that
> "feature" bitmask once the CPU has been matched during early boot. Darwin
> stores the CPU infos (including the CPU number) stuffed in an SPRG
> reserved to that for the entire kernel life. This looks like a good idea,
> but I'm not sure we have an SPRG free in our kernel. In most case,
> accessing the current CPU feature mask in memory would be harmless as
> it's really useful for one-shot operations, like tapping the L2CR.
> However, there is at least one critical place (and maybe more) which is
> testing for an altivec in the context switch code.

For this an SPRG is a waste, there are not enough. I'd rather have it in
memory, cache lines which are only or mostly read are shared among
processors and do not cause ping-pongs (unless you have MP 603/603e/750).
They are even more efficiently shared with maxbus.

> If we store it in memory, we can hard-code the location, meaning that we
> only support configs where all CPUs share the same "feature" mask and
> potentially causing cache ping-pong. Or we store it in a per-CPU
> location, effectively doubling the memory overhead (retreive CPU number
> via r2, then figure out the feature bit mask for the current CPU).
>
> For now, I'm going toward the single location in memory shared by all
> CPUs, as this is only read (never written), lowering the possibility of
> cache ping-pong, with the address of this location resolved at compile/
> link time.

I basically agree for now (the definitive solution would be a boot time
link of all the critical code, but that's a long term project).

	Regards,
	Gabriel.


** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

Re: PVR hell

[Dan Malek]

Gabriel Paubert wrote:
>
> On Tue, 27 Mar 2001, Benjamin Herrenschmidt wrote:
>

> > Currently we have bloat of code testing against PVR,

I have been thinking of this quite a bit recently because most of
my projects are new processors with different PVRs.  Fortunately, I
believe a majority of conditional execution is done only at set up
time, so it shouldn't be a major concern.  I would like to find a way
to opimize some of the more frequent cache management.....

> .... in different ways &
> > places, sometimes using constants, sometimes, not, and possibly missing
> > some CPUs (in fact, we do miss some: with the 7410 used in the TiPB, we
> > will miss the L2 cache and Altivec).

With 7450, we not only miss it but there are changes in bit positions
in HID0, more things to enable/initialize, and the addition of the L3
cache.

> That's exactly why I did not like the solution of emulting mfpvr in user
> space: applications or libraries using Altivec on a 7400 will have to be
> at least recompiled when run on later processors.

That has nothing to do with emulating mfpvr (which I think most people
don't like because they didn't think of it first :-).  You can't predict
the future, only anticipate what may be coming and be prepared.  You
would have to recompile software in any case, and worse link against
a specific library for all Altivec processors.  We have to even do it
again for 7450 even if we got it right for 7410.  The real advantage to
doing this is we have a single set of library functions, that once we
add something for the new processor all of the older ones will continue
to work just fine.  The funny thing is, in the kernel we seem very
sensitive to any extra instruction we execute to help with such
modularity,
but we fully expect user applications to operate this way.

I think we should just continue the ppc_md indirect function call
concept and just use it earlier in the kernel for more things.  All
of the low level initialization in 'head.S' or other processor specific
things in other files should just be indirect fucntion calls that are
processor specific.  Sure makes porting and understanding the software
much easier.


> I basically agree for now (the definitive solution would be a boot time
> link of all the critical code, but that's a long term project).

It's kind of hard to link in a cache invalidate function when you
need a cache invalidate function for the linking (for example).  It
would
also be nice if people keep in mind that a majority of systems running
Linux today are not desktops with lots of flexibility and disk
drives......

Thanks.


	-- Dan

>
>         Regards,
>         Gabriel.

** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

Re: PVR hell

[Benjamin Herrenschmidt]

>
>I think we should just continue the ppc_md indirect function call
>concept and just use it earlier in the kernel for more things.  All
>of the low level initialization in 'head.S' or other processor specific
>things in other files should just be indirect fucntion calls that are
>processor specific.  Sure makes porting and understanding the software
>much easier.

Well, at one point, we must fill up this function table depending on the PVR
value. For my immediate Altivec & L2 needs (and for quite a bit of things
with the "desktop" family of CPUs), a feature bit mask is enough.

So what I propose is to have head.S parse a table, and when a matching
CPU is found, call it's init function (pointed to by a table entry) and
load the feature value from it.

The init function can then fill other ppc_md fields if needed. That can
be done
as a second step

I'm not sure however there is real need of completely "virtualizing" CPU
features as hard as MMU implementation or cache implementation. There is
a quite clear distinction between embedded CPUs and "desktop" CPUs, and
I see no immediate need to have a binary kernel that can run them all.
(I may be wrong here, please correct me if I am).

For now, my primary concern is about things that will affect the desktop
systems, especially the powermacs using the whole range of PPC "32 bit
desktop" CPUs from 601 to 7450. I don't think it's wise to try to have
a common binary kernel that support both those and 4xx and 8xx.

But it can be done if you really want. My proposed init table allow you
to implement it on top of it.

A typical table layout would be:

 u32    pvr_mask;
 u32    pvr_value;
 u32    feature_bits;
 void   (*init_cpu)(void)
 char*  name;

The head.S code would then iterate the table, checking if

 (PVR & pvr_mask) == pvr_value

And when found, would copy feature_bits to a global, eventually the table
entry as well (to get the CPU name and whatever other stuffs we want to
put in the table for this CPU), and call the init_cpu function.

If you want to fill additional ppc_md entries from init_cpu(), then
you are free to do it ;)

For now, I want something simple that can fit in _2_4 easily

Ben.


** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

Re: PVR hell

[Dan Malek]

Benjamin Herrenschmidt wrote:

> So what I propose is to have head.S parse a table, and when a matching
> CPU is found, call it's init function (pointed to by a table entry) and
> load the feature value from it.

That will work.

> The init function can then fill other ppc_md fields if needed. That can
> be done
> as a second step

I didn't necessarily mean to use ppc_md, just the idea of a one time
initialization of function pointers early in head.S based upon the PVR.

> ......... There is
> a quite clear distinction between embedded CPUs and "desktop" CPUs,

Not among CPUs.  Less than 25% of 74xx processors are used by Apple
in all of their systems, where do you suppose all the rest go?  The
disctinction is based upon system configurations, that you can't
assume 74xx processors are going to boot with OF, or even have a disk
for that matter.  I'm just asking we don't build software with these
assumptions, as most of the 74xx processors are going into deeply
embedded hybrid DSP environments.

>  ...... and
> I see no immediate need to have a binary kernel that can run them all.
> (I may be wrong here, please correct me if I am).

That isn't what I was suggesting either.  Even among processors we
think should be similar, like the 74xx family, there are some major
differences.  The old test and branch based upon PVR is turning into
a twisted mess of code, that I think would be better understood,
written,
and more reliable if we adopted a more modular and indirect calling
method (which I believe you are suggesting as well).

> For now, my primary concern is about things that will affect the desktop
> systems, especially the powermacs using the whole range of PPC "32 bit
> desktop" CPUs from 601 to 7450.

The only 7450 systems running Linux today are embedded PPMC with a DINK
boot rom that boot from flash........those aren't desktops.  Someday
Apple may have something available for you desktop guys, but the
initial development is for Linux embedded targets :-).

I think your table lookup stuff is just fine, and I'm ready to rewrite
some of head.S to use something different.


	-- Dan

** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

Re: PVR hell

[Benjamin Herrenschmidt]

>
>The only 7450 systems running Linux today are embedded PPMC with a DINK
>boot rom that boot from flash........those aren't desktops.  Someday
>Apple may have something available for you desktop guys, but the
>initial development is for Linux embedded targets :-).
>
>I think your table lookup stuff is just fine, and I'm ready to rewrite
>some of head.S to use something different.

Heh, there are already Apple G4s with the 7450 AFAIK ;)

I'll code something today in my rsync tree and post a patch that can
be discussed before pushing.

Ben.


** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/