bootstrap stuffs

bootstrap stuffs

[Benjamin Herrenschmidt]

On Sat, Feb 13, 1999, David Edelsohn <dje@watson.ibm.com> wrote:

Thanks for your reply !

>	Just to make sure, you are not asking how to create XCOFF
>TOC-based code using GCC?

Not really, I already have CodeWarrior and MrC for that ;-)

>	If you know the physical location or can cotrol the physical
>location at which the code will be loaded, you simply can instruct the
>linker to link the executable based at that location.  Otherwise, you
>might need additional -fpic position independent code option.  If you need
>data and cannot set the GPR specifying the GOT address yourself, you can
>use the GCC SVR4 eABI PPC suppport to calculate it on entry as it does for
>embedded systems.  Note that branches generated by GCC already are
>position-independent displacements (not "ba" branch absolute instruction).

I don't know the physical location. The experiments I did yesterday with
various options showed me that the generated branches are bl's except in
some case (the linker-generated branches between .o files were sometimes
ba, but I probably missed an ld option).

So if I understand correctly, I must use -fpic to generate position
independant code. I beleive I must also indicate that this is static code
(no dynamic link). The GOT is a TOC-like register that's it ? I could
probably put a correct value in it if I knew how to find it from the
loader. Do you know which options I should use for CFLAGS and LDFLAGS to
get the result you mentioned ? (GOT calculated on entry).


>	Are all of those "sync" instructions necessary in your code?

Just paranoid. They were added around the interrupt switching to make
sure to sync an eventual second processor and everything else before
switching the MMU off. I beleive that for disabling the MMU, andi. is
probably more efficient than my ori/andc pair, could you just explain my
why you added &0xffff at it ?

>You are not touching memory.  The instructions already are serializing.  I
>think that you should be able to do something like:
>
>enable:
>        mfmsr   r0
>        ori     r0,r0,MSR_EE
>        mtmsr   r0
>
>disable:
>        mfmsr   r0
>        andi.   r0,r0,(~MSR_EE)&0xffff
>        mtmsr   r0
>


-- 
           E-Mail: <mailto:bh40@calva.net>
BenH.      Web   : <http://calvaweb.calvacom.fr/bh40/>




[[ This message was sent via the linuxppc-dev mailing list. Replies are ]]
[[ not forced back to the list, so be sure to  Cc linuxppc-dev  if your ]]
[[ reply is of general interest. To unsubscribe from linuxppc-dev, send ]]
[[ the message 'unsubscribe' to linuxppc-dev-request@lists.linuxppc.org ]]

Re: bootstrap stuffs

[David Edelsohn]

>>>>> Benjamin Herrenschmidt writes:

>> Are all of those "sync" instructions necessary in your code?

Benjamin> Just paranoid. They were added around the interrupt switching to make
Benjamin> sure to sync an eventual second processor and everything else before
Benjamin> switching the MMU off. I beleive that for disabling the MMU, andi. is
Benjamin> probably more efficient than my ori/andc pair, could you just explain my
Benjamin> why you added &0xffff at it ?

	"andi." only operates on halfword, so one needs to make sure that
the constant is valid or the assembler will complain.  (~ 0x8000) will
produce values in the upper half of the 32-bit word.  PowerPC immediate
instructions apply to either the upper-half or lower-half of a 32-bit
word. 

	For correctness, "sync" must be used, but for best performance,
"sync" should be used sparingly, only when necessary.

	Michael's comment about the -mrelocatable flag are a more detailed
answer to your question.

David

[[ This message was sent via the linuxppc-dev mailing list. Replies are ]]
[[ not forced back to the list, so be sure to  Cc linuxppc-dev  if your ]]
[[ reply is of general interest. To unsubscribe from linuxppc-dev, send ]]
[[ the message 'unsubscribe' to linuxppc-dev-request@lists.linuxppc.org ]]

Re: bootstrap stuffs

[Benjamin Herrenschmidt]

On Sun, Feb 14, 1999, David Edelsohn <dje@watson.ibm.com> wrote:

>	For correctness, "sync" must be used, but for best performance,
>"sync" should be used sparingly, only when necessary.

Ok thanks. In this specific case (jump to bootstrap), I don't really care
about the performance loss introduced by a sync, I'll keep one just after
the mtmsr to make sure that the interrupt switch is fully done before I
hack with srr0 and the MMU.

For the relocatable code, I'll still unvestigate the issue for my own
education, but I beleive I don't need it anymore, I just wrote most of
the code in asm this morning ;-) (PPC asm is frightening at fist, but
finally it was not so difficult).

So thanks David and Michael (and thanks again to Terry Greeniaus), I now
have all the elements I need to do what I want.


-- 
           E-Mail: <mailto:bh40@calva.net>
BenH.      Web   : <http://calvaweb.calvacom.fr/bh40/>




[[ This message was sent via the linuxppc-dev mailing list. Replies are ]]
[[ not forced back to the list, so be sure to  Cc linuxppc-dev  if your ]]
[[ reply is of general interest. To unsubscribe from linuxppc-dev, send ]]
[[ the message 'unsubscribe' to linuxppc-dev-request@lists.linuxppc.org ]]

Re: bootstrap stuffs

[David Edelsohn]

>>>>> Benjamin Herrenschmidt writes:

Benjamin> On Sun, Feb 14, 1999, David Edelsohn <dje@watson.ibm.com> wrote:
>> For correctness, "sync" must be used, but for best performance,
>> "sync" should be used sparingly, only when necessary.

Benjamin> Ok thanks. In this specific case (jump to bootstrap), I don't really care
Benjamin> about the performance loss introduced by a sync, I'll keep one just after
Benjamin> the mtmsr to make sure that the interrupt switch is fully done before I
Benjamin> hack with srr0 and the MMU.

	"isync" is used to discard instruction pre-fetch and ensure that
all previous instructions have occurred.  isync is not necessary for your
situation either.

	As Gabriel correctly explained, a "sync" instruction may be
necessary before interrupts are enabled if some off-chip operation, like
modifying an interrupt controller on the bus, was performed.  That is
because of the bus operations which need to complete before interrupts can
be enabled, not because of the enabling itself.

David

[[ This message was sent via the linuxppc-dev mailing list. Replies are ]]
[[ not forced back to the list, so be sure to  Cc linuxppc-dev  if your ]]
[[ reply is of general interest. To unsubscribe from linuxppc-dev, send ]]
[[ the message 'unsubscribe' to linuxppc-dev-request@lists.linuxppc.org ]]

Re: bootstrap stuffs

[Benjamin Herrenschmidt]

On Mon, Feb 15, 1999, David Edelsohn <dje@watson.ibm.com> wrote:

>	As Gabriel correctly explained, a "sync" instruction may be
>necessary before interrupts are enabled if some off-chip operation, like
>modifying an interrupt controller on the bus, was performed.  That is
>because of the bus operations which need to complete before interrupts can
>be enabled, not because of the enabling itself.

Yep, I knew this case, the sync after switching them OFF was just a
paranoid move on my part. Thanks for the tips.

-- 
           E-Mail: <mailto:bh40@calva.net>
BenH.      Web   : <http://calvaweb.calvacom.fr/bh40/>





[[ This message was sent via the linuxppc-dev mailing list. Replies are ]]
[[ not forced back to the list, so be sure to  Cc linuxppc-dev  if your ]]
[[ reply is of general interest. To unsubscribe from linuxppc-dev, send ]]
[[ the message 'unsubscribe' to linuxppc-dev-request@lists.linuxppc.org ]]

Re: bootstrap stuffs

[Paul Mackerras]

David Edelsohn <dje@watson.ibm.com> wrote:

> 	"isync" is used to discard instruction pre-fetch and ensure that
> all previous instructions have occurred.  isync is not necessary for your
> situation either.
> 
> 	As Gabriel correctly explained, a "sync" instruction may be
> necessary before interrupts are enabled if some off-chip operation, like

Sync and isync turn out to be needed on some revs of some chips
(particularly the 601) around rfi and mtmsr instructions.

(In fact the 601 seems to have another weird bug - in the hash_page
routine in head.S, the location of the rfi instruction w.r.t. cache
lines appears to be critical on the 601; if it's wrong, you get a
machine check on the rfi instruction.  At one stage I found that with
2, 3, 6, or 7 nops added, it would work; with 0, 1, 4 or 5 nops added
it wouldn't. :-)

Paul.

[[ This message was sent via the linuxppc-dev mailing list. Replies are ]]
[[ not forced back to the list, so be sure to  Cc linuxppc-dev  if your ]]
[[ reply is of general interest. To unsubscribe from linuxppc-dev, send ]]
[[ the message 'unsubscribe' to linuxppc-dev-request@lists.linuxppc.org ]]

Re: bootstrap stuffs

[Gabriel Paubert]

On Tue, 16 Feb 1999, Paul Mackerras wrote:

> > 	As Gabriel correctly explained, a "sync" instruction may be
> > necessary before interrupts are enabled if some off-chip operation, like
> 
> Sync and isync turn out to be needed on some revs of some chips
> (particularly the 601) around rfi and mtmsr instructions.

Strange, I thought at least that rfi included a full context
synchronization for obvious reasons, so at least the isync should be
superfluous (for the sync it's different). 

Is there any documentation of these weird bus somewhere ?

I never saw anything like this (and the appendix and errata list for
the 601 from Motorola turns out to only contain the appendices).

Also could we think of implementing config options for people with buggy
processors, especially since the 601 is so weird in many respects
(CONFIG_PPC_NOT601 set to Y would remove a significant amount of
conditional code). I don't want my kernel spread with isync and sync while
I have perfectly valid processors (especially sync should be used
sparingly). 

> (In fact the 601 seems to have another weird bug - in the hash_page
> routine in head.S, the location of the rfi instruction w.r.t. cache
> lines appears to be critical on the 601; if it's wrong, you get a
> machine check on the rfi instruction.  At one stage I found that with
> 2, 3, 6, or 7 nops added, it would work; with 0, 1, 4 or 5 nops added
> it wouldn't. :-)

Weird indeed :-)

	Gabriel.


[[ This message was sent via the linuxppc-dev mailing list. Replies are ]]
[[ not forced back to the list, so be sure to  Cc linuxppc-dev  if your ]]
[[ reply is of general interest. To unsubscribe from linuxppc-dev, send ]]
[[ the message 'unsubscribe' to linuxppc-dev-request@lists.linuxppc.org ]]

Re: bootstrap stuffs

[Gabriel Paubert]

On Mon, 15 Feb 1999, David Edelsohn wrote:

> 	As Gabriel correctly explained, a "sync" instruction may be
> necessary before interrupts are enabled if some off-chip operation, like
> modifying an interrupt controller on the bus, was performed.  That is
> because of the bus operations which need to complete before interrupts can
> be enabled, not because of the enabling itself.

Actually I have come to the conclusion that the best way to handle this is
to add a sync instruction in the mask/disable interrupt routines (the ones
that access the interrupt controllers), so that all following __cli/__sti
only need to access the MSR (for enable/unmask you don't care to
synchronize of course).

For a driver I'm writing, I've suppressed all references to
read/write[bwl] when accessing MMIO space because they include an eieio. 
I'm explicitly adding iobarrier() instead.

I would also like to see versions of in/out which do not have an implied
eieio so that you can control when/if you want to synchronize(sync), force
ordering and or prevent gathering (eieio), or don't care (nothing).

However, I have question about sync: how do the bridges handle a sync
bus operation after a posted write ? If they don't flush the write
queues, it means that the only solution for the 8259 is to read again the
interrupt mask after having written it when masking an interrupt. 

	Gabriel.

P.S: actually I might try a lot of these modifications once my driver
works and I have put my VME boards in operation at the telescope. But for
now I have no time.


[[ This message was sent via the linuxppc-dev mailing list. Replies are ]]
[[ not forced back to the list, so be sure to  Cc linuxppc-dev  if your ]]
[[ reply is of general interest. To unsubscribe from linuxppc-dev, send ]]
[[ the message 'unsubscribe' to linuxppc-dev-request@lists.linuxppc.org ]]

Re: bootstrap stuffs

[Gabriel Paubert]

On Tue, 16 Feb 1999, Benjamin Herrenschmidt wrote:

> On Tue, Feb 16, 1999, Gabriel Paubert <paubert@iram.es> wrote:
> 
> >However, I have question about sync: how do the bridges handle a sync
> >bus operation after a posted write ? If they don't flush the write
> >queues, it means that the only solution for the 8259 is to read again the
> >interrupt mask after having written it when masking an interrupt. 
> 
> I beleive the sync is _not_ forwarded to the bridge, at least not on the
> MPC106. I fixed the "bogus interrupts" problem on pmac by adding, after
> the sync, a loop that reads the mask until it's actually the value that
> we wrote. I beleive a single read is enough to flush the bridges, but I
> am a little bit paranoid as far as Apple's interrupt controller is
> concerned so I added the loop.

Please forgive the noise. I just checked that the 603 and 750 do not
perform any bus operation when excuting sync or eieio. So the only
solution when disabling an interrupt is to read again the same register in
the controller (which is guaranteed to be ordered). OTOH 604 perform bus
operations on eieio and sync, but it still has potential problems with
posted writes IMHO. 

	Gabriel.


[[ This message was sent via the linuxppc-dev mailing list. Replies are ]]
[[ not forced back to the list, so be sure to  Cc linuxppc-dev  if your ]]
[[ reply is of general interest. To unsubscribe from linuxppc-dev, send ]]
[[ the message 'unsubscribe' to linuxppc-dev-request@lists.linuxppc.org ]]

Re: bootstrap stuffs

[Geert Uytterhoeven]

On Tue, 16 Feb 1999, Gabriel Paubert wrote:
> On Mon, 15 Feb 1999, David Edelsohn wrote:
> 
> > 	As Gabriel correctly explained, a "sync" instruction may be
> > necessary before interrupts are enabled if some off-chip operation, like
> > modifying an interrupt controller on the bus, was performed.  That is
> > because of the bus operations which need to complete before interrupts can
> > be enabled, not because of the enabling itself.
> 
> Actually I have come to the conclusion that the best way to handle this is
> to add a sync instruction in the mask/disable interrupt routines (the ones
> that access the interrupt controllers), so that all following __cli/__sti
> only need to access the MSR (for enable/unmask you don't care to
> synchronize of course).

Then do we need sync in the OpenPIC routines, too?

Greetings,

						Geert

--
Geert Uytterhoeven                     Geert.Uytterhoeven@cs.kuleuven.ac.be
Wavelets, Linux/{m68k~Amiga,PPC~CHRP}  http://www.cs.kuleuven.ac.be/~geert/
Department of Computer Science -- Katholieke Universiteit Leuven -- Belgium


[[ This message was sent via the linuxppc-dev mailing list. Replies are ]]
[[ not forced back to the list, so be sure to  Cc linuxppc-dev  if your ]]
[[ reply is of general interest. To unsubscribe from linuxppc-dev, send ]]
[[ the message 'unsubscribe' to linuxppc-dev-request@lists.linuxppc.org ]]

Re: bootstrap stuffs

[Gabriel Paubert]

On Tue, 16 Feb 1999, Geert Uytterhoeven wrote:

> > Actually I have come to the conclusion that the best way to handle this is
> > to add a sync instruction in the mask/disable interrupt routines (the ones
> > that access the interrupt controllers), so that all following __cli/__sti
> > only need to access the MSR (for enable/unmask you don't care to
> > synchronize of course).
> 
> Then do we need sync in the OpenPIC routines, too?

I think so. I would even say that you need to read again the OpenPIC
_Vector_Priority register when you mask an interrupt to make sure
that the interrupt has actually been masked. I don't see any problem 
with my Raven/Falcon right now because the OpenPIC is integrated with the
host bridge so there is actually no arbitration to access the register and
almost no write posting issue with a 603 which has at most 6 instructions
in flight (although the write buffers might delay the actual write for
quite some time). 

However, on the Hydra in the Longtrail, if the GG2 suppports write posting
(which is hopefully true), the delays between even the sync operation
on the bus and the actual update of the OpenPIC register might cause
strange side effects (I don't know how the GG2 reacts to a sync bus
operation on the 604). 

I think that some bridges are even allowed to reorder reads before posted
writes if the addresses do not match, but I should check the specs.
This is probably not possible with processors that do not run bus cycles
to signal the bridges on eieio and sync instructions (603 and 750).

So, if I'm not mistaken, a read from the same location is the only
operation that guarantees that the register has actually been updated, but
it's even slower than a sync :-(

	Gabriel.


[[ This message was sent via the linuxppc-dev mailing list. Replies are ]]
[[ not forced back to the list, so be sure to  Cc linuxppc-dev  if your ]]
[[ reply is of general interest. To unsubscribe from linuxppc-dev, send ]]
[[ the message 'unsubscribe' to linuxppc-dev-request@lists.linuxppc.org ]]

Re: bootstrap stuffs

[Benjamin Herrenschmidt]

On Tue, Feb 16, 1999, Gabriel Paubert <paubert@iram.es> wrote:

>So, if I'm not mistaken, a read from the same location is the only
>operation that guarantees that the register has actually been updated, but
>it's even slower than a sync :-(

On a side note, there is an interesting implementation of interrupts in
MkLinux mach kernel. Since they consider the controller as a really slow
device (and this seems to be confirmed by the bogus interrupt we had on
PowerMacs), they actually do it differently:

Mach maintains the current interrupt mask locally in RAM. When it
disables an interrupt, it does so only in the mask, the controller gets
updated only if this interrupt actually occurs. The enable code will
enable it in the mask only if it has been disabled by the interrupt.


-- 
           E-Mail: <mailto:bh40@calva.net>
BenH.      Web   : <http://calvaweb.calvacom.fr/bh40/>





[[ This message was sent via the linuxppc-dev mailing list. Replies are ]]
[[ not forced back to the list, so be sure to  Cc linuxppc-dev  if your ]]
[[ reply is of general interest. To unsubscribe from linuxppc-dev, send ]]
[[ the message 'unsubscribe' to linuxppc-dev-request@lists.linuxppc.org ]]

Re: bootstrap stuffs

[Cort Dougan]

The same sort of thing (optimistic disable) is in the real-time linux
system.  I want to add it when that comes in.

}Mach maintains the current interrupt mask locally in RAM. When it
}disables an interrupt, it does so only in the mask, the controller gets
}updated only if this interrupt actually occurs. The enable code will
}enable it in the mask only if it has been disabled by the interrupt.


[[ This message was sent via the linuxppc-dev mailing list. Replies are ]]
[[ not forced back to the list, so be sure to  Cc linuxppc-dev  if your ]]
[[ reply is of general interest. To unsubscribe from linuxppc-dev, send ]]
[[ the message 'unsubscribe' to linuxppc-dev-request@lists.linuxppc.org ]]