u64 fun and PCI DMA not working for things behind bridges?

u64 fun and PCI DMA not working for things behind bridges?

[Daniel Berlin]

In getting firewire working, and trying to get sbp-2 working, on
pismo, i noticed two things:
u64 comparisons seem broken in some cases, and PCI DMA doesn't work
all the time for the firewire device.

First, the u64 stuff
(these are %llx printk's, they have 0's to the left, verified by
looking at the actual memory, since i don't trust u64 on ppc right
now)

Feb 27 18:39:47 dynamic-addr-83-177 kernel: as->start = 0, addr = b2f2040
Feb 27 18:39:47 dynamic-addr-83-177 kernel: as->end = 0, as->end > addr == 0
Feb 27 18:39:47 dynamic-addr-83-177 kernel: as->end = fffffffc, as->end > addr == 0


note that as->end > addr is false.

It should be true.
It's *clearly* greater.

If i trunc them to u32's, it is greater, and besides the crap off to
the left going away (which wasn't relevant to the number anyway),
nothing is different.

The only structures that are troublesome, and don't work without
explicit fiddling, in sbp-2, is the login structure, which contains a
u64 for the password. It seems it somehow screws the fields next to
it, which happen to contain the address to put the login response, or
causes it to read/write those fields wrong.

The password is normally just set to 0.

I think it may be in the wrong byte or bit order, which would explain the
compares failing, and the fact that if I cpu_to_le32 the structure
when copying it from host memory, it gets the right addresses for the
login response, but the rest of it is wrong.

Reversing them before, and letting the controller read them through
physical dma doesn't work.


Oh, and nothing else needs to, or works properly, if byteswapped. We
shouldn't need to byteswap the login "orb" (sbp-2 term),  there's
something screwing us. And i bet it's my friend, u64.


I can't compile the drivers without optimization, or else i get
references to functions that don't exist in the kernel, for some
reason.

Simple u64 tests elsewhere seem to work, but i can definitively show
they don't work here (i've got piles of logs of the raw data of the
stuff it's doing those comparisons on).


Also, as i mentioned, physical dma doesn't seem to work right all the
time, and i'm pretty sure it's not the drivers.

The firewire device is behind the same bridge the GMAC is behind. Is
there something special I need to do to make physical dma work all the
time when the device you are dma'ing to/from is behind a bridge?

It could be a hardware implementation problem in the uni-north FW, of
course. I'm just trying to see if anyone has ever noticed anything.


Right now i have to resort to having the SBP-2 driver emulate physical
dma reads/writes by telling the firewire subsystem driver to redirect
all reads/writes into host memory through a routine that simply does
memcpys.
I also tell the controller it's not allowed to allow physical dma
accesses to be performed (there are filter registers that let you
specify what addresses you'll allow physical dma access to, and what
you want to go through the normal block read/write/quadlet read/write
process) , and that the other side must do it through normal block
reads/write packets.

This works, but it's a little slow, of course.


--Dan

** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

Re: u64 fun and PCI DMA not working for things behind bridges?

[Michael Schmitz]

> I can't compile the drivers without optimization, or else i get
> references to functions that don't exist in the kernel, for some
> reason.

What exact functions don't exist?

	Michael


** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

Re: u64 fun and PCI DMA not working for things behind bridges?

[Benjamin Herrenschmidt]

>
>The firewire device is behind the same bridge the GMAC is behind. Is
>there something special I need to do to make physical dma work all the
>time when the device you are dma'ing to/from is behind a bridge?
>
>It could be a hardware implementation problem in the uni-north FW, of
>course. I'm just trying to see if anyone has ever noticed anything.
>
>
>Right now i have to resort to having the SBP-2 driver emulate physical
>dma reads/writes by telling the firewire subsystem driver to redirect
>all reads/writes into host memory through a routine that simply does
>memcpys.
>I also tell the controller it's not allowed to allow physical dma
>accesses to be performed (there are filter registers that let you
>specify what addresses you'll allow physical dma access to, and what
>you want to go through the normal block read/write/quadlet read/write
>process) , and that the other side must do it through normal block
>reads/write packets.
>
>This works, but it's a little slow, of course.

It might be possible that we have an apple bug here... Could you try
doing as if the device was not cache coherent ?

Basically, invalidate the cache for a data block that will be filled by
the device, and flush the cache for a data block that will be read by the
device. You can do the first one with dcbi, the second with dcbf, both
loops followed by a "sync" instruction to make sure the ache operations
are completed.

Ben


** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

Re: u64 fun and PCI DMA not working for things behind bridges?

[Benjamin Herrenschmidt]

>> I can't compile the drivers without optimization, or else i get
>> references to functions that don't exist in the kernel, for some
>> reason.
>
>What exact functions don't exist?

If you compile with -O0, you also disable inlining, which cause tons of
link errors for all inline functions. The lowest optimisation level you
can use is -O1

Ben.


** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

Re: u64 fun and PCI DMA not working for things behind bridges?

[Michael Schmitz]

> >What exact functions don't exist?
>
> If you compile with -O0, you also disable inlining, which cause tons of
> link errors for all inline functions. The lowest optimisation level you
> can use is -O1

And I thought it might be as simple as a funny varargs call again :-)

	Michael


** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

Re: u64 fun and PCI DMA not working for things behind bridges?

[Daniel Berlin]

Michael Schmitz <schmitz@opal.biophys.uni-duesseldorf.de> writes:

> > I can't compile the drivers without optimization, or else i get
> > references to functions that don't exist in the kernel, for some
> > reason.
>
> What exact functions don't exist?
>
> 	Michael

Any extern inline functions.

--Dan

** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

Re: u64 fun and PCI DMA not working for things behind bridges?

[Daniel Berlin]

Benjamin Herrenschmidt <benh@kernel.crashing.org> writes:

> >
> >The firewire device is behind the same bridge the GMAC is behind. Is
> >there something special I need to do to make physical dma work all the
> >time when the device you are dma'ing to/from is behind a bridge?
> >
> >It could be a hardware implementation problem in the uni-north FW, of
> >course. I'm just trying to see if anyone has ever noticed anything.
> >
> >
> >Right now i have to resort to having the SBP-2 driver emulate physical
> >dma reads/writes by telling the firewire subsystem driver to redirect
> >all reads/writes into host memory through a routine that simply does
> >memcpys.
> >I also tell the controller it's not allowed to allow physical dma
> >accesses to be performed (there are filter registers that let you
> >specify what addresses you'll allow physical dma access to, and what
> >you want to go through the normal block read/write/quadlet read/write
> >process) , and that the other side must do it through normal block
> >reads/write packets.
> >
> >This works, but it's a little slow, of course.
>
> It might be possible that we have an apple bug here... Could you try
> doing as if the device was not cache coherent ?

I'm on it.

>
> Basically, invalidate the cache for a data block that will be filled by
> the device, and flush the cache for a data block that will be read by the
> device. You can do the first one with dcbi, the second with dcbf, both
> loops followed by a "sync" instruction to make sure the ache operations
> are completed.
>

This would, of course, be much easier if i had something snooping the
1394 bus, so i could see if it's even really *trying* to do the DMA.

The thing that makes me think it is is the fact that i get random
kernel crashes right after it should be doing the DMA (IE it's
writing the wrong memory).

Let me give your idea a try.

> Ben

** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

Re: u64 fun and PCI DMA not working for things behind bridges?

[Geert Uytterhoeven]

On 1 Mar 2001, Daniel Berlin wrote:
> Michael Schmitz <schmitz@opal.biophys.uni-duesseldorf.de> writes:
> > > I can't compile the drivers without optimization, or else i get
> > > references to functions that don't exist in the kernel, for some
> > > reason.
> >
> > What exact functions don't exist?
>
> Any extern inline functions.

`extern inline' is deprecated, in favor of `static inline'. During the last few
weeks, many of them were converted, albeit not in include/asm-ppc/.

I suppose this will solve the link problem with -O0 as well, since the compiler
will no longer think non-inlined functions are avilable externally?

Gr{oetje,eeting}s,

						Geert

--
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- geert@linux-m68k.org

In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
							    -- Linus Torvalds


** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/