Re: memcpy and prefetch

[David Daney <ddaney@caviumnetworks.com>]

Michael Sundius wrote:
> David Daney wrote:
[...]
>> The Cavium OCTEON port overrides the default memcpy and does use 
>> prefetch.  It was recently merged (2.6.29-rc2).  Look at octeon-memcpy.S
[...]
> 
> thanks!!! that's really useful. I have a few questions tho:
> 
> 1) So you made this function explicitly for the Octeon.

I didn't personally write the code. But yes, as its name implies, it was 
created specifically for OCTEON.

> and that is 
> because you know the cache-line is 128 bytes long
> on the octeon? is that right?

That and we know exactly how prefetching works on this CPU *and* we know 
we can do unaligned accesses quickly.

> 
> 2) It seems as though you always prefectch the first cache line..  what 
> happens if the memcopy is less than 1 cache line long?
> wouldn't you risk prefetching beyond the end of the buffer?

It is a risk we were willing to take.  Cache lines are loaded with 
unneeded data all the time.

> 
> 3) why do you only do the "pref   0 offset(src)" and not a prefetch for 
> the destination?

I don't know.  But the interaction between the writeback buffers, the 
cache and RAM are somewhat complicated.  It may not be enough of a win 
to overcome the cost of the code that would determine when to do it.

> 
> 4) on line 244 you check to see if len is less than 128. while on the 
> other checks you check for (offset)+1
> why would you not do the prefetch if len was exactly 256 bytes? (or 128 
> in the case of line 196)?

We are always prefetching 256 bytes ahead of the current position.  If 
we prefetch beyound the end of the buffer it is truly wasting memory 
bandwidth, also if we prefetch to memory addresses where there is no 
physical memory, bad things happen.

David Daney