Re: memcpy and prefetch

[Chad Reese <kreese@caviumnetworks.com>]

> Michael Sundius wrote:
>> David Daney wrote:
>> 2) It seems as though you always prefectch the first cache line..  what 
>> happens if the memcopy is less than 1 cache line long?
>> wouldn't you risk prefetching beyond the end of the buffer?
> 
> It is a risk we were willing to take.  Cache lines are loaded with 
> unneeded data all the time.

If you assume that the memcpy is going to copy at least one byte, then
it is always safe to prefetch the first source address.

>> 3) why do you only do the "pref   0 offset(src)" and not a prefetch for 
>> the destination?
> 
> I don't know.  But the interaction between the writeback buffers, the 
> cache and RAM are somewhat complicated.  It may not be enough of a win 
> to overcome the cost of the code that would determine when to do it.

Octeon's write buffer merges all writes to single store transactions.
Since this store contains a full cache line, the L2 controller
automatically optimizes for it. With Octeon, the prepare to store
operations normally slow things down by creating needless bus traffic.
There are a few times where it is useful, but a generic memcpy isn't one
of them.

>> 4) on line 244 you check to see if len is less than 128. while on the 
>> other checks you check for (offset)+1
>> why would you not do the prefetch if len was exactly 256 bytes? (or 128 
>> in the case of line 196)?
> 
> We are always prefetching 256 bytes ahead of the current position.  If 
> we prefetch beyound the end of the buffer it is truly wasting memory 
> bandwidth, also if we prefetch to memory addresses where there is no 
> physical memory, bad things happen.

We prefetch 256 bytes ahead on every 128 bytes copied except for the
last two. Since we are fetching two lines ahead, the last two iterations
don't need prefetches. I think the code stops prefetching at the correct
time, but there is always the possibility that I messed up...

Chad