Syncing CPU caches from userland on MIPS

Syncing CPU caches from userland on MIPS

[Aurelien Jarno]

Hi all,

This question is not really kernel related, but still MIPS related, I
hope you don't mind.

Arnaud Patard and myself are trying to get qemu working on MIPS [1],
which includes translating TCG code (internal representation) into MIPS
instructions, that are then executed. Most of the code works, but we 
have some strange behaviors that seems related to CPU caches.

The code is written to a buffer, which is then executed. Before the
execution, the caches are synced using the cacheflush syscall:

| #include <sys/cachectl.h>
|  
| 
| static inline void flush_icache_range(unsigned long start, unsigned long stop)
| {
|     cacheflush ((void *)start, stop-start, ICACHE);
| }

It seems this is not enough, as sometimes, some executed code does not
correspond to the assembly dump of this memory region. This seems to be 
especially the case of memory regions that are written twice, due to
relocations:
1) a branch instruction is written with an offset of 0
2) the offset is patched
3) cacheflush is called

Sometimes the executed code correspond to the code written in 1), which
means the branch is skipped.

Does someone knows and/or has example code to correctly sync the CPU 
caches from userland on MIPS?

Thanks in advance,
Aurelien

[1] http://git.aurel32.net/?p=qemu.git;a=shortlog;h=refs/heads/tcg-mips

-- 
Aurelien Jarno	                        GPG: 1024D/F1BCDB73
aurelien@aurel32.net                 http://www.aurel32.net

Re: Syncing CPU caches from userland on MIPS

[David Daney]

Aurelien Jarno wrote:
> Hi all,
> 
> This question is not really kernel related, but still MIPS related, I
> hope you don't mind.
> 
> Arnaud Patard and myself are trying to get qemu working on MIPS [1],
> which includes translating TCG code (internal representation) into MIPS
> instructions, that are then executed. Most of the code works, but we 
> have some strange behaviors that seems related to CPU caches.
> 
> The code is written to a buffer, which is then executed. Before the
> execution, the caches are synced using the cacheflush syscall:
> 
> | #include <sys/cachectl.h>
> |  
> | 
> | static inline void flush_icache_range(unsigned long start, unsigned long stop)
> | {
> |     cacheflush ((void *)start, stop-start, ICACHE);
> | }
> 
> It seems this is not enough, as sometimes, some executed code does not
> correspond to the assembly dump of this memory region. This seems to be 
> especially the case of memory regions that are written twice, due to
> relocations:
> 1) a branch instruction is written with an offset of 0
> 2) the offset is patched

Try inserting an 'asm volatile ("sync" ::: "memory");' here.  If that 
fixes things, then we can assume that your cacheflush system call is 
buggy, and would need to add a sync.

> 3) cacheflush is called
> 
> Sometimes the executed code correspond to the code written in 1), which
> means the branch is skipped.
> 
> Does someone knows and/or has example code to correctly sync the CPU 
> caches from userland on MIPS?
> 

http://gcc.gnu.org/viewcvs/trunk/libffi/src/mips/ffi.c

Look at ffi_prep_closure_loc() around line 721.


We also do a similar thing in the kernel in flush_cache_sigtramp(), but 
that is not really userland.


David Daney

Re: Syncing CPU caches from userland on MIPS

[Aurelien Jarno]

On Tue, Nov 24, 2009 at 01:04:55PM -0800, David Daney wrote:
> Aurelien Jarno wrote:
>> Hi all,
>>
>> This question is not really kernel related, but still MIPS related, I
>> hope you don't mind.
>>
>> Arnaud Patard and myself are trying to get qemu working on MIPS [1],
>> which includes translating TCG code (internal representation) into MIPS
>> instructions, that are then executed. Most of the code works, but we  
>> have some strange behaviors that seems related to CPU caches.
>>
>> The code is written to a buffer, which is then executed. Before the
>> execution, the caches are synced using the cacheflush syscall:
>>
>> | #include <sys/cachectl.h>
>> |  | | static inline void flush_icache_range(unsigned long start, 
>> unsigned long stop)
>> | {
>> |     cacheflush ((void *)start, stop-start, ICACHE);
>> | }
>>
>> It seems this is not enough, as sometimes, some executed code does not
>> correspond to the assembly dump of this memory region. This seems to be 
>> especially the case of memory regions that are written twice, due to
>> relocations:
>> 1) a branch instruction is written with an offset of 0
>> 2) the offset is patched
>
> Try inserting an 'asm volatile ("sync" ::: "memory");' here.  If that  
> fixes things, then we can assume that your cacheflush system call is  
> buggy, and would need to add a sync.
>

That doesn't help, it still crashes at the same location.

Aurelien

-- 
Aurelien Jarno	                        GPG: 1024D/F1BCDB73
aurelien@aurel32.net                 http://www.aurel32.net

Re: Syncing CPU caches from userland on MIPS

[Florian Lohoff]

[-- Attachment #1: Type: text/plain, Size: 2067 bytes --]

On Tue, Nov 24, 2009 at 07:28:41PM +0100, Aurelien Jarno wrote:
> Hi all,
> 
> This question is not really kernel related, but still MIPS related, I
> hope you don't mind.
> 
> Arnaud Patard and myself are trying to get qemu working on MIPS [1],
> which includes translating TCG code (internal representation) into MIPS
> instructions, that are then executed. Most of the code works, but we 
> have some strange behaviors that seems related to CPU caches.
> 
> The code is written to a buffer, which is then executed. Before the
> execution, the caches are synced using the cacheflush syscall:
> 
> | #include <sys/cachectl.h>
> |  
> | 
> | static inline void flush_icache_range(unsigned long start, unsigned long stop)
> | {
> |     cacheflush ((void *)start, stop-start, ICACHE);
> | }

Would this only evict stuff from the ICACHE? When trying to execute
a just written buffer and with a writeback DCACHE you would need to 
explicitly writeback the DCACHE to memory and invalidate the ICACHE.

> It seems this is not enough, as sometimes, some executed code does not
> correspond to the assembly dump of this memory region. This seems to be 
> especially the case of memory regions that are written twice, due to
> relocations:
> 1) a branch instruction is written with an offset of 0
> 2) the offset is patched
> 3) cacheflush is called
> 
> Sometimes the executed code correspond to the code written in 1), which
> means the branch is skipped.

Which proves my theory - as long as you have cache pressure you will happily
writeback the contents to memory before trying to execute (you invalidate
the ICACHE above) - In case you DCACHE does not suffer from pressure
the contents will not been written back and you'll execute stale code.

Flo
-- 
Florian Lohoff                                         flo@rfc822.org
"Es ist ein grobes Missverständnis und eine Fehlwahrnehmung, dem Staat
im Internet Zensur- und Überwachungsabsichten zu unterstellen."
- - Bundesminister Dr. Wolfgang Schäuble -- 10. Juli in Berlin 

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: Syncing CPU caches from userland on MIPS

[Arnaud Patard]

Florian Lohoff <flo@rfc822.org> writes:
Hi,

> On Tue, Nov 24, 2009 at 07:28:41PM +0100, Aurelien Jarno wrote:
>> Hi all,
>> 
>> This question is not really kernel related, but still MIPS related, I
>> hope you don't mind.
>> 
>> Arnaud Patard and myself are trying to get qemu working on MIPS [1],
>> which includes translating TCG code (internal representation) into MIPS
>> instructions, that are then executed. Most of the code works, but we 
>> have some strange behaviors that seems related to CPU caches.
>> 
>> The code is written to a buffer, which is then executed. Before the
>> execution, the caches are synced using the cacheflush syscall:
>> 
>> | #include <sys/cachectl.h>
>> |  
>> | 
>> | static inline void flush_icache_range(unsigned long start, unsigned long stop)
>> | {
>> |     cacheflush ((void *)start, stop-start, ICACHE);
>> | }
>
> Would this only evict stuff from the ICACHE? When trying to execute
> a just written buffer and with a writeback DCACHE you would need to 
> explicitly writeback the DCACHE to memory and invalidate the ICACHE.

we already though about using BCACHE instead of ICACHE only but it
didn't make any difference. the bug is still there.

Arnaud

Re: Syncing CPU caches from userland on MIPS

[Florian Lohoff]

[-- Attachment #1: Type: text/plain, Size: 868 bytes --]

On Wed, Nov 25, 2009 at 03:39:01PM +0100, Arnaud Patard wrote:
> > Would this only evict stuff from the ICACHE? When trying to execute
> > a just written buffer and with a writeback DCACHE you would need to 
> > explicitly writeback the DCACHE to memory and invalidate the ICACHE.
> 
> we already though about using BCACHE instead of ICACHE only but it
> didn't make any difference. the bug is still there.

My understanding is you need both ...

FLUSH/WRITEBACK the dcache and INVALIDATE the icache - the icache needs
to load the data which is in the dcache via memory.

Flo
-- 
Florian Lohoff                                         flo@rfc822.org
"Es ist ein grobes Missverständnis und eine Fehlwahrnehmung, dem Staat
im Internet Zensur- und Überwachungsabsichten zu unterstellen."
- - Bundesminister Dr. Wolfgang Schäuble -- 10. Juli in Berlin 

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: Syncing CPU caches from userland on MIPS

[Arnaud Patard]

Florian Lohoff <flo@rfc822.org> writes:

> On Wed, Nov 25, 2009 at 03:39:01PM +0100, Arnaud Patard wrote:
>> > Would this only evict stuff from the ICACHE? When trying to execute
>> > a just written buffer and with a writeback DCACHE you would need to 
>> > explicitly writeback the DCACHE to memory and invalidate the ICACHE.
>> 
>> we already though about using BCACHE instead of ICACHE only but it
>> didn't make any difference. the bug is still there.
>
> My understanding is you need both ...
>
> FLUSH/WRITEBACK the dcache and INVALIDATE the icache - the icache needs
> to load the data which is in the dcache via memory.

I undertstood that using BCACHE would be better but still, it doesn't
solve our issue. Can we please go ahead ? :)

Arnaud

Re: Syncing CPU caches from userland on MIPS

[peter fuerst]

Hi Arnaud,

this is an age-old "feature", still resistent to all the patches, which
tried to change it since ;-)

from MIPS git-head:

SYSCALL_DEFINE3(cacheflush, unsigned long, addr, unsigned long, bytes,
        unsigned int, cache)
{
	if (bytes == 0)
		return 0;
	if (!access_ok(VERIFY_WRITE, (void __user *) addr, bytes))
		return -EFAULT;

	flush_icache_range(addr, addr + bytes);

	return 0;
}

DCACHE as well as BCACHE are silently taken as ICACHE (You have to fix
your local copy of the kernel source, and you are not alone with this 
problem: anyone who wants to run the Xserver on IP28 has to do)


kind regards

peter


On Wed, 25 Nov 2009, Arnaud Patard wrote:

> Date: Wed, 25 Nov 2009 15:48:16 +0100
> From: Arnaud Patard <arnaud.patard@rtp-net.org>
> To: Florian Lohoff <flo@rfc822.org>
> Cc: Aurelien Jarno <aurelien@aurel32.net>, linux-mips@linux-mips.org
> Subject: Re: Syncing CPU caches from userland on MIPS
> 
> Florian Lohoff <flo@rfc822.org> writes:
> 
> > On Wed, Nov 25, 2009 at 03:39:01PM +0100, Arnaud Patard wrote:
> >> > Would this only evict stuff from the ICACHE? When trying to execute
> >> > a just written buffer and with a writeback DCACHE you would need to 
> >> > explicitly writeback the DCACHE to memory and invalidate the ICACHE.
> >> 
> >> we already though about using BCACHE instead of ICACHE only but it
> >> didn't make any difference. the bug is still there.
> >
> > My understanding is you need both ...
> >
> > FLUSH/WRITEBACK the dcache and INVALIDATE the icache - the icache needs
> > to load the data which is in the dcache via memory.
> 
> I undertstood that using BCACHE would be better but still, it doesn't
> solve our issue. Can we please go ahead ? :)
> 
> Arnaud
> 
> 

Re: Syncing CPU caches from userland on MIPS

[Ralf Baechle]

On Wed, Nov 25, 2009 at 03:39:01PM +0100, Arnaud Patard wrote:

> > Would this only evict stuff from the ICACHE? When trying to execute
> > a just written buffer and with a writeback DCACHE you would need to 
> > explicitly writeback the DCACHE to memory and invalidate the ICACHE.
> 
> we already though about using BCACHE instead of ICACHE only but it
> didn't make any difference. the bug is still there.

That argument is ignored; the kernel will always do whatever it takes to
get the I-cache consistent.

  Ralf

Re: Syncing CPU caches from userland on MIPS

[Ralf Baechle]

On Wed, Nov 25, 2009 at 03:01:05PM +0100, Florian Lohoff wrote:

> > | static inline void flush_icache_range(unsigned long start, unsigned long stop)
> > | {
> > |     cacheflush ((void *)start, stop-start, ICACHE);
> > | }
> 
> Would this only evict stuff from the ICACHE? When trying to execute
> a just written buffer and with a writeback DCACHE you would need to 
> explicitly writeback the DCACHE to memory and invalidate the ICACHE.

No; ICACHE really means the kernel will do whatever it takes to make the
I-cache coherent with the D-cache.  For most processors this requires
writing back the D-cache to S-cache or if no S-cache present, memory
then invalidating the I-cache.

> > It seems this is not enough, as sometimes, some executed code does not
> > correspond to the assembly dump of this memory region. This seems to be 
> > especially the case of memory regions that are written twice, due to
> > relocations:
> > 1) a branch instruction is written with an offset of 0
> > 2) the offset is patched
> > 3) cacheflush is called
> > 
> > Sometimes the executed code correspond to the code written in 1), which
> > means the branch is skipped.
> 
> Which proves my theory - as long as you have cache pressure you will happily
> writeback the contents to memory before trying to execute (you invalidate
> the ICACHE above) - In case you DCACHE does not suffer from pressure
> the contents will not been written back and you'll execute stale code.

You could - on a uni-processor system - do something like this to invalidate
the cache:

int array[32768 / sizeof(int)];	/* size of D-cache on SB1250  */

blow_away_d(void)
{
	memset(array, 0, sizeof(array) / sizeof(int) - 2);
	array[sizeof(int) - 2] = 0x03E00008;	/* jr $ra	*/
	array[sizeof(int) - 1] = 0x00000000;	/* nop		*/
}

blow_away_i(void)
{
	void (*fn)(void) = (void *) array;

	blow_away_d();
	fn();
}

blow_away_setup(void)
{
	blow_away_d();
	cacheflush(array, sizeof(array), BCACHE);
}

The memset and array assignment operations result in victim writebacks
that is basically an D-cache writeback.  Later on executing the code in
the array will result in the entire I-cache getting refilled so anything
of your qemu code array that might happen to live in the I-cache will be
discarded out of the cache.

  Ralf