Reading an entire cacheline

Reading an entire cacheline

[Kim, Jong-Sung]

Hi,

Is there any way to read an entire cacheline from the MIPS5Kf? Four
sequential cache instructions do similarly, but sometimes the tags from a
same cacheline differ. How can I read a consistent cacheline safely?

Thanks.

Reading an entire cacheline

[Kim, Jong-Sung]

Hi,

Is there any way to read an entire cacheline from the MIPS5Kf? Four
sequential cache instructions do similarly, but sometimes the tags from a
same cacheline differ. How can I read a consistent cacheline safely?

Thanks.

RE: Reading an entire cacheline

[Kim, Jong-Sung]

Hi all,

Was my question too brief? I'm using MIPS5Kf core included in a Broadcom
implemented processor and it has a 32kB d-cache and a same sized i-cache.
Cache configuration is 2-way, 32-byte cacheline for both d- and i-cache
without L2 cache. Of course, MIPS-port Linux as the kernel for it. And, a
onboard block device has problem in its operation. I don't think it's a
kernel problem. So I wanna inspect the contents of cache RAMs at the device
driver's context. I've added some lines of codes look like follow:

	__asm__ __volatile__(
		".set	noreorder\n\t"
		".set	mips3\n\t"
		"cache	5, 0x00(%3)\n\t"	// load d-cache tag
		".set	mips0\n\t"		// by index
		".set	mips32\n\t"
		"mfc0	%0, $28, 0\n\t"
		"mfc0	%1, $28, 1\n\t"
		"mfc0	%2, $29, 1\n\t"
		.set	mips0\n\t"
		.set	reorder"
		: "=r" (tag), "=r" (data_lo), "=r" (data_hi)
		: "r" (0x80000000 | (way << 14) | (index << 5) | (word <<
3))
	);

This is executed for every index, every way, and every word. The thing aches
my head is the tags from a single cacheline differs. (very often for
d-cache) So I modified the code to reread the cacheline from the first word
when the tag was modified before reading all four words. Then the code never
return. I couldn't find any related information from manual.

Please give me a hint for reading whole cacheline safely.


-----Original Message-----
From: linux-mips-bounce@linux-mips.org
[mailto:linux-mips-bounce@linux-mips.org] On Behalf Of Kim, Jong-Sung
Sent: Tuesday, April 18, 2006 9:37 AM
To: linux-mips@linux-mips.org
Subject: Reading an entire cacheline

Hi,

Is there any way to read an entire cacheline from the MIPS5Kf? Four
sequential cache instructions do similarly, but sometimes the tags from a
same cacheline differ. How can I read a consistent cacheline safely?

Thanks.

RE: Reading an entire cacheline

[Kim, Jong-Sung]

Hi all,

Was my question too brief? I'm using MIPS5Kf core included in a Broadcom
implemented processor and it has a 32kB d-cache and a same sized i-cache.
Cache configuration is 2-way, 32-byte cacheline for both d- and i-cache
without L2 cache. Of course, MIPS-port Linux as the kernel for it. And, a
onboard block device has problem in its operation. I don't think it's a
kernel problem. So I wanna inspect the contents of cache RAMs at the device
driver's context. I've added some lines of codes look like follow:

	__asm__ __volatile__(
		".set	noreorder\n\t"
		".set	mips3\n\t"
		"cache	5, 0x00(%3)\n\t"	// load d-cache tag
		".set	mips0\n\t"		// by index
		".set	mips32\n\t"
		"mfc0	%0, $28, 0\n\t"
		"mfc0	%1, $28, 1\n\t"
		"mfc0	%2, $29, 1\n\t"
		.set	mips0\n\t"
		.set	reorder"
		: "=r" (tag), "=r" (data_lo), "=r" (data_hi)
		: "r" (0x80000000 | (way << 14) | (index << 5) | (word <<
3))
	);

This is executed for every index, every way, and every word. The thing aches
my head is the tags from a single cacheline differs. (very often for
d-cache) So I modified the code to reread the cacheline from the first word
when the tag was modified before reading all four words. Then the code never
return. I couldn't find any related information from manual.

Please give me a hint for reading whole cacheline safely.


-----Original Message-----
From: linux-mips-bounce@linux-mips.org
[mailto:linux-mips-bounce@linux-mips.org] On Behalf Of Kim, Jong-Sung
Sent: Tuesday, April 18, 2006 9:37 AM
To: linux-mips@linux-mips.org
Subject: Reading an entire cacheline

Hi,

Is there any way to read an entire cacheline from the MIPS5Kf? Four
sequential cache instructions do similarly, but sometimes the tags from a
same cacheline differ. How can I read a consistent cacheline safely?

Thanks.

Re: Reading an entire cacheline

[Kevin D. Kissell]

Yes, your question was too brief the first time around - I couldn't
tell what you were really asking.  I'm still not 100% sure of what
you're doing, though.   I don't want to sound insulting, but you're
disabling interrupts during this operation, right?  And if you're
copying the values into variables in memory, you're doing nothing
but non-cached references, right?  When you see the tags changing
on you, which bits are changing, and what are they defined to mean?

            Regards,

            Kevin K.

----- Original Message ----- 
From: "Kim, Jong-Sung" <jsungkim@lge.com>
To: <linux-mips@linux-mips.org>
Sent: Monday, April 24, 2006 9:23 AM
Subject: RE: Reading an entire cacheline


> Hi all,
> 
> Was my question too brief? I'm using MIPS5Kf core included in a Broadcom
> implemented processor and it has a 32kB d-cache and a same sized i-cache.
> Cache configuration is 2-way, 32-byte cacheline for both d- and i-cache
> without L2 cache. Of course, MIPS-port Linux as the kernel for it. And, a
> onboard block device has problem in its operation. I don't think it's a
> kernel problem. So I wanna inspect the contents of cache RAMs at the device
> driver's context. I've added some lines of codes look like follow:
> 
> __asm__ __volatile__(
> ".set noreorder\n\t"
> ".set mips3\n\t"
> "cache 5, 0x00(%3)\n\t" // load d-cache tag
> ".set mips0\n\t" // by index
> ".set mips32\n\t"
> "mfc0 %0, $28, 0\n\t"
> "mfc0 %1, $28, 1\n\t"
> "mfc0 %2, $29, 1\n\t"
> .set mips0\n\t"
> .set reorder"
> : "=r" (tag), "=r" (data_lo), "=r" (data_hi)
> : "r" (0x80000000 | (way << 14) | (index << 5) | (word <<
> 3))
> );
> 
> This is executed for every index, every way, and every word. The thing aches
> my head is the tags from a single cacheline differs. (very often for
> d-cache) So I modified the code to reread the cacheline from the first word
> when the tag was modified before reading all four words. Then the code never
> return. I couldn't find any related information from manual.
> 
> Please give me a hint for reading whole cacheline safely.
> 
> 
> -----Original Message-----
> From: linux-mips-bounce@linux-mips.org
> [mailto:linux-mips-bounce@linux-mips.org] On Behalf Of Kim, Jong-Sung
> Sent: Tuesday, April 18, 2006 9:37 AM
> To: linux-mips@linux-mips.org
> Subject: Reading an entire cacheline
> 
> Hi,
> 
> Is there any way to read an entire cacheline from the MIPS5Kf? Four
> sequential cache instructions do similarly, but sometimes the tags from a
> same cacheline differ. How can I read a consistent cacheline safely?
> 
> Thanks.
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 

Re: Reading an entire cacheline

[Kevin D. Kissell]

Yes, your question was too brief the first time around - I couldn't
tell what you were really asking.  I'm still not 100% sure of what
you're doing, though.   I don't want to sound insulting, but you're
disabling interrupts during this operation, right?  And if you're
copying the values into variables in memory, you're doing nothing
but non-cached references, right?  When you see the tags changing
on you, which bits are changing, and what are they defined to mean?

            Regards,

            Kevin K.

----- Original Message ----- 
From: "Kim, Jong-Sung" <jsungkim@lge.com>
To: <linux-mips@linux-mips.org>
Sent: Monday, April 24, 2006 9:23 AM
Subject: RE: Reading an entire cacheline


> Hi all,
> 
> Was my question too brief? I'm using MIPS5Kf core included in a Broadcom
> implemented processor and it has a 32kB d-cache and a same sized i-cache.
> Cache configuration is 2-way, 32-byte cacheline for both d- and i-cache
> without L2 cache. Of course, MIPS-port Linux as the kernel for it. And, a
> onboard block device has problem in its operation. I don't think it's a
> kernel problem. So I wanna inspect the contents of cache RAMs at the device
> driver's context. I've added some lines of codes look like follow:
> 
> __asm__ __volatile__(
> ".set noreorder\n\t"
> ".set mips3\n\t"
> "cache 5, 0x00(%3)\n\t" // load d-cache tag
> ".set mips0\n\t" // by index
> ".set mips32\n\t"
> "mfc0 %0, $28, 0\n\t"
> "mfc0 %1, $28, 1\n\t"
> "mfc0 %2, $29, 1\n\t"
> .set mips0\n\t"
> .set reorder"
> : "=r" (tag), "=r" (data_lo), "=r" (data_hi)
> : "r" (0x80000000 | (way << 14) | (index << 5) | (word <<
> 3))
> );
> 
> This is executed for every index, every way, and every word. The thing aches
> my head is the tags from a single cacheline differs. (very often for
> d-cache) So I modified the code to reread the cacheline from the first word
> when the tag was modified before reading all four words. Then the code never
> return. I couldn't find any related information from manual.
> 
> Please give me a hint for reading whole cacheline safely.
> 
> 
> -----Original Message-----
> From: linux-mips-bounce@linux-mips.org
> [mailto:linux-mips-bounce@linux-mips.org] On Behalf Of Kim, Jong-Sung
> Sent: Tuesday, April 18, 2006 9:37 AM
> To: linux-mips@linux-mips.org
> Subject: Reading an entire cacheline
> 
> Hi,
> 
> Is there any way to read an entire cacheline from the MIPS5Kf? Four
> sequential cache instructions do similarly, but sometimes the tags from a
> same cacheline differ. How can I read a consistent cacheline safely?
> 
> Thanks.
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 

RE: Reading an entire cacheline

[Kim, Jong-Sung]

Dear Mr. Kissell and the others,

I apologize for my rude question if you felt like. I was first in this
mailing list and I didn't think I had to describe the background of my
problem such in depth. I have an application stored in an onboard block
device and it sometimes makes bus error or segmentation fault during
do_page_fault().
When the application is stored in and executed from an nfs device or an
ramfs or any other device, it works without any problem. So I think the
problem comes from the device driver or something below.
In some experiments, the cache invalidation in a point reduces the frequency
of abnormal terminations. (I don't know it's a walk-around or just reduction
yet) When the application was copied from the device onto the other (ram or
network) device, it works well. So I guess the problem is related to the
cache and not from just d-cache incoherency. I targeted the consistency
between d- and i-cache as first. If you have any idea, please let me know.
For the device driver for the block device is very large and not my work,
I'm still in the dark. ^^;

Thank you for your hints about the interrupt disabling and the variable
cachability. They could be very helpful for me. For the tag, I meant the
physical address field of the valid tag. Thank you again.


Regards,
JS.


-----Original Message-----
From: Kevin D. Kissell [mailto:kevink@mips.com] 
Sent: Monday, April 24, 2006 4:41 PM
To: Kim, Jong-Sung; linux-mips@linux-mips.org
Subject: Re: Reading an entire cacheline

Yes, your question was too brief the first time around - I couldn't
tell what you were really asking.  I'm still not 100% sure of what
you're doing, though.   I don't want to sound insulting, but you're
disabling interrupts during this operation, right?  And if you're
copying the values into variables in memory, you're doing nothing
but non-cached references, right?  When you see the tags changing
on you, which bits are changing, and what are they defined to mean?

            Regards,

            Kevin K.

----- Original Message ----- 
From: "Kim, Jong-Sung" <jsungkim@lge.com>
To: <linux-mips@linux-mips.org>
Sent: Monday, April 24, 2006 9:23 AM
Subject: RE: Reading an entire cacheline


> Hi all,
> 
> Was my question too brief? I'm using MIPS5Kf core included in a Broadcom
> implemented processor and it has a 32kB d-cache and a same sized i-cache.
> Cache configuration is 2-way, 32-byte cacheline for both d- and i-cache
> without L2 cache. Of course, MIPS-port Linux as the kernel for it. And, a
> onboard block device has problem in its operation. I don't think it's a
> kernel problem. So I wanna inspect the contents of cache RAMs at the
device
> driver's context. I've added some lines of codes look like follow:
> 
> __asm__ __volatile__(
> ".set noreorder\n\t"
> ".set mips3\n\t"
> "cache 5, 0x00(%3)\n\t" // load d-cache tag
> ".set mips0\n\t" // by index
> ".set mips32\n\t"
> "mfc0 %0, $28, 0\n\t"
> "mfc0 %1, $28, 1\n\t"
> "mfc0 %2, $29, 1\n\t"
> .set mips0\n\t"
> .set reorder"
> : "=r" (tag), "=r" (data_lo), "=r" (data_hi)
> : "r" (0x80000000 | (way << 14) | (index << 5) | (word <<
> 3))
> );
> 
> This is executed for every index, every way, and every word. The thing
aches
> my head is the tags from a single cacheline differs. (very often for
> d-cache) So I modified the code to reread the cacheline from the first
word
> when the tag was modified before reading all four words. Then the code
never
> return. I couldn't find any related information from manual.
> 
> Please give me a hint for reading whole cacheline safely.
> 
> 
> -----Original Message-----
> From: linux-mips-bounce@linux-mips.org
> [mailto:linux-mips-bounce@linux-mips.org] On Behalf Of Kim, Jong-Sung
> Sent: Tuesday, April 18, 2006 9:37 AM
> To: linux-mips@linux-mips.org
> Subject: Reading an entire cacheline
> 
> Hi,
> 
> Is there any way to read an entire cacheline from the MIPS5Kf? Four
> sequential cache instructions do similarly, but sometimes the tags from a
> same cacheline differ. How can I read a consistent cacheline safely?
> 
> Thanks.
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 

RE: Reading an entire cacheline

[Kim, Jong-Sung]

Dear Mr. Kissell and the others,

I apologize for my rude question if you felt like. I was first in this
mailing list and I didn't think I had to describe the background of my
problem such in depth. I have an application stored in an onboard block
device and it sometimes makes bus error or segmentation fault during
do_page_fault().
When the application is stored in and executed from an nfs device or an
ramfs or any other device, it works without any problem. So I think the
problem comes from the device driver or something below.
In some experiments, the cache invalidation in a point reduces the frequency
of abnormal terminations. (I don't know it's a walk-around or just reduction
yet) When the application was copied from the device onto the other (ram or
network) device, it works well. So I guess the problem is related to the
cache and not from just d-cache incoherency. I targeted the consistency
between d- and i-cache as first. If you have any idea, please let me know.
For the device driver for the block device is very large and not my work,
I'm still in the dark. ^^;

Thank you for your hints about the interrupt disabling and the variable
cachability. They could be very helpful for me. For the tag, I meant the
physical address field of the valid tag. Thank you again.


Regards,
JS.


-----Original Message-----
From: Kevin D. Kissell [mailto:kevink@mips.com] 
Sent: Monday, April 24, 2006 4:41 PM
To: Kim, Jong-Sung; linux-mips@linux-mips.org
Subject: Re: Reading an entire cacheline

Yes, your question was too brief the first time around - I couldn't
tell what you were really asking.  I'm still not 100% sure of what
you're doing, though.   I don't want to sound insulting, but you're
disabling interrupts during this operation, right?  And if you're
copying the values into variables in memory, you're doing nothing
but non-cached references, right?  When you see the tags changing
on you, which bits are changing, and what are they defined to mean?

            Regards,

            Kevin K.

----- Original Message ----- 
From: "Kim, Jong-Sung" <jsungkim@lge.com>
To: <linux-mips@linux-mips.org>
Sent: Monday, April 24, 2006 9:23 AM
Subject: RE: Reading an entire cacheline


> Hi all,
> 
> Was my question too brief? I'm using MIPS5Kf core included in a Broadcom
> implemented processor and it has a 32kB d-cache and a same sized i-cache.
> Cache configuration is 2-way, 32-byte cacheline for both d- and i-cache
> without L2 cache. Of course, MIPS-port Linux as the kernel for it. And, a
> onboard block device has problem in its operation. I don't think it's a
> kernel problem. So I wanna inspect the contents of cache RAMs at the
device
> driver's context. I've added some lines of codes look like follow:
> 
> __asm__ __volatile__(
> ".set noreorder\n\t"
> ".set mips3\n\t"
> "cache 5, 0x00(%3)\n\t" // load d-cache tag
> ".set mips0\n\t" // by index
> ".set mips32\n\t"
> "mfc0 %0, $28, 0\n\t"
> "mfc0 %1, $28, 1\n\t"
> "mfc0 %2, $29, 1\n\t"
> .set mips0\n\t"
> .set reorder"
> : "=r" (tag), "=r" (data_lo), "=r" (data_hi)
> : "r" (0x80000000 | (way << 14) | (index << 5) | (word <<
> 3))
> );
> 
> This is executed for every index, every way, and every word. The thing
aches
> my head is the tags from a single cacheline differs. (very often for
> d-cache) So I modified the code to reread the cacheline from the first
word
> when the tag was modified before reading all four words. Then the code
never
> return. I couldn't find any related information from manual.
> 
> Please give me a hint for reading whole cacheline safely.
> 
> 
> -----Original Message-----
> From: linux-mips-bounce@linux-mips.org
> [mailto:linux-mips-bounce@linux-mips.org] On Behalf Of Kim, Jong-Sung
> Sent: Tuesday, April 18, 2006 9:37 AM
> To: linux-mips@linux-mips.org
> Subject: Reading an entire cacheline
> 
> Hi,
> 
> Is there any way to read an entire cacheline from the MIPS5Kf? Four
> sequential cache instructions do similarly, but sometimes the tags from a
> same cacheline differ. How can I read a consistent cacheline safely?
> 
> Thanks.
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 

RE: Reading an entire cacheline

[Kim, Jong-Sung]

Hi all,

Please look at following codes:

save_flags(flags);
cli();
__asm__ __volatile__(
	"	.set	noreorder\n"
	"	.set	mips32\n"
	"2:	.set	mips3\n"
	"	cache	5, 0x00(%12)\n"
	"	.set	mips0\n"
	"	mfc0	%0, $28, 0\n"
	"	mfc0	%1, $28, 1\n"
	"	mfc0	%2, $29, 1\n"
	"	.set	mips3\n"
	"	cache	5, 0x08(%12)\n"
	"	.set	mips0\n"
	"	mfc0	%3, $28, 0\n"
	"	mfc0	%4, $28, 1\n"
	"	mfc0	%5, $29, 1\n"
	//"	bne	%0, %3, 2b\n"
	"	.set	mips3\n"
	"	cache	5, 0x10(%12)\n"
	"	.set	mips0\n"
	"	mfc0	%6, $28, 0\n"
	"	mfc0	%7, $28, 1\n"
	"	mfc0	%8, $29, 1\n"
	//"	bne	%0, %6, 2b\n"
	"	.set	mips3\n"
	"	cache	5, 0x18(%12)\n"
	"	.set	mips0\n"
	"	mfc0	%9, $28, 0\n"
	"	mfc0	%10, $28, 1\n"
	"	mfc0	%11, $29, 1\n"
	//"	bne	%0, %9, 2b\n"
	"	.set	mips0\n"
	"	.set	reorder"
	: "=r" (tag[1][way][0]), "=r" (datalo[1][way][0]),
	  "=r" (datahi[1][way][0]),
	  "=r" (tag[1][way][1]), "=r" (datalo[1][way][1]),
	  "=r" (datahi[1][way][1]),
	  "=r" (tag[1][way][2]), "=r" (datalo[1][way][2]),
	  "=r" (datahi[1][way][2]),
	  "=r" (tag[1][way][3]), "=r" (datalo[1][way][3]),
	  "=r" (datahi[1][way][3])
	: "r" (0x80000000 | (way << 14) | (line << 5))
);
restore_flags(flags);

Interrupt is disabled and no memory variable is touched while reading four
words in a cacheline. When bne instructions're activated, this code still
makes a infinitive loop. I don't know what does make changes in the d-cache.
Or.. am I doing something wrong? Any idea'll be welcomed. Thanks.

Regards,
JS.


PS. if you think my question is not proper in this mailing list, please let
me know.


-----Original Message-----
From: linux-mips-bounce@linux-mips.org
[mailto:linux-mips-bounce@linux-mips.org] On Behalf Of Kim, Jong-Sung
Sent: Monday, April 24, 2006 6:52 PM
To: linux-mips@linux-mips.org
Subject: RE: Reading an entire cacheline

Dear Mr. Kissell and the others,

I apologize for my rude question if you felt like. I was first in this
mailing list and I didn't think I had to describe the background of my
problem such in depth. I have an application stored in an onboard block
device and it sometimes makes bus error or segmentation fault during
do_page_fault().
When the application is stored in and executed from an nfs device or an
ramfs or any other device, it works without any problem. So I think the
problem comes from the device driver or something below.
In some experiments, the cache invalidation in a point reduces the frequency
of abnormal terminations. (I don't know it's a walk-around or just reduction
yet) When the application was copied from the device onto the other (ram or
network) device, it works well. So I guess the problem is related to the
cache and not from just d-cache incoherency. I targeted the consistency
between d- and i-cache as first. If you have any idea, please let me know.
For the device driver for the block device is very large and not my work,
I'm still in the dark. ^^;

Thank you for your hints about the interrupt disabling and the variable
cachability. They could be very helpful for me. For the tag, I meant the
physical address field of the valid tag. Thank you again.


Regards,
JS.


-----Original Message-----
From: Kevin D. Kissell [mailto:kevink@mips.com] 
Sent: Monday, April 24, 2006 4:41 PM
To: Kim, Jong-Sung; linux-mips@linux-mips.org
Subject: Re: Reading an entire cacheline

Yes, your question was too brief the first time around - I couldn't
tell what you were really asking.  I'm still not 100% sure of what
you're doing, though.   I don't want to sound insulting, but you're
disabling interrupts during this operation, right?  And if you're
copying the values into variables in memory, you're doing nothing
but non-cached references, right?  When you see the tags changing
on you, which bits are changing, and what are they defined to mean?

            Regards,

            Kevin K.

----- Original Message ----- 
From: "Kim, Jong-Sung" <jsungkim@lge.com>
To: <linux-mips@linux-mips.org>
Sent: Monday, April 24, 2006 9:23 AM
Subject: RE: Reading an entire cacheline


> Hi all,
> 
> Was my question too brief? I'm using MIPS5Kf core included in a Broadcom
> implemented processor and it has a 32kB d-cache and a same sized i-cache.
> Cache configuration is 2-way, 32-byte cacheline for both d- and i-cache
> without L2 cache. Of course, MIPS-port Linux as the kernel for it. And, a
> onboard block device has problem in its operation. I don't think it's a
> kernel problem. So I wanna inspect the contents of cache RAMs at the
device
> driver's context. I've added some lines of codes look like follow:
> 
> __asm__ __volatile__(
> ".set noreorder\n\t"
> ".set mips3\n\t"
> "cache 5, 0x00(%3)\n\t" // load d-cache tag
> ".set mips0\n\t" // by index
> ".set mips32\n\t"
> "mfc0 %0, $28, 0\n\t"
> "mfc0 %1, $28, 1\n\t"
> "mfc0 %2, $29, 1\n\t"
> .set mips0\n\t"
> .set reorder"
> : "=r" (tag), "=r" (data_lo), "=r" (data_hi)
> : "r" (0x80000000 | (way << 14) | (index << 5) | (word <<
> 3))
> );
> 
> This is executed for every index, every way, and every word. The thing
aches
> my head is the tags from a single cacheline differs. (very often for
> d-cache) So I modified the code to reread the cacheline from the first
word
> when the tag was modified before reading all four words. Then the code
never
> return. I couldn't find any related information from manual.
> 
> Please give me a hint for reading whole cacheline safely.
> 
> 
> -----Original Message-----
> From: linux-mips-bounce@linux-mips.org
> [mailto:linux-mips-bounce@linux-mips.org] On Behalf Of Kim, Jong-Sung
> Sent: Tuesday, April 18, 2006 9:37 AM
> To: linux-mips@linux-mips.org
> Subject: Reading an entire cacheline
> 
> Hi,
> 
> Is there any way to read an entire cacheline from the MIPS5Kf? Four
> sequential cache instructions do similarly, but sometimes the tags from a
> same cacheline differ. How can I read a consistent cacheline safely?
> 
> Thanks.
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 

RE: Reading an entire cacheline

[Kim, Jong-Sung]

Hi all,

Please look at following codes:

save_flags(flags);
cli();
__asm__ __volatile__(
	"	.set	noreorder\n"
	"	.set	mips32\n"
	"2:	.set	mips3\n"
	"	cache	5, 0x00(%12)\n"
	"	.set	mips0\n"
	"	mfc0	%0, $28, 0\n"
	"	mfc0	%1, $28, 1\n"
	"	mfc0	%2, $29, 1\n"
	"	.set	mips3\n"
	"	cache	5, 0x08(%12)\n"
	"	.set	mips0\n"
	"	mfc0	%3, $28, 0\n"
	"	mfc0	%4, $28, 1\n"
	"	mfc0	%5, $29, 1\n"
	//"	bne	%0, %3, 2b\n"
	"	.set	mips3\n"
	"	cache	5, 0x10(%12)\n"
	"	.set	mips0\n"
	"	mfc0	%6, $28, 0\n"
	"	mfc0	%7, $28, 1\n"
	"	mfc0	%8, $29, 1\n"
	//"	bne	%0, %6, 2b\n"
	"	.set	mips3\n"
	"	cache	5, 0x18(%12)\n"
	"	.set	mips0\n"
	"	mfc0	%9, $28, 0\n"
	"	mfc0	%10, $28, 1\n"
	"	mfc0	%11, $29, 1\n"
	//"	bne	%0, %9, 2b\n"
	"	.set	mips0\n"
	"	.set	reorder"
	: "=r" (tag[1][way][0]), "=r" (datalo[1][way][0]),
	  "=r" (datahi[1][way][0]),
	  "=r" (tag[1][way][1]), "=r" (datalo[1][way][1]),
	  "=r" (datahi[1][way][1]),
	  "=r" (tag[1][way][2]), "=r" (datalo[1][way][2]),
	  "=r" (datahi[1][way][2]),
	  "=r" (tag[1][way][3]), "=r" (datalo[1][way][3]),
	  "=r" (datahi[1][way][3])
	: "r" (0x80000000 | (way << 14) | (line << 5))
);
restore_flags(flags);

Interrupt is disabled and no memory variable is touched while reading four
words in a cacheline. When bne instructions're activated, this code still
makes a infinitive loop. I don't know what does make changes in the d-cache.
Or.. am I doing something wrong? Any idea'll be welcomed. Thanks.

Regards,
JS.


PS. if you think my question is not proper in this mailing list, please let
me know.


-----Original Message-----
From: linux-mips-bounce@linux-mips.org
[mailto:linux-mips-bounce@linux-mips.org] On Behalf Of Kim, Jong-Sung
Sent: Monday, April 24, 2006 6:52 PM
To: linux-mips@linux-mips.org
Subject: RE: Reading an entire cacheline

Dear Mr. Kissell and the others,

I apologize for my rude question if you felt like. I was first in this
mailing list and I didn't think I had to describe the background of my
problem such in depth. I have an application stored in an onboard block
device and it sometimes makes bus error or segmentation fault during
do_page_fault().
When the application is stored in and executed from an nfs device or an
ramfs or any other device, it works without any problem. So I think the
problem comes from the device driver or something below.
In some experiments, the cache invalidation in a point reduces the frequency
of abnormal terminations. (I don't know it's a walk-around or just reduction
yet) When the application was copied from the device onto the other (ram or
network) device, it works well. So I guess the problem is related to the
cache and not from just d-cache incoherency. I targeted the consistency
between d- and i-cache as first. If you have any idea, please let me know.
For the device driver for the block device is very large and not my work,
I'm still in the dark. ^^;

Thank you for your hints about the interrupt disabling and the variable
cachability. They could be very helpful for me. For the tag, I meant the
physical address field of the valid tag. Thank you again.


Regards,
JS.


-----Original Message-----
From: Kevin D. Kissell [mailto:kevink@mips.com] 
Sent: Monday, April 24, 2006 4:41 PM
To: Kim, Jong-Sung; linux-mips@linux-mips.org
Subject: Re: Reading an entire cacheline

Yes, your question was too brief the first time around - I couldn't
tell what you were really asking.  I'm still not 100% sure of what
you're doing, though.   I don't want to sound insulting, but you're
disabling interrupts during this operation, right?  And if you're
copying the values into variables in memory, you're doing nothing
but non-cached references, right?  When you see the tags changing
on you, which bits are changing, and what are they defined to mean?

            Regards,

            Kevin K.

----- Original Message ----- 
From: "Kim, Jong-Sung" <jsungkim@lge.com>
To: <linux-mips@linux-mips.org>
Sent: Monday, April 24, 2006 9:23 AM
Subject: RE: Reading an entire cacheline


> Hi all,
> 
> Was my question too brief? I'm using MIPS5Kf core included in a Broadcom
> implemented processor and it has a 32kB d-cache and a same sized i-cache.
> Cache configuration is 2-way, 32-byte cacheline for both d- and i-cache
> without L2 cache. Of course, MIPS-port Linux as the kernel for it. And, a
> onboard block device has problem in its operation. I don't think it's a
> kernel problem. So I wanna inspect the contents of cache RAMs at the
device
> driver's context. I've added some lines of codes look like follow:
> 
> __asm__ __volatile__(
> ".set noreorder\n\t"
> ".set mips3\n\t"
> "cache 5, 0x00(%3)\n\t" // load d-cache tag
> ".set mips0\n\t" // by index
> ".set mips32\n\t"
> "mfc0 %0, $28, 0\n\t"
> "mfc0 %1, $28, 1\n\t"
> "mfc0 %2, $29, 1\n\t"
> .set mips0\n\t"
> .set reorder"
> : "=r" (tag), "=r" (data_lo), "=r" (data_hi)
> : "r" (0x80000000 | (way << 14) | (index << 5) | (word <<
> 3))
> );
> 
> This is executed for every index, every way, and every word. The thing
aches
> my head is the tags from a single cacheline differs. (very often for
> d-cache) So I modified the code to reread the cacheline from the first
word
> when the tag was modified before reading all four words. Then the code
never
> return. I couldn't find any related information from manual.
> 
> Please give me a hint for reading whole cacheline safely.
> 
> 
> -----Original Message-----
> From: linux-mips-bounce@linux-mips.org
> [mailto:linux-mips-bounce@linux-mips.org] On Behalf Of Kim, Jong-Sung
> Sent: Tuesday, April 18, 2006 9:37 AM
> To: linux-mips@linux-mips.org
> Subject: Reading an entire cacheline
> 
> Hi,
> 
> Is there any way to read an entire cacheline from the MIPS5Kf? Four
> sequential cache instructions do similarly, but sometimes the tags from a
> same cacheline differ. How can I read a consistent cacheline safely?
> 
> Thanks.
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 

Re: Reading an entire cacheline

[Kevin D. Kissell]

There are some things that look a little suspicious in you code below.
Please disassemble(mips{el}-linux-objdump --disassemble) the resulting
binary and look at what *really* gets generated. Pay special attention 
to branch  delay slots - you've set noreorder, so it's your responsibility 
to make sure they're handled correctly.

            Regards,

            Kevin K.

----- Original Message ----- 
From: "Kim, Jong-Sung" <jsungkim@lge.com>
To: <linux-mips@linux-mips.org>
Sent: Wednesday, April 26, 2006 10:47 AM
Subject: RE: Reading an entire cacheline


> Hi all,
> 
> Please look at following codes:
> 
> save_flags(flags);
> cli();
> __asm__ __volatile__(
> " .set noreorder\n"
> " .set mips32\n"
> "2: .set mips3\n"
> " cache 5, 0x00(%12)\n"
> " .set mips0\n"
> " mfc0 %0, $28, 0\n"
> " mfc0 %1, $28, 1\n"
> " mfc0 %2, $29, 1\n"
> " .set mips3\n"
> " cache 5, 0x08(%12)\n"
> " .set mips0\n"
> " mfc0 %3, $28, 0\n"
> " mfc0 %4, $28, 1\n"
> " mfc0 %5, $29, 1\n"
> //" bne %0, %3, 2b\n"
> " .set mips3\n"
> " cache 5, 0x10(%12)\n"
> " .set mips0\n"
> " mfc0 %6, $28, 0\n"
> " mfc0 %7, $28, 1\n"
> " mfc0 %8, $29, 1\n"
> //" bne %0, %6, 2b\n"
> " .set mips3\n"
> " cache 5, 0x18(%12)\n"
> " .set mips0\n"
> " mfc0 %9, $28, 0\n"
> " mfc0 %10, $28, 1\n"
> " mfc0 %11, $29, 1\n"
> //" bne %0, %9, 2b\n"
> " .set mips0\n"
> " .set reorder"
> : "=r" (tag[1][way][0]), "=r" (datalo[1][way][0]),
>   "=r" (datahi[1][way][0]),
>   "=r" (tag[1][way][1]), "=r" (datalo[1][way][1]),
>   "=r" (datahi[1][way][1]),
>   "=r" (tag[1][way][2]), "=r" (datalo[1][way][2]),
>   "=r" (datahi[1][way][2]),
>   "=r" (tag[1][way][3]), "=r" (datalo[1][way][3]),
>   "=r" (datahi[1][way][3])
> : "r" (0x80000000 | (way << 14) | (line << 5))
> );
> restore_flags(flags);
> 
> Interrupt is disabled and no memory variable is touched while reading four
> words in a cacheline. When bne instructions're activated, this code still
> makes a infinitive loop. I don't know what does make changes in the d-cache.
> Or.. am I doing something wrong? Any idea'll be welcomed. Thanks.
> 
> Regards,
> JS.
> 
> 
> PS. if you think my question is not proper in this mailing list, please let
> me know.
> 
> 
> -----Original Message-----
> From: linux-mips-bounce@linux-mips.org
> [mailto:linux-mips-bounce@linux-mips.org] On Behalf Of Kim, Jong-Sung
> Sent: Monday, April 24, 2006 6:52 PM
> To: linux-mips@linux-mips.org
> Subject: RE: Reading an entire cacheline
> 
> Dear Mr. Kissell and the others,
> 
> I apologize for my rude question if you felt like. I was first in this
> mailing list and I didn't think I had to describe the background of my
> problem such in depth. I have an application stored in an onboard block
> device and it sometimes makes bus error or segmentation fault during
> do_page_fault().
> When the application is stored in and executed from an nfs device or an
> ramfs or any other device, it works without any problem. So I think the
> problem comes from the device driver or something below.
> In some experiments, the cache invalidation in a point reduces the frequency
> of abnormal terminations. (I don't know it's a walk-around or just reduction
> yet) When the application was copied from the device onto the other (ram or
> network) device, it works well. So I guess the problem is related to the
> cache and not from just d-cache incoherency. I targeted the consistency
> between d- and i-cache as first. If you have any idea, please let me know.
> For the device driver for the block device is very large and not my work,
> I'm still in the dark. ^^;
> 
> Thank you for your hints about the interrupt disabling and the variable
> cachability. They could be very helpful for me. For the tag, I meant the
> physical address field of the valid tag. Thank you again.
> 
> 
> Regards,
> JS.
> 
> 
> -----Original Message-----
> From: Kevin D. Kissell [mailto:kevink@mips.com] 
> Sent: Monday, April 24, 2006 4:41 PM
> To: Kim, Jong-Sung; linux-mips@linux-mips.org
> Subject: Re: Reading an entire cacheline
> 
> Yes, your question was too brief the first time around - I couldn't
> tell what you were really asking.  I'm still not 100% sure of what
> you're doing, though.   I don't want to sound insulting, but you're
> disabling interrupts during this operation, right?  And if you're
> copying the values into variables in memory, you're doing nothing
> but non-cached references, right?  When you see the tags changing
> on you, which bits are changing, and what are they defined to mean?
> 
>             Regards,
> 
>             Kevin K.
> 
> ----- Original Message ----- 
> From: "Kim, Jong-Sung" <jsungkim@lge.com>
> To: <linux-mips@linux-mips.org>
> Sent: Monday, April 24, 2006 9:23 AM
> Subject: RE: Reading an entire cacheline
> 
> 
> > Hi all,
> > 
> > Was my question too brief? I'm using MIPS5Kf core included in a Broadcom
> > implemented processor and it has a 32kB d-cache and a same sized i-cache.
> > Cache configuration is 2-way, 32-byte cacheline for both d- and i-cache
> > without L2 cache. Of course, MIPS-port Linux as the kernel for it. And, a
> > onboard block device has problem in its operation. I don't think it's a
> > kernel problem. So I wanna inspect the contents of cache RAMs at the
> device
> > driver's context. I've added some lines of codes look like follow:
> > 
> > __asm__ __volatile__(
> > ".set noreorder\n\t"
> > ".set mips3\n\t"
> > "cache 5, 0x00(%3)\n\t" // load d-cache tag
> > ".set mips0\n\t" // by index
> > ".set mips32\n\t"
> > "mfc0 %0, $28, 0\n\t"
> > "mfc0 %1, $28, 1\n\t"
> > "mfc0 %2, $29, 1\n\t"
> > .set mips0\n\t"
> > .set reorder"
> > : "=r" (tag), "=r" (data_lo), "=r" (data_hi)
> > : "r" (0x80000000 | (way << 14) | (index << 5) | (word <<
> > 3))
> > );
> > 
> > This is executed for every index, every way, and every word. The thing
> aches
> > my head is the tags from a single cacheline differs. (very often for
> > d-cache) So I modified the code to reread the cacheline from the first
> word
> > when the tag was modified before reading all four words. Then the code
> never
> > return. I couldn't find any related information from manual.
> > 
> > Please give me a hint for reading whole cacheline safely.
> > 
> > 
> > -----Original Message-----
> > From: linux-mips-bounce@linux-mips.org
> > [mailto:linux-mips-bounce@linux-mips.org] On Behalf Of Kim, Jong-Sung
> > Sent: Tuesday, April 18, 2006 9:37 AM
> > To: linux-mips@linux-mips.org
> > Subject: Reading an entire cacheline
> > 
> > Hi,
> > 
> > Is there any way to read an entire cacheline from the MIPS5Kf? Four
> > sequential cache instructions do similarly, but sometimes the tags from a
> > same cacheline differ. How can I read a consistent cacheline safely?
> > 
> > Thanks. 

Re: Reading an entire cacheline

[Kevin D. Kissell]

There are some things that look a little suspicious in you code below.
Please disassemble(mips{el}-linux-objdump --disassemble) the resulting
binary and look at what *really* gets generated. Pay special attention 
to branch  delay slots - you've set noreorder, so it's your responsibility 
to make sure they're handled correctly.

            Regards,

            Kevin K.

----- Original Message ----- 
From: "Kim, Jong-Sung" <jsungkim@lge.com>
To: <linux-mips@linux-mips.org>
Sent: Wednesday, April 26, 2006 10:47 AM
Subject: RE: Reading an entire cacheline


> Hi all,
> 
> Please look at following codes:
> 
> save_flags(flags);
> cli();
> __asm__ __volatile__(
> " .set noreorder\n"
> " .set mips32\n"
> "2: .set mips3\n"
> " cache 5, 0x00(%12)\n"
> " .set mips0\n"
> " mfc0 %0, $28, 0\n"
> " mfc0 %1, $28, 1\n"
> " mfc0 %2, $29, 1\n"
> " .set mips3\n"
> " cache 5, 0x08(%12)\n"
> " .set mips0\n"
> " mfc0 %3, $28, 0\n"
> " mfc0 %4, $28, 1\n"
> " mfc0 %5, $29, 1\n"
> //" bne %0, %3, 2b\n"
> " .set mips3\n"
> " cache 5, 0x10(%12)\n"
> " .set mips0\n"
> " mfc0 %6, $28, 0\n"
> " mfc0 %7, $28, 1\n"
> " mfc0 %8, $29, 1\n"
> //" bne %0, %6, 2b\n"
> " .set mips3\n"
> " cache 5, 0x18(%12)\n"
> " .set mips0\n"
> " mfc0 %9, $28, 0\n"
> " mfc0 %10, $28, 1\n"
> " mfc0 %11, $29, 1\n"
> //" bne %0, %9, 2b\n"
> " .set mips0\n"
> " .set reorder"
> : "=r" (tag[1][way][0]), "=r" (datalo[1][way][0]),
>   "=r" (datahi[1][way][0]),
>   "=r" (tag[1][way][1]), "=r" (datalo[1][way][1]),
>   "=r" (datahi[1][way][1]),
>   "=r" (tag[1][way][2]), "=r" (datalo[1][way][2]),
>   "=r" (datahi[1][way][2]),
>   "=r" (tag[1][way][3]), "=r" (datalo[1][way][3]),
>   "=r" (datahi[1][way][3])
> : "r" (0x80000000 | (way << 14) | (line << 5))
> );
> restore_flags(flags);
> 
> Interrupt is disabled and no memory variable is touched while reading four
> words in a cacheline. When bne instructions're activated, this code still
> makes a infinitive loop. I don't know what does make changes in the d-cache.
> Or.. am I doing something wrong? Any idea'll be welcomed. Thanks.
> 
> Regards,
> JS.
> 
> 
> PS. if you think my question is not proper in this mailing list, please let
> me know.
> 
> 
> -----Original Message-----
> From: linux-mips-bounce@linux-mips.org
> [mailto:linux-mips-bounce@linux-mips.org] On Behalf Of Kim, Jong-Sung
> Sent: Monday, April 24, 2006 6:52 PM
> To: linux-mips@linux-mips.org
> Subject: RE: Reading an entire cacheline
> 
> Dear Mr. Kissell and the others,
> 
> I apologize for my rude question if you felt like. I was first in this
> mailing list and I didn't think I had to describe the background of my
> problem such in depth. I have an application stored in an onboard block
> device and it sometimes makes bus error or segmentation fault during
> do_page_fault().
> When the application is stored in and executed from an nfs device or an
> ramfs or any other device, it works without any problem. So I think the
> problem comes from the device driver or something below.
> In some experiments, the cache invalidation in a point reduces the frequency
> of abnormal terminations. (I don't know it's a walk-around or just reduction
> yet) When the application was copied from the device onto the other (ram or
> network) device, it works well. So I guess the problem is related to the
> cache and not from just d-cache incoherency. I targeted the consistency
> between d- and i-cache as first. If you have any idea, please let me know.
> For the device driver for the block device is very large and not my work,
> I'm still in the dark. ^^;
> 
> Thank you for your hints about the interrupt disabling and the variable
> cachability. They could be very helpful for me. For the tag, I meant the
> physical address field of the valid tag. Thank you again.
> 
> 
> Regards,
> JS.
> 
> 
> -----Original Message-----
> From: Kevin D. Kissell [mailto:kevink@mips.com] 
> Sent: Monday, April 24, 2006 4:41 PM
> To: Kim, Jong-Sung; linux-mips@linux-mips.org
> Subject: Re: Reading an entire cacheline
> 
> Yes, your question was too brief the first time around - I couldn't
> tell what you were really asking.  I'm still not 100% sure of what
> you're doing, though.   I don't want to sound insulting, but you're
> disabling interrupts during this operation, right?  And if you're
> copying the values into variables in memory, you're doing nothing
> but non-cached references, right?  When you see the tags changing
> on you, which bits are changing, and what are they defined to mean?
> 
>             Regards,
> 
>             Kevin K.
> 
> ----- Original Message ----- 
> From: "Kim, Jong-Sung" <jsungkim@lge.com>
> To: <linux-mips@linux-mips.org>
> Sent: Monday, April 24, 2006 9:23 AM
> Subject: RE: Reading an entire cacheline
> 
> 
> > Hi all,
> > 
> > Was my question too brief? I'm using MIPS5Kf core included in a Broadcom
> > implemented processor and it has a 32kB d-cache and a same sized i-cache.
> > Cache configuration is 2-way, 32-byte cacheline for both d- and i-cache
> > without L2 cache. Of course, MIPS-port Linux as the kernel for it. And, a
> > onboard block device has problem in its operation. I don't think it's a
> > kernel problem. So I wanna inspect the contents of cache RAMs at the
> device
> > driver's context. I've added some lines of codes look like follow:
> > 
> > __asm__ __volatile__(
> > ".set noreorder\n\t"
> > ".set mips3\n\t"
> > "cache 5, 0x00(%3)\n\t" // load d-cache tag
> > ".set mips0\n\t" // by index
> > ".set mips32\n\t"
> > "mfc0 %0, $28, 0\n\t"
> > "mfc0 %1, $28, 1\n\t"
> > "mfc0 %2, $29, 1\n\t"
> > .set mips0\n\t"
> > .set reorder"
> > : "=r" (tag), "=r" (data_lo), "=r" (data_hi)
> > : "r" (0x80000000 | (way << 14) | (index << 5) | (word <<
> > 3))
> > );
> > 
> > This is executed for every index, every way, and every word. The thing
> aches
> > my head is the tags from a single cacheline differs. (very often for
> > d-cache) So I modified the code to reread the cacheline from the first
> word
> > when the tag was modified before reading all four words. Then the code
> never
> > return. I couldn't find any related information from manual.
> > 
> > Please give me a hint for reading whole cacheline safely.
> > 
> > 
> > -----Original Message-----
> > From: linux-mips-bounce@linux-mips.org
> > [mailto:linux-mips-bounce@linux-mips.org] On Behalf Of Kim, Jong-Sung
> > Sent: Tuesday, April 18, 2006 9:37 AM
> > To: linux-mips@linux-mips.org
> > Subject: Reading an entire cacheline
> > 
> > Hi,
> > 
> > Is there any way to read an entire cacheline from the MIPS5Kf? Four
> > sequential cache instructions do similarly, but sometimes the tags from a
> > same cacheline differ. How can I read a consistent cacheline safely?
> > 
> > Thanks. 

Re: Reading an entire cacheline

[Thiemo Seufer]

Kim, Jong-Sung wrote:
> Hi all,
> 
> Please look at following codes:
> 
> save_flags(flags);
> cli();
> __asm__ __volatile__(
> 	"	.set	noreorder\n"
> 	"	.set	mips32\n"
> 	"2:	.set	mips3\n"
> 	"	cache	5, 0x00(%12)\n"
> 	"	.set	mips0\n"

Irrelevant sidemark:
.set mips0 resets to the original value, not to mips32. You probably want

	.set	push
	.set	noreorder
	.set 	mips32

	... <the whole code sequence>

	.set	pop

[snip]
> 	//"	bne	%0, %9, 2b\n"
> 	"	.set	mips0\n"
> 	"	.set	reorder"
> 	: "=r" (tag[1][way][0]), "=r" (datalo[1][way][0]),
> 	  "=r" (datahi[1][way][0]),
> 	  "=r" (tag[1][way][1]), "=r" (datalo[1][way][1]),
> 	  "=r" (datahi[1][way][1]),
> 	  "=r" (tag[1][way][2]), "=r" (datalo[1][way][2]),
> 	  "=r" (datahi[1][way][2]),
> 	  "=r" (tag[1][way][3]), "=r" (datalo[1][way][3]),
> 	  "=r" (datahi[1][way][3])
> 	: "r" (0x80000000 | (way << 14) | (line << 5))
> );

And this part may cause the problem you are seeing, I presume
datalo/datahi live in memory, and accesses of it change the dcache.

As Kevin mentioned, disassembling the binary might be helpful.


Thiemo

Re: Reading an entire cacheline

[Kevin D. Kissell]

> [snip]
> > //" bne %0, %9, 2b\n"
> > " .set mips0\n"
> > " .set reorder"
> > : "=r" (tag[1][way][0]), "=r" (datalo[1][way][0]),
> >   "=r" (datahi[1][way][0]),
> >   "=r" (tag[1][way][1]), "=r" (datalo[1][way][1]),
> >   "=r" (datahi[1][way][1]),
> >   "=r" (tag[1][way][2]), "=r" (datalo[1][way][2]),
> >   "=r" (datahi[1][way][2]),
> >   "=r" (tag[1][way][3]), "=r" (datalo[1][way][3]),
> >   "=r" (datahi[1][way][3])
> > : "r" (0x80000000 | (way << 14) | (line << 5))
> > );
> 
> And this part may cause the problem you are seeing, I presume
> datalo/datahi live in memory, and accesses of it change the dcache.

As I was hoping disassembly would demonstrate for you,
declaring "=r" doesn't mean that the variable has no life
outside a register.

            Regards,

            Kevin K.

Re: Reading an entire cacheline

[Kevin D. Kissell]

> [snip]
> > //" bne %0, %9, 2b\n"
> > " .set mips0\n"
> > " .set reorder"
> > : "=r" (tag[1][way][0]), "=r" (datalo[1][way][0]),
> >   "=r" (datahi[1][way][0]),
> >   "=r" (tag[1][way][1]), "=r" (datalo[1][way][1]),
> >   "=r" (datahi[1][way][1]),
> >   "=r" (tag[1][way][2]), "=r" (datalo[1][way][2]),
> >   "=r" (datahi[1][way][2]),
> >   "=r" (tag[1][way][3]), "=r" (datalo[1][way][3]),
> >   "=r" (datahi[1][way][3])
> > : "r" (0x80000000 | (way << 14) | (line << 5))
> > );
> 
> And this part may cause the problem you are seeing, I presume
> datalo/datahi live in memory, and accesses of it change the dcache.

As I was hoping disassembly would demonstrate for you,
declaring "=r" doesn't mean that the variable has no life
outside a register.

            Regards,

            Kevin K.

RE: Reading an entire cacheline

[Kim, Jong-Sung]

Dear Mr. Kissell,

Thank you for your advices about disassembling. Please see:

1380:	...
1384:	00431025	or	v0,v0,v1
1388:	00441025	or	v0,v0,a0
138c:	bc450000	cache	0x5,0(v0)
1390:	4002e000	mfc0	v0,c0_taglo
1394:	4003e001	mfc0	v1,c0_datalo
1398:	4004e801	mfc0	a0,c0_datahi
139c:	bc450008	cache	0x5,8(v0)
13a0:	4005e000	mfc0	a1,c0_taglo
13a4:	4006e001	mfc0	a2,c0_datalo
13a8:	4007e801	mfc0	a3,c0_datahi
13ac:	bc450010	cache	0x5,16(v0)
13b0:	4008e000	mfc0	t0,c0_taglo
13b4:	4009e001	mfc0	t1,c0_datalo
13b8:	400ae801	mfc0	t2,c0_datahi
13bc:	bc450010	cache	0x5,16(v0)
13c0:	400be000	mfc0	t3,c0_taglo
13c4:	400ee001	mfc0	t6,c0_datalo
13c8:	400ce801	mfc0	t4,c0_datahi
13cc:	ae020020	sw	v0,32(s0)
13d0:	ae030060	sw	v1,96(s0)
13d4:	...

It seems no memory variable is accessed during the sequence. The obviously
unexpected thing is the use of register v0. It should have the value of the
index and should not be touched before the execution of the last cache
instruction. In this way, tags read are doomed to be different.

It's my fault. So, I corrected the "=r" to be "=&r", and it seems work.
Thank you again.

Regards,
JS.


-----Original Message-----
From: linux-mips-bounce@linux-mips.org
[mailto:linux-mips-bounce@linux-mips.org] On Behalf Of Kevin D. Kissell
Sent: Wednesday, April 26, 2006 8:25 PM
To: Kim, Jong-Sung; Thiemo Seufer
Cc: linux-mips@linux-mips.org
Subject: Re: Reading an entire cacheline

> [snip]
> > //" bne %0, %9, 2b\n"
> > " .set mips0\n"
> > " .set reorder"
> > : "=r" (tag[1][way][0]), "=r" (datalo[1][way][0]),
> >   "=r" (datahi[1][way][0]),
> >   "=r" (tag[1][way][1]), "=r" (datalo[1][way][1]),
> >   "=r" (datahi[1][way][1]),
> >   "=r" (tag[1][way][2]), "=r" (datalo[1][way][2]),
> >   "=r" (datahi[1][way][2]),
> >   "=r" (tag[1][way][3]), "=r" (datalo[1][way][3]),
> >   "=r" (datahi[1][way][3])
> > : "r" (0x80000000 | (way << 14) | (line << 5))
> > );
> 
> And this part may cause the problem you are seeing, I presume
> datalo/datahi live in memory, and accesses of it change the dcache.

As I was hoping disassembly would demonstrate for you,
declaring "=r" doesn't mean that the variable has no life
outside a register.

            Regards,

            Kevin K.

RE: Reading an entire cacheline

[Kim, Jong-Sung]

Dear Mr. Kissell,

Thank you for your advices about disassembling. Please see:

1380:	...
1384:	00431025	or	v0,v0,v1
1388:	00441025	or	v0,v0,a0
138c:	bc450000	cache	0x5,0(v0)
1390:	4002e000	mfc0	v0,c0_taglo
1394:	4003e001	mfc0	v1,c0_datalo
1398:	4004e801	mfc0	a0,c0_datahi
139c:	bc450008	cache	0x5,8(v0)
13a0:	4005e000	mfc0	a1,c0_taglo
13a4:	4006e001	mfc0	a2,c0_datalo
13a8:	4007e801	mfc0	a3,c0_datahi
13ac:	bc450010	cache	0x5,16(v0)
13b0:	4008e000	mfc0	t0,c0_taglo
13b4:	4009e001	mfc0	t1,c0_datalo
13b8:	400ae801	mfc0	t2,c0_datahi
13bc:	bc450010	cache	0x5,16(v0)
13c0:	400be000	mfc0	t3,c0_taglo
13c4:	400ee001	mfc0	t6,c0_datalo
13c8:	400ce801	mfc0	t4,c0_datahi
13cc:	ae020020	sw	v0,32(s0)
13d0:	ae030060	sw	v1,96(s0)
13d4:	...

It seems no memory variable is accessed during the sequence. The obviously
unexpected thing is the use of register v0. It should have the value of the
index and should not be touched before the execution of the last cache
instruction. In this way, tags read are doomed to be different.

It's my fault. So, I corrected the "=r" to be "=&r", and it seems work.
Thank you again.

Regards,
JS.


-----Original Message-----
From: linux-mips-bounce@linux-mips.org
[mailto:linux-mips-bounce@linux-mips.org] On Behalf Of Kevin D. Kissell
Sent: Wednesday, April 26, 2006 8:25 PM
To: Kim, Jong-Sung; Thiemo Seufer
Cc: linux-mips@linux-mips.org
Subject: Re: Reading an entire cacheline

> [snip]
> > //" bne %0, %9, 2b\n"
> > " .set mips0\n"
> > " .set reorder"
> > : "=r" (tag[1][way][0]), "=r" (datalo[1][way][0]),
> >   "=r" (datahi[1][way][0]),
> >   "=r" (tag[1][way][1]), "=r" (datalo[1][way][1]),
> >   "=r" (datahi[1][way][1]),
> >   "=r" (tag[1][way][2]), "=r" (datalo[1][way][2]),
> >   "=r" (datahi[1][way][2]),
> >   "=r" (tag[1][way][3]), "=r" (datalo[1][way][3]),
> >   "=r" (datahi[1][way][3])
> > : "r" (0x80000000 | (way << 14) | (line << 5))
> > );
> 
> And this part may cause the problem you are seeing, I presume
> datalo/datahi live in memory, and accesses of it change the dcache.

As I was hoping disassembly would demonstrate for you,
declaring "=r" doesn't mean that the variable has no life
outside a register.

            Regards,

            Kevin K.

RE: Reading an entire cacheline

[Kim, Jong-Sung]

Dear Mr. Seufer,

Thank you for your advice about directives. That's exactly what I meant. I
have confused them with push/pop style operations. Thanks.

Regards,
JS.

-----Original Message-----
From: Thiemo Seufer [mailto:ths@networkno.de] 
Sent: Wednesday, April 26, 2006 7:16 PM
To: Kim, Jong-Sung
Cc: linux-mips@linux-mips.org
Subject: Re: Reading an entire cacheline

Kim, Jong-Sung wrote:
> Hi all,
> 
> Please look at following codes:
> 
> save_flags(flags);
> cli();
> __asm__ __volatile__(
> 	"	.set	noreorder\n"
> 	"	.set	mips32\n"
> 	"2:	.set	mips3\n"
> 	"	cache	5, 0x00(%12)\n"
> 	"	.set	mips0\n"

Irrelevant sidemark:
.set mips0 resets to the original value, not to mips32. You probably want

	.set	push
	.set	noreorder
	.set 	mips32

	... <the whole code sequence>

	.set	pop

[snip]
> 	//"	bne	%0, %9, 2b\n"
> 	"	.set	mips0\n"
> 	"	.set	reorder"
> 	: "=r" (tag[1][way][0]), "=r" (datalo[1][way][0]),
> 	  "=r" (datahi[1][way][0]),
> 	  "=r" (tag[1][way][1]), "=r" (datalo[1][way][1]),
> 	  "=r" (datahi[1][way][1]),
> 	  "=r" (tag[1][way][2]), "=r" (datalo[1][way][2]),
> 	  "=r" (datahi[1][way][2]),
> 	  "=r" (tag[1][way][3]), "=r" (datalo[1][way][3]),
> 	  "=r" (datahi[1][way][3])
> 	: "r" (0x80000000 | (way << 14) | (line << 5))
> );

And this part may cause the problem you are seeing, I presume
datalo/datahi live in memory, and accesses of it change the dcache.

As Kevin mentioned, disassembling the binary might be helpful.


Thiemo

RE: Reading an entire cacheline

[Kim, Jong-Sung]

Dear Mr. Seufer,

Thank you for your advice about directives. That's exactly what I meant. I
have confused them with push/pop style operations. Thanks.

Regards,
JS.

-----Original Message-----
From: Thiemo Seufer [mailto:ths@networkno.de] 
Sent: Wednesday, April 26, 2006 7:16 PM
To: Kim, Jong-Sung
Cc: linux-mips@linux-mips.org
Subject: Re: Reading an entire cacheline

Kim, Jong-Sung wrote:
> Hi all,
> 
> Please look at following codes:
> 
> save_flags(flags);
> cli();
> __asm__ __volatile__(
> 	"	.set	noreorder\n"
> 	"	.set	mips32\n"
> 	"2:	.set	mips3\n"
> 	"	cache	5, 0x00(%12)\n"
> 	"	.set	mips0\n"

Irrelevant sidemark:
.set mips0 resets to the original value, not to mips32. You probably want

	.set	push
	.set	noreorder
	.set 	mips32

	... <the whole code sequence>

	.set	pop

[snip]
> 	//"	bne	%0, %9, 2b\n"
> 	"	.set	mips0\n"
> 	"	.set	reorder"
> 	: "=r" (tag[1][way][0]), "=r" (datalo[1][way][0]),
> 	  "=r" (datahi[1][way][0]),
> 	  "=r" (tag[1][way][1]), "=r" (datalo[1][way][1]),
> 	  "=r" (datahi[1][way][1]),
> 	  "=r" (tag[1][way][2]), "=r" (datalo[1][way][2]),
> 	  "=r" (datahi[1][way][2]),
> 	  "=r" (tag[1][way][3]), "=r" (datalo[1][way][3]),
> 	  "=r" (datahi[1][way][3])
> 	: "r" (0x80000000 | (way << 14) | (line << 5))
> );

And this part may cause the problem you are seeing, I presume
datalo/datahi live in memory, and accesses of it change the dcache.

As Kevin mentioned, disassembling the binary might be helpful.


Thiemo