Problem in getting shared memory access on P1022RDK

Problem in getting shared memory access on P1022RDK

[Arshad, Farrukh]

[-- Attachment #1: Type: text/plain, Size: 1381 bytes --]

Greetings All,

Please excuse me for a 'not very precise' question but I just need some clue or point to look for the trouble.
I am running linux in AMP configuration on P1022RDK dual core. My memory partitioning is given below

Core Base Address (CONFIG_PHYSICAL_START / bootm_low) Size (bootm_size / mem kernel boot parameter)
Core 0 0x0000,0000 0x1000,0000
Core 1 0x1000,0000 0x0C00,0000
Shared Memory 0x1C00,0000 0x0400,0000

I have set bootm_low / bootm_size and provided mem parameter in both kernel configurations, both
kernels are booting up fine. I have a shared memory driver running in both cores. My shared memory
driver just exposes the physical memory (0x1C00,0000) to user-space.

The problem I am facing is I can not get access to same shared memory in user-space on both kernels.
I have verified my shared memory driver on another platform (p1022ds) and it works perfect. I have also
tried using non-cached memory mapping in shared memory driver but that does not work as well.

To me it seems one of my kernels is going beyond its defined memory partition (which I have set above), I have
set kernel's partitioning as CONFIG_PHYSICAL_START, bootm_size, bootm_low and mem kernel boot param.

Besides the shared memory driver, can anyone suggest or give me clue where should I look for the
trouble in the kernel.

Regards,
Farrukh Arshad.

[-- Attachment #2: Type: text/html, Size: 2532 bytes --]

RE: Problem in getting shared memory access on P1022RDK

[Arshad, Farrukh]

[-- Attachment #1: Type: text/plain, Size: 1807 bytes --]

Adding more to it,

When I write from Core 1 on the shared memory region it is visible at Core 0 and it can read what I have written from Core 1 but when I write from Core 0 on this shared memory it is not visible on Core 1.

Regards,
Farrukh Arshad

From: Arshad, Farrukh
Sent: Tuesday, January 03, 2012 12:11 AM
To: linuxppc-dev@lists.ozlabs.org
Subject: Problem in getting shared memory access on P1022RDK

Greetings All,

Please excuse me for a 'not very precise' question but I just need some clue or point to look for the trouble.
I am running linux in AMP configuration on P1022RDK dual core. My memory partitioning is given below

Core Base Address (CONFIG_PHYSICAL_START / bootm_low)
Size (bootm_size / mem kernel boot parameter)
Core 0 0x0000,0000
0x1000,0000
Core 1 0x1000,0000
0x0C00,0000
Shared Memory 0x1C00,0000
0x0400,0000

I have set bootm_low / bootm_size and provided mem parameter in both kernel configurations, both
kernels are booting up fine. I have a shared memory driver running in both cores. My shared memory
driver just exposes the physical memory (0x1C00,0000) to user-space.

The problem I am facing is I can not get access to same shared memory in user-space on both kernels.
I have verified my shared memory driver on another platform (p1022ds) and it works perfect. I have also
tried using non-cached memory mapping in shared memory driver but that does not work as well.

To me it seems one of my kernels is going beyond its defined memory partition (which I have set above), I have
set kernel's partitioning as CONFIG_PHYSICAL_START, bootm_size, bootm_low and mem kernel boot param.

Besides the shared memory driver, can anyone suggest or give me clue where should I look for the
trouble in the kernel.

Regards,
Farrukh Arshad.

[-- Attachment #2: Type: text/html, Size: 9938 bytes --]

Re: Problem in getting shared memory access on P1022RDK

[Scott Wood]

On 01/03/2012 03:42 AM, Arshad, Farrukh wrote:
> Adding more to it,
> 
>  
> 
> When I write from Core 1 on the shared memory region it is visible at
> Core 0 and it can read what I have written from Core 1 but when I write
> from Core 0 on this shared memory it is not visible on Core 1.

Is the memory mapped coherent on both cores?

-Scott

RE: Problem in getting shared memory access on P1022RDK

[Arshad, Farrukh]

How can I verify if the memory mapped is coherent on both cores. My memory =
partitioning is given below

Core		Base Address		Size
Core 0		0x0000,0000		0x1000,0000	--> CONFIG_PHYSICAL_START =3D bootm_low =
=3D Base Address
Core 1		0x1000,0000		0x0C00,0000	--> CONFIG_PHYSICAL_START =3D bootm_low =
=3D Base Address
Shared Mem	0x1C00,0000		0x0400,0000

Regards,
Farrukh Arshad

-----Original Message-----
From: Scott Wood [mailto:scottwood@freescale.com]=20
Sent: Tuesday, January 03, 2012 10:10 PM
To: Arshad, Farrukh
Cc: linuxppc-dev@lists.ozlabs.org
Subject: Re: Problem in getting shared memory access on P1022RDK

On 01/03/2012 03:42 AM, Arshad, Farrukh wrote:
> Adding more to it,
>=20
> =20
>=20
> When I write from Core 1 on the shared memory region it is visible at=20
> Core 0 and it can read what I have written from Core 1 but when I=20
> write from Core 0 on this shared memory it is not visible on Core 1.

Is the memory mapped coherent on both cores?

-Scott

RE: Problem in getting shared memory access on P1022RDK

[Arshad, Farrukh]

Some of the other things which I am experiencing which may help to trace th=
is issue.=20

1: For Core 0 printk is not working in setup_arch "just before call to pagi=
ng_init", where as on Core 1 printk is working in the same function. I see =
the source uses ppc_md.progress to display debug messages so it seems print=
k will not work then why it is working on Core 1.=20
2: During kernel boot of both cores boot messages of Core 0 are mixed in Co=
re 1 messages.=20
3: If I change memory partitioning to following, then on Core 1 when I mmap=
 shared memory area (starting 0x0C00,0000) I receive error that can not mma=
p RAM. If my both systems memory partitions are defined clearly then this a=
rea does not belong to system RAM then I shouldn't receive this error in th=
is memory partitioning.

Core		Base Address		Size
Core 0		0x0000,0000		0x0C00,0000	--> CONFIG_PHYSICAL_START =3D bootm_low =
=3D Base Address
Shared Mem	0x0C00,0000		0x0400,0000
Core 1		0x1000,0000		0x1000,0000	--> CONFIG_PHYSICAL_START =3D bootm_low =
=3D Base Address

I believe there is some bug / or patch which I am missing in both kernels m=
emory initialization which is causing all these issues. Kernel sources / co=
nfiguration of both cores are 100% same except (CONFIG_PHYSICAL_START) para=
m and kernel version is 2.6.32.13.=20

Any thoughts.

Regards,
Farrukh Arshad


-----Original Message-----
From: Arshad, Farrukh=20
Sent: Wednesday, January 04, 2012 11:50 AM
To: 'Scott Wood'
Cc: linuxppc-dev@lists.ozlabs.org
Subject: RE: Problem in getting shared memory access on P1022RDK

How can I verify if the memory mapped is coherent on both cores. My memory =
partitioning is given below

Core		Base Address		Size
Core 0		0x0000,0000		0x1000,0000	--> CONFIG_PHYSICAL_START =3D bootm_low =
=3D Base Address
Core 1		0x1000,0000		0x0C00,0000	--> CONFIG_PHYSICAL_START =3D bootm_low =
=3D Base Address
Shared Mem	0x1C00,0000		0x0400,0000

Regards,
Farrukh Arshad

-----Original Message-----
From: Scott Wood [mailto:scottwood@freescale.com]
Sent: Tuesday, January 03, 2012 10:10 PM
To: Arshad, Farrukh
Cc: linuxppc-dev@lists.ozlabs.org
Subject: Re: Problem in getting shared memory access on P1022RDK

On 01/03/2012 03:42 AM, Arshad, Farrukh wrote:
> Adding more to it,
>=20
> =20
>=20
> When I write from Core 1 on the shared memory region it is visible at=20
> Core 0 and it can read what I have written from Core 1 but when I=20
> write from Core 0 on this shared memory it is not visible on Core 1.

Is the memory mapped coherent on both cores?

-Scott

Re: Problem in getting shared memory access on P1022RDK

[tiejun.chen]

Arshad, Farrukh wrote:
> How can I verify if the memory mapped is coherent on both cores. My memory partitioning is given below
> 
> Core		Base Address		Size
> Core 0		0x0000,0000		0x1000,0000	--> CONFIG_PHYSICAL_START = bootm_low = Base Address
> Core 1		0x1000,0000		0x0C00,0000	--> CONFIG_PHYSICAL_START = bootm_low = Base Address
> Shared Mem	0x1C00,0000		0x0400,0000

Was the kernel option, CONFIG_SMP, enabled for both two kernels?

CONFIG_SMP would affect the memory attribute for cache coherency. Maybe you
should make sure if kernel have a appropriate memory attribute by dumping TLB entry.

Tiejun

> 
> Regards,
> Farrukh Arshad
> 
> -----Original Message-----
> From: Scott Wood [mailto:scottwood@freescale.com] 
> Sent: Tuesday, January 03, 2012 10:10 PM
> To: Arshad, Farrukh
> Cc: linuxppc-dev@lists.ozlabs.org
> Subject: Re: Problem in getting shared memory access on P1022RDK
> 
> On 01/03/2012 03:42 AM, Arshad, Farrukh wrote:
>> Adding more to it,
>>
>>  
>>
>> When I write from Core 1 on the shared memory region it is visible at 
>> Core 0 and it can read what I have written from Core 1 but when I 
>> write from Core 0 on this shared memory it is not visible on Core 1.
> 
> Is the memory mapped coherent on both cores?
> 
> -Scott

RE: Problem in getting shared memory access on P1022RDK

[Arshad, Farrukh]

Pj4gV2FzIHRoZSBrZXJuZWwgb3B0aW9uLCBDT05GSUdfU01QLCBlbmFibGVkIGZvciBib3RoIHR3
byBrZXJuZWxzPw0KWWVzIENPTkZJR19TTVAgaXMgZW5hYmxlZCBmb3IgYm90aCBrZXJuZWxzLiAN
Cg0KPj4gTWF5YmUgeW91IHNob3VsZCBtYWtlIHN1cmUgaWYga2VybmVsIGhhdmUgYSBhcHByb3By
aWF0ZSBtZW1vcnkgYXR0cmlidXRlIGJ5IGR1bXBpbmcgVExCIGVudHJ5Lg0KVGhhbmtzIGZvciB0
aGUgcG9pbnRlci4gSSB3aWxsIHRyeSB0aGlzIG9uZS4NCg0KUmVnYXJkcywNCkZhcnJ1a2ggQXJz
aGFkDQo=

RE: Problem in getting shared memory access on P1022RDK

[Arshad, Farrukh]

QWRkaW5nIG1vcmUgaXQsIA0KDQpJIGhhdmUgcmVtb3ZlZCB0aGUgc2hhcmVkIG1lbW9yeSBrZXJu
ZWwgZHJpdmVyIGRlcGVuZGVuY3kganVzdCB0byBuYXJyb3cgZG93biB0aGUgcHJvYmxlbSBhcmVh
IGFuZCBJIGhhdmUgd3JpdHRlbiBhIHNtYWxsIHBpZWNlIG9mIGNvZGUgaW4gdXNlciBzcGFjZS4g
QSB3cml0ZXIgJiBhIHJlYWRlciBhcHBsaWNhdGlvbiB3aGljaCBhY2Nlc3MgdGhlIHNoYXJlZCBt
ZW1vcnkgYW5kIEkgZ290IHRoZSBzYW1lIGJlaGF2aW9yIGFzIHdpdGggdGhlIHNoYXJlZCBtZW1v
cnkga2VybmVsIGRyaXZlci4gSW50ZXJlc3RpbmdseSwgbXkgdXNlciBzcGFjZSBhcHBsaWNhdGlv
biB3b3JrIGZpbmUgb24gUDEwMjJEUyBidXQgbm90IG9uIFAxMDIyUkRLIGhvd2V2ZXIgYm90aCB1
c2luZyB0aGUgc2FtZSBDUFUgbW9kdWxlcy4NCg0KV2hlbiBJIHdyaXRlIGEgc2ltcGxlIHN0cmlu
ZyBvbiBzaGFyZWQgbWVtb3J5IGZyb20gQ29yZSAxIGl0IGlzIHJlYWQgYXQgQ29yZSAwIHByb3Bl
cmx5DQpXaGVuIEkgd3JpdGUgYSBzaW1wbGUgc3RyaW5nIG9uIHNoYXJlZCBtZW1vcnkgZnJvbSBD
b3JlIDAgaXQgaXMgbm90IHJlYWQgYXQgQ29yZSAxLiANCg0KV2l0aCB0aGlzIHRlc3Qgbm93IEkg
YW0gc3VyZSB0aGUgcHJvYmxlbSBsaWVzIGluIHRoZSBrZXJuZWwgaXRzZWxmLiBBbnkgcG9pbnRl
cnMgdG8gbG9vayBmb3IgdGhlIHRyb3VibGVkIGFyZWEgPw0KDQpNeSBhcHBsaWNhdGlvbiBjb2Rl
IGlzIChlcnJvciBjaGVja2luZyBhbmQgb3RoZXIgY29kZSBpcyBvbWl0dGVkKQ0KDQojZGVmaW5l
IAlTSE1fQkFTRSAJMHgxQzAwMDAwMA0KI2RlZmluZQlTSE1fU0laRSAJMHg0MDAwMDAJCS8vIDQg
TUIgb2YgU2hhcmVkIE1lbW9yeQ0KI2RlZmluZSAJUEFHRV9TSVpFIAkoNCoxMDI0KQ0KDQpmZCA9
IG9wZW4oZGV2aWNlLCBPX1JEV1IpOw0KDQpzaG0gPSBtYWxsb2MoU0hNX1NJWkUgKyAoUEFHRV9T
SVpFIC0gMSkpOw0KaWYgKCAodW5zaWduZWQgbG9uZykgc2htICUgUEFHRV9TSVpFKSB7DQoJc2ht
ICs9IFBBR0VfU0laRSAtICgodW5zaWduZWQgbG9uZylzaG0gJSBQQUdFX1NJWkUpOw0KfQ0KDQpz
aG0gPSBtbWFwKHNobSwgU0hNX1NJWkUsIFBST1RfUkVBRHxQUk9UX1dSSVRFLCBNQVBfU0hBUkVE
IHwgTUFQX0ZJWEVELCBmZCwgU0hNX0JBU0UpOw0KLi4uLi4uDQouLi4uLi4gd3JpdGUgc29tZSBz
dHJpbmcgYXQgc2htLg0KDQpNeSBtZW1vcnkgcGFydGl0aW9uaW5nIGZvciBib3RoIHN5c3RlbXMg
aXMgDQoNCkNvcmUgICAgICAgICAgICAgICAgIAkJQmFzZSBBZGRyZXNzICAgICAgICAgCVNpemUN
CkNvcmUgMCAgICAgICAgICAgICAgIAkJMHgwMDAwLDAwMDAgICAgICAgICAgCTB4MTAwMCwwMDAw
DQpDb3JlIDEgICAgICAgICAgICAgICAJCTB4MTAwMCwwMDAwICAgICAgICAgIAkweDBDMDAsMDAw
MA0KU2hhcmVkIE1lbW9yeSAgICAJMHgxQzAwLDAwMDAgICAgICAgICAgCTB4MDQwMCwwMDAwDQoN
ClJlZ2FyZHMsDQpGYXJydWtoIEFyc2hhZC4NCk1lbnRvciBHcmFwaGljcyBQYWtpc3Rhbg0K

Re: Problem in getting shared memory access on P1022RDK

[tiejun.chen]

Arshad, Farrukh wrote:
> Adding more it, 
> 
> I have removed the shared memory kernel driver dependency just to narrow down the problem area and I have written a small piece of code in user space. A writer & a reader application which access the shared memory and I got the same behavior as with the shared memory kernel driver. Interestingly, my user space application work fine on P1022DS but not on P1022RDK however both using the same CPU modules.
> 
> When I write a simple string on shared memory from Core 1 it is read at Core 0 properly
> When I write a simple string on shared memory from Core 0 it is not read at Core 1. 
>

Did you dump TLB entry to check page memory coherence attribute for a shared
memory as I mentioned previously? This should be consistent on both sides.

> With this test now I am sure the problem lies in the kernel itself. Any pointers to look for the troubled area ?
> 
> My application code is (error checking and other code is omitted)
> 
> #define 	SHM_BASE 	0x1C000000
> #define	SHM_SIZE 	0x400000		// 4 MB of Shared Memory
> #define 	PAGE_SIZE 	(4*1024)
> 
> fd = open(device, O_RDWR);

You may need to add with 'O_SYNC'.

Tiejun

> 
> shm = malloc(SHM_SIZE + (PAGE_SIZE - 1));
> if ( (unsigned long) shm % PAGE_SIZE) {
> 	shm += PAGE_SIZE - ((unsigned long)shm % PAGE_SIZE);
> }
> 
> shm = mmap(shm, SHM_SIZE, PROT_READ|PROT_WRITE, MAP_SHARED | MAP_FIXED, fd, SHM_BASE);
> ......
> ...... write some string at shm.
> 
> My memory partitioning for both systems is 
> 
> Core                 		Base Address         	Size
> Core 0               		0x0000,0000          	0x1000,0000
> Core 1               		0x1000,0000          	0x0C00,0000
> Shared Memory    	0x1C00,0000          	0x0400,0000
> 
> Regards,
> Farrukh Arshad.
> Mentor Graphics Pakistan

RE: Problem in getting shared memory access on P1022RDK

[Arshad, Farrukh]

SSBoYXZlIGR1bXBlZCBUTEIgZW50cmllcyB3aGlsZSBtYXBwaW5nIHNoYXJlZCBtZW1vcnkuIE9u
IGJvdGggY29yZXMgTS1CaXQgKE1BUzJbNjFdKSBpcyBzZXQgaW4gVExCMCBlbnRyaWVzLiBPbiBi
b3RoIGNvcmVzIE0tQml0IGlzIHNldCBmb3IgYWxsIHZhbGlkIFRMQjEgZW50cmllcy4gVExCMSBk
b2VzIGNvbnRhaW5zIHNvbWUgaW52YWxpZCBlbnRyaWVzIHdoaWNoIGhhcyBNLUJpdCBjbGVhcmVk
LiBTbyBJIGJlbGlldmUgYXQgdGhpcyB0aW1lIHRoZSBjb2hlcmVuY3kgaXMgbm90IHRoZSBpc3N1
ZS4gQW55IGZ1cnRoZXIgdGhvdWdodHMgb24gdGhlIGlzc3VlID8NCg0KSSBoYXZlIG1vZGlmaWVk
IGR1bXBfdGxiX2Jvb2szZSBmdW5jdGlvbiAoZm91bmQgaW4gYXJjaC9wb3dlcnBjL3htb24veG1v
bi5jKSBmdW5jdGlvbiBmb3IgQm9va0UgTU1VIHRvIGR1bXAgVExCIGVudHJpZXMuDQoNClJlZ2Fy
ZHMsDQpGYXJydWtoIEFyc2hhZA0KDQoNCi0tLS0tT3JpZ2luYWwgTWVzc2FnZS0tLS0tDQpGcm9t
OiB0aWVqdW4uY2hlbiBbbWFpbHRvOnRpZWp1bi5jaGVuQHdpbmRyaXZlci5jb21dIA0KU2VudDog
VGh1cnNkYXksIEphbnVhcnkgMTIsIDIwMTIgMTowOSBQTQ0KVG86IEFyc2hhZCwgRmFycnVraA0K
Q2M6IFNjb3R0IFdvb2Q7IGxpbnV4cHBjLWRldkBsaXN0cy5vemxhYnMub3JnDQpTdWJqZWN0OiBS
ZTogUHJvYmxlbSBpbiBnZXR0aW5nIHNoYXJlZCBtZW1vcnkgYWNjZXNzIG9uIFAxMDIyUkRLDQoN
CkFyc2hhZCwgRmFycnVraCB3cm90ZToNCj4gQWRkaW5nIG1vcmUgaXQsDQo+IA0KPiBJIGhhdmUg
cmVtb3ZlZCB0aGUgc2hhcmVkIG1lbW9yeSBrZXJuZWwgZHJpdmVyIGRlcGVuZGVuY3kganVzdCB0
byBuYXJyb3cgZG93biB0aGUgcHJvYmxlbSBhcmVhIGFuZCBJIGhhdmUgd3JpdHRlbiBhIHNtYWxs
IHBpZWNlIG9mIGNvZGUgaW4gdXNlciBzcGFjZS4gQSB3cml0ZXIgJiBhIHJlYWRlciBhcHBsaWNh
dGlvbiB3aGljaCBhY2Nlc3MgdGhlIHNoYXJlZCBtZW1vcnkgYW5kIEkgZ290IHRoZSBzYW1lIGJl
aGF2aW9yIGFzIHdpdGggdGhlIHNoYXJlZCBtZW1vcnkga2VybmVsIGRyaXZlci4gSW50ZXJlc3Rp
bmdseSwgbXkgdXNlciBzcGFjZSBhcHBsaWNhdGlvbiB3b3JrIGZpbmUgb24gUDEwMjJEUyBidXQg
bm90IG9uIFAxMDIyUkRLIGhvd2V2ZXIgYm90aCB1c2luZyB0aGUgc2FtZSBDUFUgbW9kdWxlcy4N
Cj4gDQo+IFdoZW4gSSB3cml0ZSBhIHNpbXBsZSBzdHJpbmcgb24gc2hhcmVkIG1lbW9yeSBmcm9t
IENvcmUgMSBpdCBpcyByZWFkIA0KPiBhdCBDb3JlIDAgcHJvcGVybHkgV2hlbiBJIHdyaXRlIGEg
c2ltcGxlIHN0cmluZyBvbiBzaGFyZWQgbWVtb3J5IGZyb20gQ29yZSAwIGl0IGlzIG5vdCByZWFk
IGF0IENvcmUgMS4NCj4NCg0KRGlkIHlvdSBkdW1wIFRMQiBlbnRyeSB0byBjaGVjayBwYWdlIG1l
bW9yeSBjb2hlcmVuY2UgYXR0cmlidXRlIGZvciBhIHNoYXJlZCBtZW1vcnkgYXMgSSBtZW50aW9u
ZWQgcHJldmlvdXNseT8gVGhpcyBzaG91bGQgYmUgY29uc2lzdGVudCBvbiBib3RoIHNpZGVzLg0K
DQo+IFdpdGggdGhpcyB0ZXN0IG5vdyBJIGFtIHN1cmUgdGhlIHByb2JsZW0gbGllcyBpbiB0aGUg
a2VybmVsIGl0c2VsZi4gQW55IHBvaW50ZXJzIHRvIGxvb2sgZm9yIHRoZSB0cm91YmxlZCBhcmVh
ID8NCj4gDQo+IE15IGFwcGxpY2F0aW9uIGNvZGUgaXMgKGVycm9yIGNoZWNraW5nIGFuZCBvdGhl
ciBjb2RlIGlzIG9taXR0ZWQpDQo+IA0KPiAjZGVmaW5lIAlTSE1fQkFTRSAJMHgxQzAwMDAwMA0K
PiAjZGVmaW5lCVNITV9TSVpFIAkweDQwMDAwMAkJLy8gNCBNQiBvZiBTaGFyZWQgTWVtb3J5DQo+
ICNkZWZpbmUgCVBBR0VfU0laRSAJKDQqMTAyNCkNCj4gDQo+IGZkID0gb3BlbihkZXZpY2UsIE9f
UkRXUik7DQoNCllvdSBtYXkgbmVlZCB0byBhZGQgd2l0aCAnT19TWU5DJy4NCg0KVGllanVuDQoN
Cj4gDQo+IHNobSA9IG1hbGxvYyhTSE1fU0laRSArIChQQUdFX1NJWkUgLSAxKSk7IGlmICggKHVu
c2lnbmVkIGxvbmcpIHNobSAlIA0KPiBQQUdFX1NJWkUpIHsNCj4gCXNobSArPSBQQUdFX1NJWkUg
LSAoKHVuc2lnbmVkIGxvbmcpc2htICUgUEFHRV9TSVpFKTsgfQ0KPiANCj4gc2htID0gbW1hcChz
aG0sIFNITV9TSVpFLCBQUk9UX1JFQUR8UFJPVF9XUklURSwgTUFQX1NIQVJFRCB8IA0KPiBNQVBf
RklYRUQsIGZkLCBTSE1fQkFTRSk7IC4uLi4uLg0KPiAuLi4uLi4gd3JpdGUgc29tZSBzdHJpbmcg
YXQgc2htLg0KPiANCj4gTXkgbWVtb3J5IHBhcnRpdGlvbmluZyBmb3IgYm90aCBzeXN0ZW1zIGlz
DQo+IA0KPiBDb3JlICAgICAgICAgICAgICAgICAJCUJhc2UgQWRkcmVzcyAgICAgICAgIAlTaXpl
DQo+IENvcmUgMCAgICAgICAgICAgICAgIAkJMHgwMDAwLDAwMDAgICAgICAgICAgCTB4MTAwMCww
MDAwDQo+IENvcmUgMSAgICAgICAgICAgICAgIAkJMHgxMDAwLDAwMDAgICAgICAgICAgCTB4MEMw
MCwwMDAwDQo+IFNoYXJlZCBNZW1vcnkgICAgCTB4MUMwMCwwMDAwICAgICAgICAgIAkweDA0MDAs
MDAwMA0KPiANCj4gUmVnYXJkcywNCj4gRmFycnVraCBBcnNoYWQuDQo+IE1lbnRvciBHcmFwaGlj
cyBQYWtpc3Rhbg0KDQo=

Re: Problem in getting shared memory access on P1022RDK

[tiejun.chen]

Arshad, Farrukh wrote:
> I have dumped TLB entries while mapping shared memory. On both cores M-Bit (MAS2[61]) is set in TLB0 entries. On both cores M-Bit is set for all valid TLB1 entries. TLB1 does contains some invalid entries which has M-Bit cleared. So I believe at this time the coherency is not the issue. Any further thoughts on the issue ?
>

Did you check all associated TLBx VPN/RPN and attribute setting are same between
two scenarios: W -> Core0 & R <- Core1 and W -> Core1 & R <- Core0?

Can you send me your TLB dump log separately if possible?

And did you try this flag 'O_SYNC'?

Tiejun

> I have modified dump_tlb_book3e function (found in arch/powerpc/xmon/xmon.c) function for BookE MMU to dump TLB entries.
> 
> Regards,
> Farrukh Arshad
> 
> 
> -----Original Message-----
> From: tiejun.chen [mailto:tiejun.chen@windriver.com] 
> Sent: Thursday, January 12, 2012 1:09 PM
> To: Arshad, Farrukh
> Cc: Scott Wood; linuxppc-dev@lists.ozlabs.org
> Subject: Re: Problem in getting shared memory access on P1022RDK
> 
> Arshad, Farrukh wrote:
>> Adding more it,
>>
>> I have removed the shared memory kernel driver dependency just to narrow down the problem area and I have written a small piece of code in user space. A writer & a reader application which access the shared memory and I got the same behavior as with the shared memory kernel driver. Interestingly, my user space application work fine on P1022DS but not on P1022RDK however both using the same CPU modules.
>>
>> When I write a simple string on shared memory from Core 1 it is read 
>> at Core 0 properly When I write a simple string on shared memory from Core 0 it is not read at Core 1.
>>
> 
> Did you dump TLB entry to check page memory coherence attribute for a shared memory as I mentioned previously? This should be consistent on both sides.
> 
>> With this test now I am sure the problem lies in the kernel itself. Any pointers to look for the troubled area ?
>>
>> My application code is (error checking and other code is omitted)
>>
>> #define 	SHM_BASE 	0x1C000000
>> #define	SHM_SIZE 	0x400000		// 4 MB of Shared Memory
>> #define 	PAGE_SIZE 	(4*1024)
>>
>> fd = open(device, O_RDWR);
> 
> You may need to add with 'O_SYNC'.
> 
> Tiejun
> 
>> shm = malloc(SHM_SIZE + (PAGE_SIZE - 1)); if ( (unsigned long) shm % 
>> PAGE_SIZE) {
>> 	shm += PAGE_SIZE - ((unsigned long)shm % PAGE_SIZE); }
>>
>> shm = mmap(shm, SHM_SIZE, PROT_READ|PROT_WRITE, MAP_SHARED | 
>> MAP_FIXED, fd, SHM_BASE); ......
>> ...... write some string at shm.
>>
>> My memory partitioning for both systems is
>>
>> Core                 		Base Address         	Size
>> Core 0               		0x0000,0000          	0x1000,0000
>> Core 1               		0x1000,0000          	0x0C00,0000
>> Shared Memory    	0x1C00,0000          	0x0400,0000
>>
>> Regards,
>> Farrukh Arshad.
>> Mentor Graphics Pakistan
>