parisc: fix mmap(MAP_FIXED|MAP_SHARED) to already mmapped address

parisc: fix mmap(MAP_FIXED|MAP_SHARED) to already mmapped address

[Aaro Koskinen]

Hi,

This commit (0576da2c08e3d332f1b0653030d28ab804585ab6) and the current
mainline kernel (3.13-rc4) gives me the following with GLIBC 2.18:

$ localedef -c -i en_US -f UTF-8 en_US.UTF-8
cannot map archive header: Invalid argument

strace looks like this:

mmap2(NULL, 536870912, PROT_NONE, MAP_SHARED, 3, 0) = 0x42f34000
mmap2(0x43000000, 1607632, PROT_READ|PROT_WRITE, MAP_SHARED|MAP_FIXED, 3, 0) = -1 EINVAL (Invalid argument)

With the patch reverted, it works:

mmap2(NULL, 536870912, PROT_NONE, MAP_SHARED, 3, 0) = 0x42d74000
mmap2(0x43000000, 1607632, PROT_READ|PROT_WRITE, MAP_SHARED|MAP_FIXED, 3, 0) = 0x43000000

BTW, note that for GLIBC 2.18 some changes were done regarding this:
https://sourceware.org/bugzilla/show_bug.cgi?id=10283

A.

Re: parisc: fix mmap(MAP_FIXED|MAP_SHARED) to already mmapped address

[John David Anglin]

On 12/19/2013 2:17 PM, Aaro Koskinen wrote:
> Hi,
>
> This commit (0576da2c08e3d332f1b0653030d28ab804585ab6) and the current
> mainline kernel (3.13-rc4) gives me the following with GLIBC 2.18:
>
> $ localedef -c -i en_US -f UTF-8 en_US.UTF-8
> cannot map archive header: Invalid argument
>
> strace looks like this:
>
> mmap2(NULL, 536870912, PROT_NONE, MAP_SHARED, 3, 0) = 0x42f34000
> mmap2(0x43000000, 1607632, PROT_READ|PROT_WRITE, MAP_SHARED|MAP_FIXED, 3, 0) = -1 EINVAL (Invalid argument)
>
> With the patch reverted, it works:
>
> mmap2(NULL, 536870912, PROT_NONE, MAP_SHARED, 3, 0) = 0x42d74000
> mmap2(0x43000000, 1607632, PROT_READ|PROT_WRITE, MAP_SHARED|MAP_FIXED, 3, 0) = 0x43000000
>
> BTW, note that for GLIBC 2.18 some changes were done regarding this:
> https://sourceware.org/bugzilla/show_bug.cgi?id=10283
Are you sure the glibc changes are correct?  PARISC addresses are not 
strictly aligned to SHMLBA.
There's also a page offset and "random" offset derived from the kernel 
address of the mapping struct.

Helge is still looking at this to try and improve the allocation density 
of small maps as we run out of
memory far too quickly.

Dave

-- 
John David Anglin    dave.anglin@bell.net

Re: parisc: fix mmap(MAP_FIXED|MAP_SHARED) to already mmapped address

[Aaro Koskinen]

Hi,

On Thu, Dec 19, 2013 at 02:44:40PM -0500, John David Anglin wrote:
> On 12/19/2013 2:17 PM, Aaro Koskinen wrote:
> >This commit (0576da2c08e3d332f1b0653030d28ab804585ab6) and the current
> >mainline kernel (3.13-rc4) gives me the following with GLIBC 2.18:
> >
> >$ localedef -c -i en_US -f UTF-8 en_US.UTF-8
> >cannot map archive header: Invalid argument
> >
> >strace looks like this:
> >
> >mmap2(NULL, 536870912, PROT_NONE, MAP_SHARED, 3, 0) = 0x42f34000
> >mmap2(0x43000000, 1607632, PROT_READ|PROT_WRITE, MAP_SHARED|MAP_FIXED, 3, 0) = -1 EINVAL (Invalid argument)
> >
> >With the patch reverted, it works:
> >
> >mmap2(NULL, 536870912, PROT_NONE, MAP_SHARED, 3, 0) = 0x42d74000
> >mmap2(0x43000000, 1607632, PROT_READ|PROT_WRITE, MAP_SHARED|MAP_FIXED, 3, 0) = 0x43000000
> >
> >BTW, note that for GLIBC 2.18 some changes were done regarding this:
> >https://sourceware.org/bugzilla/show_bug.cgi?id=10283
> Are you sure the glibc changes are correct?  PARISC addresses are
> not strictly aligned to SHMLBA.

No, I'm not sure. I just wanted to report this issue since I see it on
my box and there is a recent kernel change that seems to be somewhat
related. Of course if the glibc is broken we should try to fix that.

A.

Re: parisc: fix mmap(MAP_FIXED|MAP_SHARED) to already mmapped address

[Mike Frysinger]

[-- Attachment #1: Type: Text/Plain, Size: 1581 bytes --]

On Thursday 19 December 2013 14:44:40 John David Anglin wrote:
> On 12/19/2013 2:17 PM, Aaro Koskinen wrote:
> > This commit (0576da2c08e3d332f1b0653030d28ab804585ab6) and the current
> > mainline kernel (3.13-rc4) gives me the following with GLIBC 2.18:
> > 
> > $ localedef -c -i en_US -f UTF-8 en_US.UTF-8
> > cannot map archive header: Invalid argument
> > 
> > strace looks like this:
> > 
> > mmap2(NULL, 536870912, PROT_NONE, MAP_SHARED, 3, 0) = 0x42f34000
> > mmap2(0x43000000, 1607632, PROT_READ|PROT_WRITE, MAP_SHARED|MAP_FIXED, 3,
> > 0) = -1 EINVAL (Invalid argument)
> > 
> > With the patch reverted, it works:
> > 
> > mmap2(NULL, 536870912, PROT_NONE, MAP_SHARED, 3, 0) = 0x42d74000
> > mmap2(0x43000000, 1607632, PROT_READ|PROT_WRITE, MAP_SHARED|MAP_FIXED, 3,
> > 0) = 0x43000000
> > 
> > BTW, note that for GLIBC 2.18 some changes were done regarding this:
> > https://sourceware.org/bugzilla/show_bug.cgi?id=10283
> 
> Are you sure the glibc changes are correct?  PARISC addresses are not
> strictly aligned to SHMLBA.
> There's also a page offset and "random" offset derived from the kernel
> address of the mapping struct.

the glibc changes were to use SHMLBA as the min alignment where as before it 
was using PAGE_SIZE.  the kernel shouldn't reject us because we manually 
increased our alignment ... the mappings should still be overlapping, so it 
shouldn't be an "out of mapping space" issue.

the first one gets a really large map (not fixed), and then the second does a 
small mapping inside of that first one.
-mike

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

Re: parisc: fix mmap(MAP_FIXED|MAP_SHARED) to already mmapped address

[John David Anglin]

On 12/19/2013 4:19 PM, Mike Frysinger wrote:
> On Thursday 19 December 2013 14:44:40 John David Anglin wrote:
>> On 12/19/2013 2:17 PM, Aaro Koskinen wrote:
>>> This commit (0576da2c08e3d332f1b0653030d28ab804585ab6) and the current
>>> mainline kernel (3.13-rc4) gives me the following with GLIBC 2.18:
>>>
>>> $ localedef -c -i en_US -f UTF-8 en_US.UTF-8
>>> cannot map archive header: Invalid argument
>>>
>>> strace looks like this:
>>>
>>> mmap2(NULL, 536870912, PROT_NONE, MAP_SHARED, 3, 0) = 0x42f34000
>>> mmap2(0x43000000, 1607632, PROT_READ|PROT_WRITE, MAP_SHARED|MAP_FIXED, 3,
>>> 0) = -1 EINVAL (Invalid argument)
>>>
>>> With the patch reverted, it works:
>>>
>>> mmap2(NULL, 536870912, PROT_NONE, MAP_SHARED, 3, 0) = 0x42d74000
>>> mmap2(0x43000000, 1607632, PROT_READ|PROT_WRITE, MAP_SHARED|MAP_FIXED, 3,
>>> 0) = 0x43000000
>>>
>>> BTW, note that for GLIBC 2.18 some changes were done regarding this:
>>> https://sourceware.org/bugzilla/show_bug.cgi?id=10283
>> Are you sure the glibc changes are correct?  PARISC addresses are not
>> strictly aligned to SHMLBA.
>> There's also a page offset and "random" offset derived from the kernel
>> address of the mapping struct.
> the glibc changes were to use SHMLBA as the min alignment where as before it
> was using PAGE_SIZE.  the kernel shouldn't reject us because we manually
> increased our alignment ... the mappings should still be overlapping, so it
> shouldn't be an "out of mapping space" issue.
>
> the first one gets a really large map (not fixed), and then the second does a
> small mapping inside of that first one.
> -mike
The mmap check is here:

         if (flags & MAP_FIXED) {
                 if ((flags & MAP_SHARED) &&
                     (addr - shared_align_offset(filp, pgoff)) & (SHMLBA 
- 1))
                         return -EINVAL;
                 return addr;
         }

In the case at hand, the addr value passed is not equivalent to the value
returned on first mmap2 call.  So, the error is correct.  The 
application has to
preserve the "SHMLBA - 1" part of the address when selecting a new map 
address.

The error is EINVAL (i.e., bad address).

The old code wouldn't have returned an error but the second mapping 
would not
have been equivalent to the first.

My head spins regarding mappings inside mappings as it seems they may 
conflict.

Dave

-- 
John David Anglin    dave.anglin@bell.net

Re: parisc: fix mmap(MAP_FIXED|MAP_SHARED) to already mmapped address

[Mike Frysinger]

[-- Attachment #1: Type: Text/Plain, Size: 3155 bytes --]

On Thursday 19 December 2013 17:38:37 John David Anglin wrote:
> On 12/19/2013 4:19 PM, Mike Frysinger wrote:
> > On Thursday 19 December 2013 14:44:40 John David Anglin wrote:
> >> On 12/19/2013 2:17 PM, Aaro Koskinen wrote:
> >>> This commit (0576da2c08e3d332f1b0653030d28ab804585ab6) and the current
> >>> mainline kernel (3.13-rc4) gives me the following with GLIBC 2.18:
> >>> 
> >>> $ localedef -c -i en_US -f UTF-8 en_US.UTF-8
> >>> cannot map archive header: Invalid argument
> >>> 
> >>> strace looks like this:
> >>> 
> >>> mmap2(NULL, 536870912, PROT_NONE, MAP_SHARED, 3, 0) = 0x42f34000
> >>> mmap2(0x43000000, 1607632, PROT_READ|PROT_WRITE, MAP_SHARED|MAP_FIXED,
> >>> 3, 0) = -1 EINVAL (Invalid argument)
> >>> 
> >>> With the patch reverted, it works:
> >>> 
> >>> mmap2(NULL, 536870912, PROT_NONE, MAP_SHARED, 3, 0) = 0x42d74000
> >>> mmap2(0x43000000, 1607632, PROT_READ|PROT_WRITE, MAP_SHARED|MAP_FIXED,
> >>> 3, 0) = 0x43000000
> >>> 
> >>> BTW, note that for GLIBC 2.18 some changes were done regarding this:
> >>> https://sourceware.org/bugzilla/show_bug.cgi?id=10283
> >> 
> >> Are you sure the glibc changes are correct?  PARISC addresses are not
> >> strictly aligned to SHMLBA.
> >> There's also a page offset and "random" offset derived from the kernel
> >> address of the mapping struct.
> > 
> > the glibc changes were to use SHMLBA as the min alignment where as before
> > it was using PAGE_SIZE.  the kernel shouldn't reject us because we
> > manually increased our alignment ... the mappings should still be
> > overlapping, so it shouldn't be an "out of mapping space" issue.
> > 
> > the first one gets a really large map (not fixed), and then the second
> > does a small mapping inside of that first one.
> 
> The mmap check is here:
> 
>          if (flags & MAP_FIXED) {
>                  if ((flags & MAP_SHARED) &&
>                      (addr - shared_align_offset(filp, pgoff)) & (SHMLBA
> - 1))
>                          return -EINVAL;
>                  return addr;
>          }
> 
> In the case at hand, the addr value passed is not equivalent to the value
> returned on first mmap2 call.  So, the error is correct.  The
> application has to
> preserve the "SHMLBA - 1" part of the address when selecting a new map
> address.
> 
> The error is EINVAL (i.e., bad address).
> 
> The old code wouldn't have returned an error but the second mapping
> would not
> have been equivalent to the first.
> 
> My head spins regarding mappings inside mappings as it seems they may
> conflict.

we designed the new locale code to work on all systems (including fixing the 
behavior on older parisc64 kernels).  the assumption is that, if we want to do 
overlapping mappings, we need to use SHMLBA as the min alignment (since that's 
what the define represents -- shared mapping alignments that the hardware is 
restricted by).  and that is what the new locale code is doing.  glibc defines 
SHMLBA to 0x00400000 for parisc which is why 0x42d74000 turned into 
0x43000000.

so either SHMLBA on parisc is wrong, or the new kernel code is wrong
-mike

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

Re: parisc: fix mmap(MAP_FIXED|MAP_SHARED) to already mmapped address

[Helge Deller]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Hi all,

On 12/20/2013 12:02 AM, Mike Frysinger wrote:
> On Thursday 19 December 2013 17:38:37 John David Anglin wrote:
>> On 12/19/2013 4:19 PM, Mike Frysinger wrote:
>>> On Thursday 19 December 2013 14:44:40 John David Anglin wrote:
>>>> On 12/19/2013 2:17 PM, Aaro Koskinen wrote:
>>>>> This commit (0576da2c08e3d332f1b0653030d28ab804585ab6) and the current
>>>>> mainline kernel (3.13-rc4) gives me the following with GLIBC 2.18:
>>>>>
>>>>> $ localedef -c -i en_US -f UTF-8 en_US.UTF-8
>>>>> cannot map archive header: Invalid argument
>>>>>
>>>>> strace looks like this:
>>>>>
>>>>> mmap2(NULL, 536870912, PROT_NONE, MAP_SHARED, 3, 0) = 0x42f34000
>>>>> mmap2(0x43000000, 1607632, PROT_READ|PROT_WRITE, MAP_SHARED|MAP_FIXED, 3, 0) = -1 EINVAL (Invalid argument)
>>>>>
>>>>> With the patch reverted, it works:
>>>>>
>>>>> mmap2(NULL, 536870912, PROT_NONE, MAP_SHARED, 3, 0) = 0x42d74000
>>>>> mmap2(0x43000000, 1607632, PROT_READ|PROT_WRITE, MAP_SHARED|MAP_FIXED, 3, 0) = 0x43000000
>>>>>
>>>>> BTW, note that for GLIBC 2.18 some changes were done regarding this:
>>>>> https://sourceware.org/bugzilla/show_bug.cgi?id=10283
>>>>
>>>> Are you sure the glibc changes are correct?  PARISC addresses are not
>>>> strictly aligned to SHMLBA.
>>>> There's also a page offset and "random" offset derived from the kernel
>>>> address of the mapping struct.

Yes, that's the important part: The "random" offset stays constant for each file (mapping)!

>>> the glibc changes were to use SHMLBA as the min alignment where as before
>>> it was using PAGE_SIZE.  the kernel shouldn't reject us because we
>>> manually increased our alignment ... the mappings should still be
>>> overlapping, so it shouldn't be an "out of mapping space" issue.
>>>
>>> the first one gets a really large map (not fixed), and then the second
>>> does a small mapping inside of that first one.
>>
>> The mmap check is here:
>>
>>          if (flags & MAP_FIXED) {
>>                  if ((flags & MAP_SHARED) &&
>>                      (addr - shared_align_offset(filp, pgoff)) & (SHMLBA
>> - 1))
>>                          return -EINVAL;
>>                  return addr;
>>          }
>>
>> In the case at hand, the addr value passed is not equivalent to the value
>> returned on first mmap2 call.  So, the error is correct.  The
>> application has to
>> preserve the "SHMLBA - 1" part of the address when selecting a new map
>> address.
>>
>> The error is EINVAL (i.e., bad address).
>>
>> The old code wouldn't have returned an error but the second mapping
>> would not
>> have been equivalent to the first.
>>
>> My head spins regarding mappings inside mappings as it seems they may
>> conflict.
> 
> we designed the new locale code to work on all systems (including fixing the 
> behavior on older parisc64 kernels).  the assumption is that, if we want to do 
> overlapping mappings, 

What you understand with "overlapping mappings" ?
Overlapping mappings for parisc means, that the bytes inside the file needs to be mapped at
exactly the same addresses (modulo SHMLBA). 

With the example above, it means for parisc:
mmap2(NULL, 536870912, PROT_NONE, MAP_SHARED, 3, 0) = 0x42f34000
- -> OK.

mmap2(0x43000000, 1607632, PROT_READ|PROT_WRITE, MAP_SHARED|MAP_FIXED, 3, 0) = -1 EINVAL
- -> This is strange.
Now you map the "beginning of the file" (offset=0) at *another* address (modulo SHMLBA) than what the
first mmap gave you. I agree that 0x43000000 is "inside" the memory range which you originally mapped
with the large mapping, but that's not relevant for parisc at all.

Instead this should work:
mmap2(NULL, 536870912, PROT_NONE, MAP_SHARED, 3, 0) = 0x42f34000
some_address = x * SHMLBA; /* some multiple of SHMLBA, e.g. 0x43000000 */
file_offset = some_address - 0x42f34000;
mmap2(some_address, 1607632, PROT_READ|PROT_WRITE, MAP_SHARED|MAP_FIXED, 3, file_offset) = -1 EINVAL

Not sure if this is what you wanted.
Anyway, it looks like we need to change glibc for that...

> we need to use SHMLBA as the min alignment (since that's 
> what the define represents -- shared mapping alignments that the hardware is 
> restricted by).  and that is what the new locale code is doing.  glibc defines 
> SHMLBA to 0x00400000 for parisc which is why 0x42d74000 turned into 
> 0x43000000.

Sounds like logically correct behavior.

> so either SHMLBA on parisc is wrong, or the new kernel code is wrong

SHMLBA on parisc is correct.
The new kernel code is correct too, and the old one was definitively wrong.

You could try my current rewrite for the mmap functions which is in my "aio_mmap_fix" branch at
git://git.kernel.org/pub/scm/linux/kernel/git/deller/parisc-linux.git
(http://git.kernel.org/cgit/linux/kernel/git/deller/parisc-linux.git/log/?h=aio_mmap_fix)
Pulling this branch on top of Linux head should work.
But I think it will not solve this problem.

Helge

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.22 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iQIcBAEBAgAGBQJStMA7AAoJEKGDlV8wpRJBwugP/j7e6z+kS6bymrpuxYEBpE/n
c+yzgxjcaVwDL+KMln9cP2Zx6UAnTwYIXv8bmOmUA65FB9ttaqP73t/l8wIgjXai
BGC/951hPgyfI2Kxkvy1R32dsbBVNyIRNQfp/MADRS8xC059nlTIXib9NduaiKLg
Gp6YA2S6d24EdKooVfVRPh5c6sThePgvjQ6kmytxxOh8uOIdwLG2ZXGs4KC/e07B
1VFTH10ecijTUnqzl8awkw2OqO3VzllGJ2kCwZo2Zynpd58FUOu1UQSWWTsk4F08
owU8HfrEGVSSc7urrsIs2na8eXJ/jq44ksmfkDtGmKt4Ati7ShT6dMZuFgXRcluY
E6PaPrTMLW9nASBSv0FDfcR+BT1Ffghl0yEZ3rmYpnH+ExcotFWfyb/an/93AM2l
vPgHy0+SFMYxgsV0rNDU+qJajf/2Cpwz3At/iQDQRFjWaL7zyu8RNJ7BMc1Dml7Y
HodYc4bpP4ucM/RIGMTnJdw35mwQdhnfXOyCFJoUR5gPxCVKso+4cz32H+yK2rdp
2MpBke0j7AHbI/AWvrqsocqVuLKt6BSMos/yFTqbvFmoBqaAVIWP0dulxsl40Otu
y5LFh36mwcVcJftJ/2Dftlw+bSYEVeZbA2l+XU99Drgaod9ifHbt13FRdkRTR2um
q1m48iZVn0YPGEXcFO81
=ObSL
-----END PGP SIGNATURE-----

Re: parisc: fix mmap(MAP_FIXED|MAP_SHARED) to already mmapped address

[Aaro Koskinen]

Hi,

On Fri, Dec 20, 2013 at 11:10:03PM +0100, Helge Deller wrote:
> You could try my current rewrite for the mmap functions which is in my "aio_mmap_fix" branch at
> git://git.kernel.org/pub/scm/linux/kernel/git/deller/parisc-linux.git
> (http://git.kernel.org/cgit/linux/kernel/git/deller/parisc-linux.git/log/?h=aio_mmap_fix)
> Pulling this branch on top of Linux head should work.
> But I think it will not solve this problem.

FWIW, I pulled this branch (aaa88432c03b) on top of 3.13-rc5 and the
system won't boot (HP C3700, 32-bit kernel). There's several stack trace
dumps during the boot; if you're interested I might be able to setup a
serial console to capture these.

A.

Re: parisc: fix mmap(MAP_FIXED|MAP_SHARED) to already mmapped address

[Helge Deller]

Hi Aaro,

On 12/23/2013 09:26 PM, Aaro Koskinen wrote:
> On Fri, Dec 20, 2013 at 11:10:03PM +0100, Helge Deller wrote:
>> You could try my current rewrite for the mmap functions which is in my "aio_mmap_fix" branch at
>> git://git.kernel.org/pub/scm/linux/kernel/git/deller/parisc-linux.git
>> (http://git.kernel.org/cgit/linux/kernel/git/deller/parisc-linux.git/log/?h=aio_mmap_fix)
>> Pulling this branch on top of Linux head should work.
>> But I think it will not solve this problem.
> 
> FWIW, I pulled this branch (aaa88432c03b) on top of 3.13-rc5 and the
> system won't boot (HP C3700, 32-bit kernel). There's several stack trace
> dumps during the boot; if you're interested I might be able to setup a
> serial console to capture these.

Yes, the code hit a BUG_ON when running with 32bit kernel.
I just committed a trivial patch to the same tree, and now it works.

Helge 

Re: parisc: fix mmap(MAP_FIXED|MAP_SHARED) to already mmapped address

[Aaro Koskinen]

Hi,

On Sun, Dec 29, 2013 at 09:50:56PM +0100, Helge Deller wrote:
> On 12/23/2013 09:26 PM, Aaro Koskinen wrote:
> > On Fri, Dec 20, 2013 at 11:10:03PM +0100, Helge Deller wrote:
> >> You could try my current rewrite for the mmap functions which is in my "aio_mmap_fix" branch at
> >> git://git.kernel.org/pub/scm/linux/kernel/git/deller/parisc-linux.git
> >> (http://git.kernel.org/cgit/linux/kernel/git/deller/parisc-linux.git/log/?h=aio_mmap_fix)
> >> Pulling this branch on top of Linux head should work.
> >> But I think it will not solve this problem.
> > 
> > FWIW, I pulled this branch (aaa88432c03b) on top of 3.13-rc5 and the
> > system won't boot (HP C3700, 32-bit kernel). There's several stack trace
> > dumps during the boot; if you're interested I might be able to setup a
> > serial console to capture these.
> 
> Yes, the code hit a BUG_ON when running with 32bit kernel.
> I just committed a trivial patch to the same tree, and now it works.

Yes, it works now, thanks. localedef still fails, but I guess that was
expected based on what you said:

mmap2(NULL, 536870912, PROT_NONE, MAP_SHARED, 3, 0) = 0xd83ee000
mmap2(0xd8400000, 103860, PROT_READ|PROT_WRITE, MAP_SHARED|MAP_FIXED, 3, 0) = -1 EINVAL (Invalid argument)

A.

Re: parisc: fix mmap(MAP_FIXED|MAP_SHARED) to already mmapped address

[John David Anglin]

This is a resend of a comment previously sent (hit wrong button and it  
only went to Mike).

On 19-Dec-13, at 6:02 PM, Mike Frysinger wrote:

> On Thursday 19 December 2013 17:38:37 John David Anglin wrote:
>> On 12/19/2013 4:19 PM, Mike Frysinger wrote:
>>> On Thursday 19 December 2013 14:44:40 John David Anglin wrote:
>>>> On 12/19/2013 2:17 PM, Aaro Koskinen wrote:
>>>>> This commit (0576da2c08e3d332f1b0653030d28ab804585ab6) and the  
>>>>> current
>>>>> mainline kernel (3.13-rc4) gives me the following with GLIBC 2.18:
>>>>>
>>>>> $ localedef -c -i en_US -f UTF-8 en_US.UTF-8
>>>>> cannot map archive header: Invalid argument
>>>>>
>>>>> strace looks like this:
>>>>>
>>>>> mmap2(NULL, 536870912, PROT_NONE, MAP_SHARED, 3, 0) = 0x42f34000
>>>>> mmap2(0x43000000, 1607632, PROT_READ|PROT_WRITE, MAP_SHARED| 
>>>>> MAP_FIXED,
>>>>> 3, 0) = -1 EINVAL (Invalid argument)
>>>>>
>>>>> With the patch reverted, it works:
>>>>>
>>>>> mmap2(NULL, 536870912, PROT_NONE, MAP_SHARED, 3, 0) = 0x42d74000
>>>>> mmap2(0x43000000, 1607632, PROT_READ|PROT_WRITE, MAP_SHARED| 
>>>>> MAP_FIXED,
>>>>> 3, 0) = 0x43000000
>>>>>
>>>>> BTW, note that for GLIBC 2.18 some changes were done regarding  
>>>>> this:
>>>>> https://sourceware.org/bugzilla/show_bug.cgi?id=10283
>>>>
>>>> Are you sure the glibc changes are correct?  PARISC addresses are  
>>>> not
>>>> strictly aligned to SHMLBA.
>>>> There's also a page offset and "random" offset derived from the  
>>>> kernel
>>>> address of the mapping struct.
>>>
>>> the glibc changes were to use SHMLBA as the min alignment where as  
>>> before
>>> it was using PAGE_SIZE.  the kernel shouldn't reject us because we
>>> manually increased our alignment ... the mappings should still be
>>> overlapping, so it shouldn't be an "out of mapping space" issue.
>>>
>>> the first one gets a really large map (not fixed), and then the  
>>> second
>>> does a small mapping inside of that first one.
>>
>> The mmap check is here:
>>
>>         if (flags & MAP_FIXED) {
>>                 if ((flags & MAP_SHARED) &&
>>                     (addr - shared_align_offset(filp, pgoff)) &  
>> (SHMLBA
>> - 1))
>>                         return -EINVAL;
>>                 return addr;
>>         }
>>
>> In the case at hand, the addr value passed is not equivalent to the  
>> value
>> returned on first mmap2 call.  So, the error is correct.  The
>> application has to
>> preserve the "SHMLBA - 1" part of the address when selecting a new  
>> map
>> address.
>>
>> The error is EINVAL (i.e., bad address).
>>
>> The old code wouldn't have returned an error but the second mapping
>> would not
>> have been equivalent to the first.
>>
>> My head spins regarding mappings inside mappings as it seems they may
>> conflict.
>
> we designed the new locale code to work on all systems (including  
> fixing the
> behavior on older parisc64 kernels).  the assumption is that, if we  
> want to do
> overlapping mappings, we need to use SHMLBA as the min alignment  
> (since that's
> what the define represents -- shared mapping alignments that the  
> hardware is
> restricted by).  and that is what the new locale code is doing.   
> glibc defines
> SHMLBA to 0x00400000 for parisc which is why 0x42d74000 turned into
> 0x43000000.
>
> so either SHMLBA on parisc is wrong, or the new kernel code is wrong
> -mike


I think there is a misunderstanding regarding the parisc alignment  
requirements.

Mappings don't have have to start on a SHMLBA boundary and they never  
have.  Equivalent mappings
have to be offset by an integer number of SHMLBA blocks.  This allows  
more than one mapping
in a SHMLBA block depending on size.

Two address are equivalent if:

	(addr1 & (SHMLBA - 1)) == (addr2 & (SHMLBA - 1)

See flush_dcache_page() implementation.

The value for SHMLBA is certainly ok for all parisc machines.

The align mask for parisc shared areas is "PAGE_MASK & (SHMLBA - 1)".
The align offset is "(get_offset(mapping) + pgoff) << PAGE_SHIFT".

I believe glibc is not taking into account the offset resulting from  
get_offset(mapping).  Some
other archs use random offsets.

Dave
--
John David Anglin	dave.anglin@bell.net

Re: parisc: fix mmap(MAP_FIXED|MAP_SHARED) to already mmapped address

[Helge Deller]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Hi Mike,

On 12/19/2013 10:19 PM, Mike Frysinger wrote:
> On Thursday 19 December 2013 14:44:40 John David Anglin wrote:
>> On 12/19/2013 2:17 PM, Aaro Koskinen wrote:
>>> This commit (0576da2c08e3d332f1b0653030d28ab804585ab6) and the current
>>> mainline kernel (3.13-rc4) gives me the following with GLIBC 2.18:
>>>
>>> $ localedef -c -i en_US -f UTF-8 en_US.UTF-8
>>> cannot map archive header: Invalid argument
>>>
>>> strace looks like this:
>>>
>>> mmap2(NULL, 536870912, PROT_NONE, MAP_SHARED, 3, 0) = 0x42f34000
>>> mmap2(0x43000000, 1607632, PROT_READ|PROT_WRITE, MAP_SHARED|MAP_FIXED, 3,
>>> 0) = -1 EINVAL (Invalid argument)
>>>
>>> With the patch reverted, it works:
>>>
>>> mmap2(NULL, 536870912, PROT_NONE, MAP_SHARED, 3, 0) = 0x42d74000
>>> mmap2(0x43000000, 1607632, PROT_READ|PROT_WRITE, MAP_SHARED|MAP_FIXED, 3,
>>> 0) = 0x43000000
>>>
>>> BTW, note that for GLIBC 2.18 some changes were done regarding this:
>>> https://sourceware.org/bugzilla/show_bug.cgi?id=10283
>>
>> Are you sure the glibc changes are correct?  PARISC addresses are not
>> strictly aligned to SHMLBA.
>> There's also a page offset and "random" offset derived from the kernel
>> address of the mapping struct.
> 
> the glibc changes were to use SHMLBA as the min alignment where as before it 
> was using PAGE_SIZE.  the kernel shouldn't reject us because we manually 
> increased our alignment ... the mappings should still be overlapping, so it 
> shouldn't be an "out of mapping space" issue.
> 
> the first one gets a really large map (not fixed), and then the second does a 
> small mapping inside of that first one.

This doesn't work for parisc.
We currently face this issue again now with eglibc on debian since we updated it.

Regarding this commit:
https://sourceware.org/git/?p=glibc.git;a=blob;f=sysdeps/generic/libc-mmap.h;h=0ddd20d42ab44282180c6d6f7c13f2875b25a614;hb=17db6e8d6b12f55e312fcab46faf5d332c806fb6

a trivial workaround would be to change libc-mmap.h like this:
#ifdef __hppa__
#define MAP_FIXED_ALIGNMENT 4096
#else
#define MAP_FIXED_ALIGNMENT SHMLBA
#endif

That works because then the new aligned address is then the same as the original
(the mmap call returns 4k aligned addresses, so it stays unchanged), but I'm not sure
if such a patch would be acceptable.
Do you have another idea/proposal?

Helge
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.22 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iQIcBAEBAgAGBQJTE6EuAAoJEKGDlV8wpRJBn1sP/2rgG74cYxazzfljoAR6Ymdc
lwgbJqKthEMpNyaFKqxEJrcgwdhBLabVPOe9NDXyS7GpFp+0NPi+EbFCdVRNEO6c
3oWV4XkiZBm2ebOI90J5YiKZN0GwrsLE9lEJD/db7JYSFdV34SbAsKOMth8vgL9K
unVtsU1fGNFOrPuND7+tclxuClsD/0zZqYpW5V6l3/Iox3rrjceSemxw5ZnlN7Eq
CnLBAw0UBbZYVYPiy1hYzFndrAdOGt5SDatM7WTHZ0AiaqZfyKpGE7/qwImjCJt0
KL9BXYcEoT+neAAdxR0hlHaSokdinh3TdutfTnDFpvM2Qpxjdwbw0YnHVj7XtIfe
EjxnFchNFW/alpzNvCMyRF/ThTcSkZ41fysOkkAPCbYameyoVhTHJEnfKIUEXrQH
/WwrcZ7THas5eauJKhN+mqNpDjoTDbFUWhAGC/Y1TMtXxj3VjGg2NP/YMNJ6KLjc
OPoF6hbm+YmJt2TbtFsZEWs0cnKse0QHCwQYrtm9289jvhTLhcTI7PdTGeavR8Lu
dj4L3ne48zXlUPjMuuOzaZC/40ePNDUhXaN+K6bgk3cbcJZlvP/b/HuNFYBC5T3k
xOgpgyoB19RksHqhliXpGhW5Xc/zuDWHtkxBkGgKKKZZ7VKpOoCrp5mMg3d8JQKG
+PzVCBVCGlF/gUT1Tsvv
=9r8c
-----END PGP SIGNATURE-----

Re: parisc: fix mmap(MAP_FIXED|MAP_SHARED) to already mmapped address

[Aaro Koskinen]

Hi,

On Sun, Mar 02, 2014 at 10:22:54PM +0100, Helge Deller wrote:
> On 12/19/2013 10:19 PM, Mike Frysinger wrote:
> > On Thursday 19 December 2013 14:44:40 John David Anglin wrote:
> >> On 12/19/2013 2:17 PM, Aaro Koskinen wrote:
> >>> This commit (0576da2c08e3d332f1b0653030d28ab804585ab6) and the current
> >>> mainline kernel (3.13-rc4) gives me the following with GLIBC 2.18:
> >>>
> >>> $ localedef -c -i en_US -f UTF-8 en_US.UTF-8
> >>> cannot map archive header: Invalid argument
> >>>
> >>> strace looks like this:
> >>>
> >>> mmap2(NULL, 536870912, PROT_NONE, MAP_SHARED, 3, 0) = 0x42f34000
> >>> mmap2(0x43000000, 1607632, PROT_READ|PROT_WRITE, MAP_SHARED|MAP_FIXED, 3,
> >>> 0) = -1 EINVAL (Invalid argument)
> >>>
> >>> With the patch reverted, it works:
> >>>
> >>> mmap2(NULL, 536870912, PROT_NONE, MAP_SHARED, 3, 0) = 0x42d74000
> >>> mmap2(0x43000000, 1607632, PROT_READ|PROT_WRITE, MAP_SHARED|MAP_FIXED, 3,
> >>> 0) = 0x43000000
> >>>
> >>> BTW, note that for GLIBC 2.18 some changes were done regarding this:
> >>> https://sourceware.org/bugzilla/show_bug.cgi?id=10283
> >>
> >> Are you sure the glibc changes are correct?  PARISC addresses are not
> >> strictly aligned to SHMLBA.
> >> There's also a page offset and "random" offset derived from the kernel
> >> address of the mapping struct.
> > 
> > the glibc changes were to use SHMLBA as the min alignment where as before it 
> > was using PAGE_SIZE.  the kernel shouldn't reject us because we manually 
> > increased our alignment ... the mappings should still be overlapping, so it 
> > shouldn't be an "out of mapping space" issue.
> > 
> > the first one gets a really large map (not fixed), and then the second does a 
> > small mapping inside of that first one.
> 
> This doesn't work for parisc.
> We currently face this issue again now with eglibc on debian since we updated
> it.

FWIW, this bug is still present with mainline Linux 3.14 + GLIBC 2.19.

A.

Re: parisc: fix mmap(MAP_FIXED|MAP_SHARED) to already mmapped address

[Helge Deller]

On 04/01/2014 08:26 PM, Aaro Koskinen wrote:
> Hi,
> 
> On Sun, Mar 02, 2014 at 10:22:54PM +0100, Helge Deller wrote:
>> On 12/19/2013 10:19 PM, Mike Frysinger wrote:
>>> On Thursday 19 December 2013 14:44:40 John David Anglin wrote:
>>>> On 12/19/2013 2:17 PM, Aaro Koskinen wrote:
>>>>> This commit (0576da2c08e3d332f1b0653030d28ab804585ab6) and the current
>>>>> mainline kernel (3.13-rc4) gives me the following with GLIBC 2.18:
>>>>>
>>>>> $ localedef -c -i en_US -f UTF-8 en_US.UTF-8
>>>>> cannot map archive header: Invalid argument
>>>>>
>>>>> strace looks like this:
>>>>>
>>>>> mmap2(NULL, 536870912, PROT_NONE, MAP_SHARED, 3, 0) = 0x42f34000
>>>>> mmap2(0x43000000, 1607632, PROT_READ|PROT_WRITE, MAP_SHARED|MAP_FIXED, 3,
>>>>> 0) = -1 EINVAL (Invalid argument)
>>>>>
>>>>> With the patch reverted, it works:
>>>>>
>>>>> mmap2(NULL, 536870912, PROT_NONE, MAP_SHARED, 3, 0) = 0x42d74000
>>>>> mmap2(0x43000000, 1607632, PROT_READ|PROT_WRITE, MAP_SHARED|MAP_FIXED, 3,
>>>>> 0) = 0x43000000
>>>>>
>>>>> BTW, note that for GLIBC 2.18 some changes were done regarding this:
>>>>> https://sourceware.org/bugzilla/show_bug.cgi?id=10283
>>>>
>>>> Are you sure the glibc changes are correct?  PARISC addresses are not
>>>> strictly aligned to SHMLBA.
>>>> There's also a page offset and "random" offset derived from the kernel
>>>> address of the mapping struct.
>>>
>>> the glibc changes were to use SHMLBA as the min alignment where as before it 
>>> was using PAGE_SIZE.  the kernel shouldn't reject us because we manually 
>>> increased our alignment ... the mappings should still be overlapping, so it 
>>> shouldn't be an "out of mapping space" issue.
>>>
>>> the first one gets a really large map (not fixed), and then the second does a 
>>> small mapping inside of that first one.
>>
>> This doesn't work for parisc.
>> We currently face this issue again now with eglibc on debian since we updated
>> it.
> 
> FWIW, this bug is still present with mainline Linux 3.14 + GLIBC 2.19.

Yes.
But it's not a kernel bug. Kernel 3.14 and previous stable releases are OK.

I did proposed a glibc change in my previous mail (http://www.spinics.net/lists/linux-parisc/msg05384.html).
Debian bug report with patch is here:
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=741243

And this is what I proposed:

A trivial FIX/workaround would be to change libc-mmap.h like this:
#ifdef __hppa__
#define MAP_FIXED_ALIGNMENT 4096
#else
#define MAP_FIXED_ALIGNMENT SHMLBA
#endif

That works because then the new aligned address is then the same as the original
(the mmap call returns 4k aligned addresses, so it stays unchanged), but I'm not sure
if such a patch would be acceptable.
Do you have another idea/proposal?

Helge

Re: parisc: fix mmap(MAP_FIXED|MAP_SHARED) to already mmapped address

[Carlos O'Donell]

On Tue, Apr 1, 2014 at 2:49 PM, Helge Deller <deller@gmx.de> wrote:
> Yes.
> But it's not a kernel bug. Kernel 3.14 and previous stable releases are OK.
>
> I did proposed a glibc change in my previous mail (http://www.spinics.net/lists/linux-parisc/msg05384.html).
> Debian bug report with patch is here:
> https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=741243
>
> And this is what I proposed:
>
> A trivial FIX/workaround would be to change libc-mmap.h like this:
> #ifdef __hppa__
> #define MAP_FIXED_ALIGNMENT 4096
> #else
> #define MAP_FIXED_ALIGNMENT SHMLBA
> #endif
>
> That works because then the new aligned address is then the same as the original
> (the mmap call returns 4k aligned addresses, so it stays unchanged), but I'm not sure
> if such a patch would be acceptable.
> Do you have another idea/proposal?

The responsibility for fixing this falls to me, but I've been busy.

If someone wants to propose a patch for glibc please email
libc-alpha@sourceware.org, TO me, and I'll review and commit the patch
granted that you show you've done the appropriate testing.

Otherwise I'll get to this at some point in the next couple of weeks :-(

Cheers,
Carlos.

Re: parisc: fix mmap(MAP_FIXED|MAP_SHARED) to already mmapped address

[Helge Deller]

On 04/02/2014 09:09 PM, Carlos O'Donell wrote:
> On Tue, Apr 1, 2014 at 2:49 PM, Helge Deller <deller@gmx.de> wrote:
>> Yes.
>> But it's not a kernel bug. Kernel 3.14 and previous stable releases are OK.
>>
>> I did proposed a glibc change in my previous mail (http://www.spinics.net/lists/linux-parisc/msg05384.html).
>> Debian bug report with patch is here:
>> https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=741243
>>
>> And this is what I proposed:
>>
>> A trivial FIX/workaround would be to change libc-mmap.h like this:
>> #ifdef __hppa__
>> #define MAP_FIXED_ALIGNMENT 4096
>> #else
>> #define MAP_FIXED_ALIGNMENT SHMLBA
>> #endif
>>
>> That works because then the new aligned address is then the same as the original
>> (the mmap call returns 4k aligned addresses, so it stays unchanged), but I'm not sure
>> if such a patch would be acceptable.
>> Do you have another idea/proposal?
> 
> The responsibility for fixing this falls to me, but I've been busy.

No problem.

> If someone wants to propose a patch for glibc please email
> libc-alpha@sourceware.org, TO me, and I'll review and commit the patch
> granted that you show you've done the appropriate testing.
> 
> Otherwise I'll get to this at some point in the next couple of weeks :-(

Hi Carlos,

I'm not really sure if my patch is the best way to go. Technically it's correct
and it's tested since all our debian buildservers currently run with this patch.
But all other options would probably involve more code changes.

So, I think I'm happy if you can look at it at some point when you find time.
Your input would be very valuable here.

Helge

Re: parisc: fix mmap(MAP_FIXED|MAP_SHARED) to already mmapped address

[John David Anglin]

On 4/2/2014 5:09 PM, Helge Deller wrote:
> On 04/02/2014 09:09 PM, Carlos O'Donell wrote:
>> On Tue, Apr 1, 2014 at 2:49 PM, Helge Deller <deller@gmx.de> wrote:
>>> Yes.
>>> But it's not a kernel bug. Kernel 3.14 and previous stable releases are OK.
>>>
>>> I did proposed a glibc change in my previous mail (http://www.spinics.net/lists/linux-parisc/msg05384.html).
>>> Debian bug report with patch is here:
>>> https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=741243
>>>
>>> And this is what I proposed:
>>>
>>> A trivial FIX/workaround would be to change libc-mmap.h like this:
>>> #ifdef __hppa__
>>> #define MAP_FIXED_ALIGNMENT 4096
>>> #else
>>> #define MAP_FIXED_ALIGNMENT SHMLBA
>>> #endif
>>>
>>> That works because then the new aligned address is then the same as the original
>>> (the mmap call returns 4k aligned addresses, so it stays unchanged), but I'm not sure
>>> if such a patch would be acceptable.
>>> Do you have another idea/proposal?
>> The responsibility for fixing this falls to me, but I've been busy.
> No problem.
>
>> If someone wants to propose a patch for glibc please email
>> libc-alpha@sourceware.org, TO me, and I'll review and commit the patch
>> granted that you show you've done the appropriate testing.
>>
>> Otherwise I'll get to this at some point in the next couple of weeks :-(
> Hi Carlos,
>
> I'm not really sure if my patch is the best way to go. Technically it's correct
> and it's tested since all our debian buildservers currently run with this patch.
> But all other options would probably involve more code changes.
>
> So, I think I'm happy if you can look at it at some point when you find time.
> Your input would be very valuable here.
I'm wondering if kernel value for SHMLBA shouldn't change to PAGE_SIZE 
to better
reflect that attach addresses are page aligned.  The color alignment for 
shared maps
seems a separate issue which maybe userspace doesn't need to worry about.

Dave

-- 
John David Anglin    dave.anglin@bell.net

Re: parisc: fix mmap(MAP_FIXED|MAP_SHARED) to already mmapped address

[Helge Deller]

[-- Attachment #1: Type: text/plain, Size: 2428 bytes --]

On 04/02/2014 11:41 PM, John David Anglin wrote:
> On 4/2/2014 5:09 PM, Helge Deller wrote:
>> On 04/02/2014 09:09 PM, Carlos O'Donell wrote:
>>> On Tue, Apr 1, 2014 at 2:49 PM, Helge Deller <deller@gmx.de> wrote:
>>>> Yes.
>>>> But it's not a kernel bug. Kernel 3.14 and previous stable releases are OK.
>>>>
>>>> I did proposed a glibc change in my previous mail (http://www.spinics.net/lists/linux-parisc/msg05384.html).
>>>> Debian bug report with patch is here:
>>>> https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=741243
>>>>
>>>> And this is what I proposed:
>>>>
>>>> A trivial FIX/workaround would be to change libc-mmap.h like this:
>>>> #ifdef __hppa__
>>>> #define MAP_FIXED_ALIGNMENT 4096
>>>> #else
>>>> #define MAP_FIXED_ALIGNMENT SHMLBA
>>>> #endif
>>>>
>>>> That works because then the new aligned address is then the same as the original
>>>> (the mmap call returns 4k aligned addresses, so it stays unchanged), but I'm not sure
>>>> if such a patch would be acceptable.
>>>> Do you have another idea/proposal?
>>> The responsibility for fixing this falls to me, but I've been busy.
>> No problem.
>>
>>> If someone wants to propose a patch for glibc please email
>>> libc-alpha@sourceware.org, TO me, and I'll review and commit the patch
>>> granted that you show you've done the appropriate testing.
>>>
>>> Otherwise I'll get to this at some point in the next couple of weeks :-(
>> Hi Carlos,
>>
>> I'm not really sure if my patch is the best way to go. Technically it's correct
>> and it's tested since all our debian buildservers currently run with this patch.
>> But all other options would probably involve more code changes.
>>
>> So, I think I'm happy if you can look at it at some point when you find time.
>> Your input would be very valuable here.

> I'm wondering if kernel value for SHMLBA shouldn't change to PAGE_SIZE to better
> reflect that attach addresses are page aligned.  The color alignment for shared maps
> seems a separate issue which maybe userspace doesn't need to worry about.

I think this is a very interesting idea and it should be pretty simple!

The attached patch for eglibc should resolve it.
And the attached patch for kernel isn't necessary, but makes it clear that the colouring is important.
 
I did tested the kernel patch - and it seems to work without problems.

I'm not sure if this might introduce userspace compile problems though (although unlikely).

Helge


[-- Attachment #2: kernel_SHMLBA.patch --]
[-- Type: text/x-patch, Size: 3648 bytes --]

diff --git a/arch/parisc/include/asm/shmparam.h b/arch/parisc/include/asm/shmparam.h
index 628ddc2..d749144 100644
--- a/arch/parisc/include/asm/shmparam.h
+++ b/arch/parisc/include/asm/shmparam.h
@@ -1,8 +1,7 @@
 #ifndef _ASMPARISC_SHMPARAM_H
 #define _ASMPARISC_SHMPARAM_H
 
-#define __ARCH_FORCE_SHMLBA 	1
-
-#define SHMLBA 0x00400000   /* attach addr needs to be 4 Mb aligned */
+#define SHMLBA	   PAGE_SIZE	/* attach addr a multiple of this */
+#define SHM_COLOUR 0x00400000	/* shared mappings coulouring */
 
 #endif /* _ASMPARISC_SHMPARAM_H */
diff --git a/arch/parisc/kernel/cache.c b/arch/parisc/kernel/cache.c
index a6ffc77..2ea77e7 100644
--- a/arch/parisc/kernel/cache.c
+++ b/arch/parisc/kernel/cache.c
@@ -323,7 +323,7 @@ void flush_dcache_page(struct page *page)
 		 * specifically accesses it, of course) */
 
 		flush_tlb_page(mpnt, addr);
-		if (old_addr == 0 || (old_addr & (SHMLBA - 1)) != (addr & (SHMLBA - 1))) {
+		if (old_addr == 0 || (old_addr & (SHM_COLOUR - 1)) != (addr & (SHM_COLOUR - 1))) {
 			__flush_cache_page(mpnt, addr, page_to_phys(page));
 			if (old_addr)
 				printk(KERN_ERR "INEQUIVALENT ALIASES 0x%lx and 0x%lx in file %s\n", old_addr, addr, mpnt->vm_file ? (char *)mpnt->vm_file->f_path.dentry->d_name.name : "(null)");
diff --git a/arch/parisc/kernel/sys_parisc.c b/arch/parisc/kernel/sys_parisc.c
index b7cadc4..31ffa9b 100644
--- a/arch/parisc/kernel/sys_parisc.c
+++ b/arch/parisc/kernel/sys_parisc.c
@@ -45,7 +45,7 @@
 
 static int get_offset(unsigned int last_mmap)
 {
-	return (last_mmap & (SHMLBA-1)) >> PAGE_SHIFT;
+	return (last_mmap & (SHM_COLOUR-1)) >> PAGE_SHIFT;
 }
 
 static unsigned long shared_align_offset(unsigned int last_mmap,
@@ -57,8 +57,8 @@ static unsigned long shared_align_offset(unsigned int last_mmap,
 static inline unsigned long COLOR_ALIGN(unsigned long addr,
 			 unsigned int last_mmap, unsigned long pgoff)
 {
-	unsigned long base = (addr+SHMLBA-1) & ~(SHMLBA-1);
-	unsigned long off  = (SHMLBA-1) &
+	unsigned long base = (addr+SHM_COLOUR-1) & ~(SHM_COLOUR-1);
+	unsigned long off  = (SHM_COLOUR-1) &
 		(shared_align_offset(last_mmap, pgoff) << PAGE_SHIFT);
 
 	return base + off;
@@ -101,7 +101,7 @@ unsigned long arch_get_unmapped_area(struct file *filp, unsigned long addr,
 	if (flags & MAP_FIXED) {
 		if ((flags & MAP_SHARED) && last_mmap &&
 		    (addr - shared_align_offset(last_mmap, pgoff))
-				& (SHMLBA - 1))
+				& (SHM_COLOUR - 1))
 			return -EINVAL;
 		goto found_addr;
 	}
@@ -122,7 +122,7 @@ unsigned long arch_get_unmapped_area(struct file *filp, unsigned long addr,
 	info.length = len;
 	info.low_limit = mm->mmap_legacy_base;
 	info.high_limit = mmap_upper_limit();
-	info.align_mask = last_mmap ? (PAGE_MASK & (SHMLBA - 1)) : 0;
+	info.align_mask = last_mmap ? (PAGE_MASK & (SHM_COLOUR - 1)) : 0;
 	info.align_offset = shared_align_offset(last_mmap, pgoff);
 	addr = vm_unmapped_area(&info);
 
@@ -161,7 +161,7 @@ arch_get_unmapped_area_topdown(struct file *filp, const unsigned long addr0,
 	if (flags & MAP_FIXED) {
 		if ((flags & MAP_SHARED) && last_mmap &&
 		    (addr - shared_align_offset(last_mmap, pgoff))
-			& (SHMLBA - 1))
+			& (SHM_COLOUR - 1))
 			return -EINVAL;
 		goto found_addr;
 	}
@@ -182,7 +182,7 @@ arch_get_unmapped_area_topdown(struct file *filp, const unsigned long addr0,
 	info.length = len;
 	info.low_limit = PAGE_SIZE;
 	info.high_limit = mm->mmap_base;
-	info.align_mask = last_mmap ? (PAGE_MASK & (SHMLBA - 1)) : 0;
+	info.align_mask = last_mmap ? (PAGE_MASK & (SHM_COLOUR - 1)) : 0;
 	info.align_offset = shared_align_offset(last_mmap, pgoff);
 	addr = vm_unmapped_area(&info);
 	if (!(addr & ~PAGE_MASK))

[-- Attachment #3: eglibc.shmlba.patch --]
[-- Type: text/x-patch, Size: 597 bytes --]

diff -up ./ports/sysdeps/unix/sysv/linux/hppa/bits/shm.h.org ./ports/sysdeps/unix/sysv/linux/hppa/bits/shm.h
--- ./ports/sysdeps/unix/sysv/linux/hppa/bits/shm.h.org	2014-04-03 13:20:43.644098000 -0600
+++ ./ports/sysdeps/unix/sysv/linux/hppa/bits/shm.h	2014-04-03 13:22:15.840098000 -0600
@@ -36,7 +36,7 @@
 #define SHM_UNLOCK	12		/* unlock segment (root only) */
 
 /* Segment low boundary address multiple.  */
-#define SHMLBA 0x00400000		/* address needs to be 4 Mb aligned */
+#define SHMLBA		(__getpagesize ())
 
 /* Type to count number of attaches.  */
 typedef unsigned long int shmatt_t;

Re: parisc: fix mmap(MAP_FIXED|MAP_SHARED) to already mmapped address

[John David Anglin]

On 4/3/2014 3:41 PM, Helge Deller wrote:
> On 04/02/2014 11:41 PM, John David Anglin wrote:
>> On 4/2/2014 5:09 PM, Helge Deller wrote:
>>> On 04/02/2014 09:09 PM, Carlos O'Donell wrote:
>>>> On Tue, Apr 1, 2014 at 2:49 PM, Helge Deller <deller@gmx.de> wrote:
>>>>> Yes.
>>>>> But it's not a kernel bug. Kernel 3.14 and previous stable releases are OK.
>>>>>
>>>>> I did proposed a glibc change in my previous mail (http://www.spinics.net/lists/linux-parisc/msg05384.html).
>>>>> Debian bug report with patch is here:
>>>>> https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=741243
>>>>>
>>>>> And this is what I proposed:
>>>>>
>>>>> A trivial FIX/workaround would be to change libc-mmap.h like this:
>>>>> #ifdef __hppa__
>>>>> #define MAP_FIXED_ALIGNMENT 4096
>>>>> #else
>>>>> #define MAP_FIXED_ALIGNMENT SHMLBA
>>>>> #endif
>>>>>
>>>>> That works because then the new aligned address is then the same as the original
>>>>> (the mmap call returns 4k aligned addresses, so it stays unchanged), but I'm not sure
>>>>> if such a patch would be acceptable.
>>>>> Do you have another idea/proposal?
>>>> The responsibility for fixing this falls to me, but I've been busy.
>>> No problem.
>>>
>>>> If someone wants to propose a patch for glibc please email
>>>> libc-alpha@sourceware.org, TO me, and I'll review and commit the patch
>>>> granted that you show you've done the appropriate testing.
>>>>
>>>> Otherwise I'll get to this at some point in the next couple of weeks :-(
>>> Hi Carlos,
>>>
>>> I'm not really sure if my patch is the best way to go. Technically it's correct
>>> and it's tested since all our debian buildservers currently run with this patch.
>>> But all other options would probably involve more code changes.
>>>
>>> So, I think I'm happy if you can look at it at some point when you find time.
>>> Your input would be very valuable here.
>> I'm wondering if kernel value for SHMLBA shouldn't change to PAGE_SIZE to better
>> reflect that attach addresses are page aligned.  The color alignment for shared maps
>> seems a separate issue which maybe userspace doesn't need to worry about.
> I think this is a very interesting idea and it should be pretty simple!
>
> The attached patch for eglibc should resolve it.
> And the attached patch for kernel isn't necessary, but makes it clear that the colouring is important.
>   
> I did tested the kernel patch - and it seems to work without problems.
>
> I'm not sure if this might introduce userspace compile problems though (although unlikely).
Very nice!  In our current Debian eglibc build, SHMLBA is set to 4096.
So, it should work just fine with the kernel patch.  The buildds have 
been running for some time
and I'm not aware of any mmap issues aside from the pthread_create 
ENOMEM errors.

Do you think this helps the allocation of small maps (perl locale test bug)?

Dave

-- 
John David Anglin    dave.anglin@bell.net

Re: parisc: fix mmap(MAP_FIXED|MAP_SHARED) to already mmapped address

[Helge Deller]

On 04/03/2014 10:03 PM, John David Anglin wrote:
> On 4/3/2014 3:41 PM, Helge Deller wrote:
>> On 04/02/2014 11:41 PM, John David Anglin wrote:
>>> On 4/2/2014 5:09 PM, Helge Deller wrote:
>>>> On 04/02/2014 09:09 PM, Carlos O'Donell wrote:
>>>>> On Tue, Apr 1, 2014 at 2:49 PM, Helge Deller <deller@gmx.de> wrote:
>>>>>> Yes.
>>>>>> But it's not a kernel bug. Kernel 3.14 and previous stable releases are OK.
>>>>>>
>>>>>> I did proposed a glibc change in my previous mail (http://www.spinics.net/lists/linux-parisc/msg05384.html).
>>>>>> Debian bug report with patch is here:
>>>>>> https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=741243
>>>>>>
>>>>>> And this is what I proposed:
>>>>>>
>>>>>> A trivial FIX/workaround would be to change libc-mmap.h like this:
>>>>>> #ifdef __hppa__
>>>>>> #define MAP_FIXED_ALIGNMENT 4096
>>>>>> #else
>>>>>> #define MAP_FIXED_ALIGNMENT SHMLBA
>>>>>> #endif
>>>>>>
>>>>>> That works because then the new aligned address is then the same as the original
>>>>>> (the mmap call returns 4k aligned addresses, so it stays unchanged), but I'm not sure
>>>>>> if such a patch would be acceptable.
>>>>>> Do you have another idea/proposal?
>>>>> The responsibility for fixing this falls to me, but I've been busy.
>>>> No problem.
>>>>
>>>>> If someone wants to propose a patch for glibc please email
>>>>> libc-alpha@sourceware.org, TO me, and I'll review and commit the patch
>>>>> granted that you show you've done the appropriate testing.
>>>>>
>>>>> Otherwise I'll get to this at some point in the next couple of weeks :-(
>>>> Hi Carlos,
>>>>
>>>> I'm not really sure if my patch is the best way to go. Technically it's correct
>>>> and it's tested since all our debian buildservers currently run with this patch.
>>>> But all other options would probably involve more code changes.
>>>>
>>>> So, I think I'm happy if you can look at it at some point when you find time.
>>>> Your input would be very valuable here.
>>> I'm wondering if kernel value for SHMLBA shouldn't change to PAGE_SIZE to better
>>> reflect that attach addresses are page aligned.  The color alignment for shared maps
>>> seems a separate issue which maybe userspace doesn't need to worry about.
>> I think this is a very interesting idea and it should be pretty simple!
>>
>> The attached patch for eglibc should resolve it.
>> And the attached patch for kernel isn't necessary, but makes it clear that the colouring is important.
>>   I did tested the kernel patch - and it seems to work without problems.
>>
>> I'm not sure if this might introduce userspace compile problems though (although unlikely).

> Very nice!  In our current Debian eglibc build, SHMLBA is set to 4096.

*No*, it's not!
In current eglibc it's set to 0x00400000
That's what my eglibc-patch changes...
I'm currently building a eglibc on hpviz with SHMLBA set to 4096 (__getpagesize()).

> So, it should work just fine with the kernel patch. 

No, the kernel patch isn't necessary. Only the glibc patch.

> The buildds have been running for some time
> and I'm not aware of any mmap issues aside from the pthread_create ENOMEM errors.

True - we need to find the cause.
I just suspected the arch_get_unmapped_area() kernel functions, but they seem correct.
 
> Do you think this helps the allocation of small maps (perl locale test bug)?

No. Only my other (unfinished) patches will resolve this.

Helge

Re: parisc: fix mmap(MAP_FIXED|MAP_SHARED) to already mmapped address

[Carlos O'Donell]

On Thu, Apr 3, 2014 at 4:26 PM, Helge Deller <deller@gmx.de> wrote:
> In current eglibc it's set to 0x00400000
> That's what my eglibc-patch changes...
> I'm currently building a eglibc on hpviz with SHMLBA set to 4096 (__getpagesize()).

Anyone object to me fixing this upstream by making SHMLBA match the kernel?

I plan to use a fixed value of 4096, since I never expect hppa
userspace to have to care (even if the kernel uses superpages).

Please correct me if I'm wrong.

Cheers,
Carlos.

Re: parisc: fix mmap(MAP_FIXED|MAP_SHARED) to already mmapped address

[John David Anglin]

On 2015-02-20, at 4:36 PM, Carlos O'Donell wrote:

> On Thu, Apr 3, 2014 at 4:26 PM, Helge Deller <deller@gmx.de> wrote:
>> In current eglibc it's set to 0x00400000
>> That's what my eglibc-patch changes...
>> I'm currently building a eglibc on hpviz with SHMLBA set to 4096 (__getpagesize()).
> 
> Anyone object to me fixing this upstream by making SHMLBA match the kernel?
> 
> I plan to use a fixed value of 4096, since I never expect hppa
> userspace to have to care (even if the kernel uses superpages).

We currently use (__getpagesize ()) in Debian and this seems to be a common definition.
Is there a performance advantage in using 4096?

> 
> Please correct me if I'm wrong.


At one time, we thought this value needed to be 4 MB.  Helge was working on improving the mmap
allocation scheme but this work stalled after some improvement.  I can't remember the issues and how
they relate to SHMLBA.

Dave
--
John David Anglin	dave.anglin@bell.net

Re: parisc: fix mmap(MAP_FIXED|MAP_SHARED) to already mmapped address

[John David Anglin]

On 2015-02-21, at 3:31 PM, John David Anglin wrote:

> On 2015-02-20, at 4:36 PM, Carlos O'Donell wrote:
> 
>> On Thu, Apr 3, 2014 at 4:26 PM, Helge Deller <deller@gmx.de> wrote:
>>> In current eglibc it's set to 0x00400000
>>> That's what my eglibc-patch changes...
>>> I'm currently building a eglibc on hpviz with SHMLBA set to 4096 (__getpagesize()).
>> 
>> Anyone object to me fixing this upstream by making SHMLBA match the kernel?
>> 
>> I plan to use a fixed value of 4096, since I never expect hppa
>> userspace to have to care (even if the kernel uses superpages).
> 
> We currently use (__getpagesize ()) in Debian and this seems to be a common definition.
> Is there a performance advantage in using 4096?
> 
>> 
>> Please correct me if I'm wrong.
> 
> 
> At one time, we thought this value needed to be 4 MB.  Helge was working on improving the mmap
> allocation scheme but this work stalled after some improvement.  I can't remember the issues and how
> they relate to SHMLBA.


Actually, the number was 4 Mb (bit).

Dave
--
John David Anglin	dave.anglin@bell.net

Re: parisc: fix mmap(MAP_FIXED|MAP_SHARED) to already mmapped address

[James Bottomley]

On Sat, 2015-02-21 at 15:40 -0500, John David Anglin wrote:
> On 2015-02-21, at 3:31 PM, John David Anglin wrote:
> 
> > On 2015-02-20, at 4:36 PM, Carlos O'Donell wrote:
> > 
> >> On Thu, Apr 3, 2014 at 4:26 PM, Helge Deller <deller@gmx.de> wrote:
> >>> In current eglibc it's set to 0x00400000
> >>> That's what my eglibc-patch changes...
> >>> I'm currently building a eglibc on hpviz with SHMLBA set to 4096 (__getpagesize()).
> >> 
> >> Anyone object to me fixing this upstream by making SHMLBA match the kernel?
> >> 
> >> I plan to use a fixed value of 4096, since I never expect hppa
> >> userspace to have to care (even if the kernel uses superpages).
> > 
> > We currently use (__getpagesize ()) in Debian and this seems to be a common definition.
> > Is there a performance advantage in using 4096?
> > 
> >> 
> >> Please correct me if I'm wrong.
> > 
> > 
> > At one time, we thought this value needed to be 4 MB.  Helge was
> working on improving the mmap
> > allocation scheme but this work stalled after some improvement.  I
> can't remember the issues and how
> > they relate to SHMLBA.
> 
> 
> Actually, the number was 4 Mb (bit).

No, it was 4MB.  That's the cache equivalency stride on PA processors
because we have a VIPT cache.  The architectural requirement according
to the dreaded appendix F is 16MB but we were assured by the PA
architects that it was 4 because they never planned producing processors
that would require 16.  The actual meaning is it's the number of bits of
the virtual address that are significant in the virtual index.

The point of SHMLBA is that if the same physical page is mapped into two
different virtual addresses but the two addresses are equal, modulo
SHMLBA, then the L1 cache sees the equivalency and you can't get
inequivalent cache aliases for the page (two writes to the two different
addresses producing two separately dirty cache lines which can never
resolve).  This means that the virtual addresses of all shared mappings
have to be equal modulo SHMLBA for the caches not to alias.

James

Re: parisc: fix mmap(MAP_FIXED|MAP_SHARED) to already mmapped address

[Helge Deller]

On 22.02.2015 00:09, James Bottomley wrote:
> On Sat, 2015-02-21 at 15:40 -0500, John David Anglin wrote:
>> On 2015-02-21, at 3:31 PM, John David Anglin wrote:
>>
>>> On 2015-02-20, at 4:36 PM, Carlos O'Donell wrote:
>>>
>>>> On Thu, Apr 3, 2014 at 4:26 PM, Helge Deller <deller@gmx.de> wrote:
>>>>> In current eglibc it's set to 0x00400000
>>>>> That's what my eglibc-patch changes...
>>>>> I'm currently building a eglibc on hpviz with SHMLBA set to 4096 (__getpagesize()).
>>>>
>>>> Anyone object to me fixing this upstream by making SHMLBA match the kernel?
>>>>
>>>> I plan to use a fixed value of 4096, since I never expect hppa
>>>> userspace to have to care (even if the kernel uses superpages).
>>>
>>> We currently use (__getpagesize ()) in Debian and this seems to be a common definition.
>>> Is there a performance advantage in using 4096?
>>>
>>>>
>>>> Please correct me if I'm wrong.
>>>
>>>
>>> At one time, we thought this value needed to be 4 MB.  Helge was
>> working on improving the mmap
>>> allocation scheme but this work stalled after some improvement.  I
>> can't remember the issues and how
>>> they relate to SHMLBA.
>>
>>
>> Actually, the number was 4 Mb (bit).
>
> No, it was 4MB.  That's the cache equivalency stride on PA processors
> because we have a VIPT cache.  The architectural requirement according
> to the dreaded appendix F is 16MB but we were assured by the PA
> architects that it was 4 because they never planned producing processors
> that would require 16.  The actual meaning is it's the number of bits of
> the virtual address that are significant in the virtual index.
>

Your following statement:

> The point of SHMLBA is that if the same physical page is mapped into two
> different virtual addresses but the two addresses are equal, modulo
> SHMLBA, then the L1 cache sees the equivalency and you can't get
> inequivalent cache aliases for the page (two writes to the two different
> addresses producing two separately dirty cache lines which can never
> resolve).  This means that the virtual addresses of all shared mappings
> have to be equal modulo SHMLBA for the caches not to alias.

With this you define SHMLBA to be the representative number which defines
what the current cache equivalency stride of the kernel is, *and* which then can
be used by userspace. I think this is a misinterpretation of SHMLBA (or at
least a parisc-specific interpretation of SHMLBA), which is not like how it
is used on other architectures with similar limitations.
Userspace should not know the kernel/architecture specifics. Instead they
should try to mmap() memory somewhere (e.g. 4KB aligned) and if they need shared mappings then
kernel/glibc will return a corrected mapping address (modulo 4MB).
I think this is important, since most userspace programs usually try to mmap at
a multiple of SHMLBA with which we then run very soon out of userspace (with SHMLBA=4MB).
This has been the issue with localedef in glibc (a strange coding which tries
to be platform-specific with mmap-calculation). Because of that in the end
it turned out to be best for parisc to have SHMLBA defined to 4kb (and not 4MB).

So, your statement above is correct, I would just not use "SHMLBA" in this term,
but maybe "KERNEL_SHMLBA" instead.

Helge

Re: parisc: fix mmap(MAP_FIXED|MAP_SHARED) to already mmapped address

[James Bottomley]

On Sun, 2015-02-22 at 00:26 +0100, Helge Deller wrote:
> On 22.02.2015 00:09, James Bottomley wrote:
> > On Sat, 2015-02-21 at 15:40 -0500, John David Anglin wrote:
> >> On 2015-02-21, at 3:31 PM, John David Anglin wrote:
> >>
> >>> On 2015-02-20, at 4:36 PM, Carlos O'Donell wrote:
> >>>
> >>>> On Thu, Apr 3, 2014 at 4:26 PM, Helge Deller <deller@gmx.de> wrote:
> >>>>> In current eglibc it's set to 0x00400000
> >>>>> That's what my eglibc-patch changes...
> >>>>> I'm currently building a eglibc on hpviz with SHMLBA set to 4096 (__getpagesize()).
> >>>>
> >>>> Anyone object to me fixing this upstream by making SHMLBA match the kernel?
> >>>>
> >>>> I plan to use a fixed value of 4096, since I never expect hppa
> >>>> userspace to have to care (even if the kernel uses superpages).
> >>>
> >>> We currently use (__getpagesize ()) in Debian and this seems to be a common definition.
> >>> Is there a performance advantage in using 4096?
> >>>
> >>>>
> >>>> Please correct me if I'm wrong.
> >>>
> >>>
> >>> At one time, we thought this value needed to be 4 MB.  Helge was
> >> working on improving the mmap
> >>> allocation scheme but this work stalled after some improvement.  I
> >> can't remember the issues and how
> >>> they relate to SHMLBA.
> >>
> >>
> >> Actually, the number was 4 Mb (bit).
> >
> > No, it was 4MB.  That's the cache equivalency stride on PA processors
> > because we have a VIPT cache.  The architectural requirement according
> > to the dreaded appendix F is 16MB but we were assured by the PA
> > architects that it was 4 because they never planned producing processors
> > that would require 16.  The actual meaning is it's the number of bits of
> > the virtual address that are significant in the virtual index.
> >
> 
> Your following statement:
> 
> > The point of SHMLBA is that if the same physical page is mapped into two
> > different virtual addresses but the two addresses are equal, modulo
> > SHMLBA, then the L1 cache sees the equivalency and you can't get
> > inequivalent cache aliases for the page (two writes to the two different
> > addresses producing two separately dirty cache lines which can never
> > resolve).  This means that the virtual addresses of all shared mappings
> > have to be equal modulo SHMLBA for the caches not to alias.
> 
> With this you define SHMLBA to be the representative number which defines
> what the current cache equivalency stride of the kernel is, *and* which then can
> be used by userspace. I think this is a misinterpretation of SHMLBA (or at
> least a parisc-specific interpretation of SHMLBA), which is not like how it
> is used on other architectures with similar limitations.
> Userspace should not know the kernel/architecture specifics. Instead they
> should try to mmap() memory somewhere (e.g. 4KB aligned) and if they need shared mappings then
> kernel/glibc will return a corrected mapping address (modulo 4MB).
> I think this is important, since most userspace programs usually try to mmap at
> a multiple of SHMLBA with which we then run very soon out of userspace (with SHMLBA=4MB).
> This has been the issue with localedef in glibc (a strange coding which tries
> to be platform-specific with mmap-calculation). Because of that in the end
> it turned out to be best for parisc to have SHMLBA defined to 4kb (and not 4MB).
> 
> So, your statement above is correct, I would just not use "SHMLBA" in this term,
> but maybe "KERNEL_SHMLBA" instead.

Um, no, SHMLBA comes from the SYS-V IPC primitives.  They were stupid
enough to allow the user pick the address of the region of shared
memory, so the user had to know these architectural details and SHMLBA
encodes them (man shmat will give you the gory details).

For mmap, we can mostly do the right thing in the kernel, except for
MAP_FIXED, where the user has to know what they're doing again.

For the cases the user thinks they know best, we can't avoid giving out
the knowledge somehow, because inequivalent aliases in writeable
mappings will HPMC a system.  We could be more relaxed about
inequivalent aliases in read only mappings (say shared libraries), but
the consequence of that is an explosion in the use of cache space, so we
would want some libraries (like glibc) with many shared copies to obey
SHMLBA.

James

Re: parisc: fix mmap(MAP_FIXED|MAP_SHARED) to already mmapped address

[John David Anglin]

On 2015-02-21, at 6:57 PM, James Bottomley wrote:

> On Sun, 2015-02-22 at 00:26 +0100, Helge Deller wrote:
>> On 22.02.2015 00:09, James Bottomley wrote:
>>> On Sat, 2015-02-21 at 15:40 -0500, John David Anglin wrote:
>>>> On 2015-02-21, at 3:31 PM, John David Anglin wrote:
>>>> 
>>>>> On 2015-02-20, at 4:36 PM, Carlos O'Donell wrote:
>>>>> 
>>>>>> On Thu, Apr 3, 2014 at 4:26 PM, Helge Deller <deller@gmx.de> wrote:
>>>>>>> In current eglibc it's set to 0x00400000
>>>>>>> That's what my eglibc-patch changes...
>>>>>>> I'm currently building a eglibc on hpviz with SHMLBA set to 4096 (__getpagesize()).
>>>>>> 
>>>>>> Anyone object to me fixing this upstream by making SHMLBA match the kernel?
>>>>>> 
>>>>>> I plan to use a fixed value of 4096, since I never expect hppa
>>>>>> userspace to have to care (even if the kernel uses superpages).
>>>>> 
>>>>> We currently use (__getpagesize ()) in Debian and this seems to be a common definition.
>>>>> Is there a performance advantage in using 4096?
>>>>> 
>>>>>> 
>>>>>> Please correct me if I'm wrong.
>>>>> 
>>>>> 
>>>>> At one time, we thought this value needed to be 4 MB.  Helge was
>>>> working on improving the mmap
>>>>> allocation scheme but this work stalled after some improvement.  I
>>>> can't remember the issues and how
>>>>> they relate to SHMLBA.
>>>> 
>>>> 
>>>> Actually, the number was 4 Mb (bit).
>>> 
>>> No, it was 4MB.  That's the cache equivalency stride on PA processors
>>> because we have a VIPT cache.  The architectural requirement according
>>> to the dreaded appendix F is 16MB but we were assured by the PA
>>> architects that it was 4 because they never planned producing processors
>>> that would require 16.  The actual meaning is it's the number of bits of
>>> the virtual address that are significant in the virtual index.
>>> 
>> 
>> Your following statement:
>> 
>>> The point of SHMLBA is that if the same physical page is mapped into two
>>> different virtual addresses but the two addresses are equal, modulo
>>> SHMLBA, then the L1 cache sees the equivalency and you can't get
>>> inequivalent cache aliases for the page (two writes to the two different
>>> addresses producing two separately dirty cache lines which can never
>>> resolve).  This means that the virtual addresses of all shared mappings
>>> have to be equal modulo SHMLBA for the caches not to alias.
>> 
>> With this you define SHMLBA to be the representative number which defines
>> what the current cache equivalency stride of the kernel is, *and* which then can
>> be used by userspace. I think this is a misinterpretation of SHMLBA (or at
>> least a parisc-specific interpretation of SHMLBA), which is not like how it
>> is used on other architectures with similar limitations.
>> Userspace should not know the kernel/architecture specifics. Instead they
>> should try to mmap() memory somewhere (e.g. 4KB aligned) and if they need shared mappings then
>> kernel/glibc will return a corrected mapping address (modulo 4MB).
>> I think this is important, since most userspace programs usually try to mmap at
>> a multiple of SHMLBA with which we then run very soon out of userspace (with SHMLBA=4MB).
>> This has been the issue with localedef in glibc (a strange coding which tries
>> to be platform-specific with mmap-calculation). Because of that in the end
>> it turned out to be best for parisc to have SHMLBA defined to 4kb (and not 4MB).
>> 
>> So, your statement above is correct, I would just not use "SHMLBA" in this term,
>> but maybe "KERNEL_SHMLBA" instead.
> 
> Um, no, SHMLBA comes from the SYS-V IPC primitives.  They were stupid
> enough to allow the user pick the address of the region of shared
> memory, so the user had to know these architectural details and SHMLBA
> encodes them (man shmat will give you the gory details).
> 
> For mmap, we can mostly do the right thing in the kernel, except for
> MAP_FIXED, where the user has to know what they're doing again.
> 
> For the cases the user thinks they know best, we can't avoid giving out
> the knowledge somehow, because inequivalent aliases in writeable
> mappings will HPMC a system.  We could be more relaxed about
> inequivalent aliases in read only mappings (say shared libraries), but
> the consequence of that is an explosion in the use of cache space, so we
> would want some libraries (like glibc) with many shared copies to obey
> SHMLBA.


I agree with Helge.  We run out of memory too quickly with 4 MB.  This resulted in various
userspace applications failing as mentioned by Helge.

MAP_FIXED can fail it the address is a problem.  I believe we check for that.

If you look at the pch implementation in gcc, you will see that the MAP_FIXED problem can be
worked around, and this problem is not specific to parisc.  The pch data has to mapped at the same
address as it was originally created as there is no way to relocate the data.

SHMLBA is 4096 /* (1 << PGSHIFT) */ on hpux.

The following is in <asm-generic/shmparam.h>:
#define SHMLBA PAGE_SIZE	 /* attach addr a multiple of this */

Shared mappings are handled with 
asm/shmparam.h:#define SHM_COLOUR 0x00400000	/* shared mappings colouring */

Dave
--
John David Anglin	dave.anglin@bell.net

Re: parisc: fix mmap(MAP_FIXED|MAP_SHARED) to already mmapped address

[James Bottomley]

On Sun, 2015-02-22 at 11:45 -0500, John David Anglin wrote:
> On 2015-02-21, at 6:57 PM, James Bottomley wrote:
> 
> > On Sun, 2015-02-22 at 00:26 +0100, Helge Deller wrote:
> >> On 22.02.2015 00:09, James Bottomley wrote:
> >>> On Sat, 2015-02-21 at 15:40 -0500, John David Anglin wrote:
> >>>> On 2015-02-21, at 3:31 PM, John David Anglin wrote:
> >>>> 
> >>>>> On 2015-02-20, at 4:36 PM, Carlos O'Donell wrote:
> >>>>> 
> >>>>>> On Thu, Apr 3, 2014 at 4:26 PM, Helge Deller <deller@gmx.de> wrote:
> >>>>>>> In current eglibc it's set to 0x00400000
> >>>>>>> That's what my eglibc-patch changes...
> >>>>>>> I'm currently building a eglibc on hpviz with SHMLBA set to 4096 (__getpagesize()).
> >>>>>> 
> >>>>>> Anyone object to me fixing this upstream by making SHMLBA match the kernel?
> >>>>>> 
> >>>>>> I plan to use a fixed value of 4096, since I never expect hppa
> >>>>>> userspace to have to care (even if the kernel uses superpages).
> >>>>> 
> >>>>> We currently use (__getpagesize ()) in Debian and this seems to be a common definition.
> >>>>> Is there a performance advantage in using 4096?
> >>>>> 
> >>>>>> 
> >>>>>> Please correct me if I'm wrong.
> >>>>> 
> >>>>> 
> >>>>> At one time, we thought this value needed to be 4 MB.  Helge was
> >>>> working on improving the mmap
> >>>>> allocation scheme but this work stalled after some improvement.  I
> >>>> can't remember the issues and how
> >>>>> they relate to SHMLBA.
> >>>> 
> >>>> 
> >>>> Actually, the number was 4 Mb (bit).
> >>> 
> >>> No, it was 4MB.  That's the cache equivalency stride on PA processors
> >>> because we have a VIPT cache.  The architectural requirement according
> >>> to the dreaded appendix F is 16MB but we were assured by the PA
> >>> architects that it was 4 because they never planned producing processors
> >>> that would require 16.  The actual meaning is it's the number of bits of
> >>> the virtual address that are significant in the virtual index.
> >>> 
> >> 
> >> Your following statement:
> >> 
> >>> The point of SHMLBA is that if the same physical page is mapped into two
> >>> different virtual addresses but the two addresses are equal, modulo
> >>> SHMLBA, then the L1 cache sees the equivalency and you can't get
> >>> inequivalent cache aliases for the page (two writes to the two different
> >>> addresses producing two separately dirty cache lines which can never
> >>> resolve).  This means that the virtual addresses of all shared mappings
> >>> have to be equal modulo SHMLBA for the caches not to alias.
> >> 
> >> With this you define SHMLBA to be the representative number which defines
> >> what the current cache equivalency stride of the kernel is, *and* which then can
> >> be used by userspace. I think this is a misinterpretation of SHMLBA (or at
> >> least a parisc-specific interpretation of SHMLBA), which is not like how it
> >> is used on other architectures with similar limitations.
> >> Userspace should not know the kernel/architecture specifics. Instead they
> >> should try to mmap() memory somewhere (e.g. 4KB aligned) and if they need shared mappings then
> >> kernel/glibc will return a corrected mapping address (modulo 4MB).
> >> I think this is important, since most userspace programs usually try to mmap at
> >> a multiple of SHMLBA with which we then run very soon out of userspace (with SHMLBA=4MB).
> >> This has been the issue with localedef in glibc (a strange coding which tries
> >> to be platform-specific with mmap-calculation). Because of that in the end
> >> it turned out to be best for parisc to have SHMLBA defined to 4kb (and not 4MB).
> >> 
> >> So, your statement above is correct, I would just not use "SHMLBA" in this term,
> >> but maybe "KERNEL_SHMLBA" instead.
> > 
> > Um, no, SHMLBA comes from the SYS-V IPC primitives.  They were stupid
> > enough to allow the user pick the address of the region of shared
> > memory, so the user had to know these architectural details and SHMLBA
> > encodes them (man shmat will give you the gory details).
> > 
> > For mmap, we can mostly do the right thing in the kernel, except for
> > MAP_FIXED, where the user has to know what they're doing again.
> > 
> > For the cases the user thinks they know best, we can't avoid giving out
> > the knowledge somehow, because inequivalent aliases in writeable
> > mappings will HPMC a system.  We could be more relaxed about
> > inequivalent aliases in read only mappings (say shared libraries), but
> > the consequence of that is an explosion in the use of cache space, so we
> > would want some libraries (like glibc) with many shared copies to obey
> > SHMLBA.
> 
> 
> I agree with Helge.  We run out of memory too quickly with 4 MB.  This resulted in various
> userspace applications failing as mentioned by Helge.
> 
> MAP_FIXED can fail it the address is a problem.  I believe we check for that.
> 
> If you look at the pch implementation in gcc, you will see that the MAP_FIXED problem can be
> worked around, and this problem is not specific to parisc.  The pch data has to mapped at the same
> address as it was originally created as there is no way to relocate the data.
> 
> SHMLBA is 4096 /* (1 << PGSHIFT) */ on hpux.
> 
> The following is in <asm-generic/shmparam.h>:
> #define SHMLBA PAGE_SIZE	 /* attach addr a multiple of this */
> 
> Shared mappings are handled with 
> asm/shmparam.h:#define SHM_COLOUR 0x00400000	/* shared mappings colouring */

So how is the sys-v ipc problem fixed?  There the user is told to select
an address which is a multiple of SHMLBA.  Programs that do this today
will start to break on writeable mappings if we set SHMLBA to PAGE_SIZE
because the colour will be wrong.

James

Re: parisc: fix mmap(MAP_FIXED|MAP_SHARED) to already mmapped address

[Helge Deller]

On 22.02.2015 18:17, James Bottomley wrote:
> On Sun, 2015-02-22 at 11:45 -0500, John David Anglin wrote:
>> On 2015-02-21, at 6:57 PM, James Bottomley wrote:
>>
>>> On Sun, 2015-02-22 at 00:26 +0100, Helge Deller wrote:
>>>> On 22.02.2015 00:09, James Bottomley wrote:
>>>>> On Sat, 2015-02-21 at 15:40 -0500, John David Anglin wrote:
>>>>>> On 2015-02-21, at 3:31 PM, John David Anglin wrote:
>>>>>>
>>>>>>> On 2015-02-20, at 4:36 PM, Carlos O'Donell wrote:
>>>>>>>
>>>>>>>> On Thu, Apr 3, 2014 at 4:26 PM, Helge Deller <deller@gmx.de> wrote:
>>>>>>>>> In current eglibc it's set to 0x00400000
>>>>>>>>> That's what my eglibc-patch changes...
>>>>>>>>> I'm currently building a eglibc on hpviz with SHMLBA set to 4096 (__getpagesize()).
>>>>>>>>
>>>>>>>> Anyone object to me fixing this upstream by making SHMLBA match the kernel?
>>>>>>>>
>>>>>>>> I plan to use a fixed value of 4096, since I never expect hppa
>>>>>>>> userspace to have to care (even if the kernel uses superpages).
>>>>>>>
>>>>>>> We currently use (__getpagesize ()) in Debian and this seems to be a common definition.
>>>>>>> Is there a performance advantage in using 4096?
>>>>>>>
>>>>>>>>
>>>>>>>> Please correct me if I'm wrong.
>>>>>>>
>>>>>>>
>>>>>>> At one time, we thought this value needed to be 4 MB.  Helge was
>>>>>> working on improving the mmap
>>>>>>> allocation scheme but this work stalled after some improvement.  I
>>>>>> can't remember the issues and how
>>>>>>> they relate to SHMLBA.
>>>>>>
>>>>>>
>>>>>> Actually, the number was 4 Mb (bit).
>>>>>
>>>>> No, it was 4MB.  That's the cache equivalency stride on PA processors
>>>>> because we have a VIPT cache.  The architectural requirement according
>>>>> to the dreaded appendix F is 16MB but we were assured by the PA
>>>>> architects that it was 4 because they never planned producing processors
>>>>> that would require 16.  The actual meaning is it's the number of bits of
>>>>> the virtual address that are significant in the virtual index.
>>>>>
>>>>
>>>> Your following statement:
>>>>
>>>>> The point of SHMLBA is that if the same physical page is mapped into two
>>>>> different virtual addresses but the two addresses are equal, modulo
>>>>> SHMLBA, then the L1 cache sees the equivalency and you can't get
>>>>> inequivalent cache aliases for the page (two writes to the two different
>>>>> addresses producing two separately dirty cache lines which can never
>>>>> resolve).  This means that the virtual addresses of all shared mappings
>>>>> have to be equal modulo SHMLBA for the caches not to alias.
>>>>
>>>> With this you define SHMLBA to be the representative number which defines
>>>> what the current cache equivalency stride of the kernel is, *and* which then can
>>>> be used by userspace. I think this is a misinterpretation of SHMLBA (or at
>>>> least a parisc-specific interpretation of SHMLBA), which is not like how it
>>>> is used on other architectures with similar limitations.
>>>> Userspace should not know the kernel/architecture specifics. Instead they
>>>> should try to mmap() memory somewhere (e.g. 4KB aligned) and if they need shared mappings then
>>>> kernel/glibc will return a corrected mapping address (modulo 4MB).
>>>> I think this is important, since most userspace programs usually try to mmap at
>>>> a multiple of SHMLBA with which we then run very soon out of userspace (with SHMLBA=4MB).
>>>> This has been the issue with localedef in glibc (a strange coding which tries
>>>> to be platform-specific with mmap-calculation). Because of that in the end
>>>> it turned out to be best for parisc to have SHMLBA defined to 4kb (and not 4MB).
>>>>
>>>> So, your statement above is correct, I would just not use "SHMLBA" in this term,
>>>> but maybe "KERNEL_SHMLBA" instead.
>>>
>>> Um, no, SHMLBA comes from the SYS-V IPC primitives.  They were stupid
>>> enough to allow the user pick the address of the region of shared
>>> memory, so the user had to know these architectural details and SHMLBA
>>> encodes them (man shmat will give you the gory details).
>>>
>>> For mmap, we can mostly do the right thing in the kernel, except for
>>> MAP_FIXED, where the user has to know what they're doing again.
>>>
>>> For the cases the user thinks they know best, we can't avoid giving out
>>> the knowledge somehow, because inequivalent aliases in writeable
>>> mappings will HPMC a system.  We could be more relaxed about
>>> inequivalent aliases in read only mappings (say shared libraries), but
>>> the consequence of that is an explosion in the use of cache space, so we
>>> would want some libraries (like glibc) with many shared copies to obey
>>> SHMLBA.
>>
>>
>> I agree with Helge.  We run out of memory too quickly with 4 MB.  This resulted in various
>> userspace applications failing as mentioned by Helge.
>>
>> MAP_FIXED can fail it the address is a problem.  I believe we check for that.
>>
>> If you look at the pch implementation in gcc, you will see that the MAP_FIXED problem can be
>> worked around, and this problem is not specific to parisc.  The pch data has to mapped at the same
>> address as it was originally created as there is no way to relocate the data.
>>
>> SHMLBA is 4096 /* (1 << PGSHIFT) */ on hpux.
>>
>> The following is in <asm-generic/shmparam.h>:
>> #define SHMLBA PAGE_SIZE	 /* attach addr a multiple of this */
>>
>> Shared mappings are handled with
>> asm/shmparam.h:#define SHM_COLOUR 0x00400000	/* shared mappings colouring */
>
> So how is the sys-v ipc problem fixed?  There the user is told to select
> an address which is a multiple of SHMLBA.  Programs that do this today
> will start to break on writeable mappings if we set SHMLBA to PAGE_SIZE
> because the colour will be wrong.

No, an mmap() to a fixed shared address which violates the colouring will fail for userspace.
Instead most userspaces today just use a shared mmap() without a fixed address and get returned
a new address (calculated by kernel) which does not violate the colouring.
Dave and me worked on quite some such userspace issues in the last years (esp. glibc), and
having SHMLBA=4096 is the way it now works best as it is similar to the other architectures
and existing userspace programs do cope correctly with it.

Helge

Re: parisc: fix mmap(MAP_FIXED|MAP_SHARED) to already mmapped address

[John David Anglin]

On 2015-02-22, at 12:17 PM, James Bottomley wrote:

> On Sun, 2015-02-22 at 11:45 -0500, John David Anglin wrote:
>> On 2015-02-21, at 6:57 PM, James Bottomley wrote:
>> 
>>> On Sun, 2015-02-22 at 00:26 +0100, Helge Deller wrote:
>>>> On 22.02.2015 00:09, James Bottomley wrote:
>>>>> On Sat, 2015-02-21 at 15:40 -0500, John David Anglin wrote:
>>>>>> On 2015-02-21, at 3:31 PM, John David Anglin wrote:
>>>>>> 
>>>>>>> On 2015-02-20, at 4:36 PM, Carlos O'Donell wrote:
>>>>>>> 
>>>>>>>> On Thu, Apr 3, 2014 at 4:26 PM, Helge Deller <deller@gmx.de> wrote:
>>>>>>>>> In current eglibc it's set to 0x00400000
>>>>>>>>> That's what my eglibc-patch changes...
>>>>>>>>> I'm currently building a eglibc on hpviz with SHMLBA set to 4096 (__getpagesize()).
>>>>>>>> 
>>>>>>>> Anyone object to me fixing this upstream by making SHMLBA match the kernel?
>>>>>>>> 
>>>>>>>> I plan to use a fixed value of 4096, since I never expect hppa
>>>>>>>> userspace to have to care (even if the kernel uses superpages).
>>>>>>> 
>>>>>>> We currently use (__getpagesize ()) in Debian and this seems to be a common definition.
>>>>>>> Is there a performance advantage in using 4096?
>>>>>>> 
>>>>>>>> 
>>>>>>>> Please correct me if I'm wrong.
>>>>>>> 
>>>>>>> 
>>>>>>> At one time, we thought this value needed to be 4 MB.  Helge was
>>>>>> working on improving the mmap
>>>>>>> allocation scheme but this work stalled after some improvement.  I
>>>>>> can't remember the issues and how
>>>>>>> they relate to SHMLBA.
>>>>>> 
>>>>>> 
>>>>>> Actually, the number was 4 Mb (bit).
>>>>> 
>>>>> No, it was 4MB.  That's the cache equivalency stride on PA processors
>>>>> because we have a VIPT cache.  The architectural requirement according
>>>>> to the dreaded appendix F is 16MB but we were assured by the PA
>>>>> architects that it was 4 because they never planned producing processors
>>>>> that would require 16.  The actual meaning is it's the number of bits of
>>>>> the virtual address that are significant in the virtual index.
>>>>> 
>>>> 
>>>> Your following statement:
>>>> 
>>>>> The point of SHMLBA is that if the same physical page is mapped into two
>>>>> different virtual addresses but the two addresses are equal, modulo
>>>>> SHMLBA, then the L1 cache sees the equivalency and you can't get
>>>>> inequivalent cache aliases for the page (two writes to the two different
>>>>> addresses producing two separately dirty cache lines which can never
>>>>> resolve).  This means that the virtual addresses of all shared mappings
>>>>> have to be equal modulo SHMLBA for the caches not to alias.
>>>> 
>>>> With this you define SHMLBA to be the representative number which defines
>>>> what the current cache equivalency stride of the kernel is, *and* which then can
>>>> be used by userspace. I think this is a misinterpretation of SHMLBA (or at
>>>> least a parisc-specific interpretation of SHMLBA), which is not like how it
>>>> is used on other architectures with similar limitations.
>>>> Userspace should not know the kernel/architecture specifics. Instead they
>>>> should try to mmap() memory somewhere (e.g. 4KB aligned) and if they need shared mappings then
>>>> kernel/glibc will return a corrected mapping address (modulo 4MB).
>>>> I think this is important, since most userspace programs usually try to mmap at
>>>> a multiple of SHMLBA with which we then run very soon out of userspace (with SHMLBA=4MB).
>>>> This has been the issue with localedef in glibc (a strange coding which tries
>>>> to be platform-specific with mmap-calculation). Because of that in the end
>>>> it turned out to be best for parisc to have SHMLBA defined to 4kb (and not 4MB).
>>>> 
>>>> So, your statement above is correct, I would just not use "SHMLBA" in this term,
>>>> but maybe "KERNEL_SHMLBA" instead.
>>> 
>>> Um, no, SHMLBA comes from the SYS-V IPC primitives.  They were stupid
>>> enough to allow the user pick the address of the region of shared
>>> memory, so the user had to know these architectural details and SHMLBA
>>> encodes them (man shmat will give you the gory details).
>>> 
>>> For mmap, we can mostly do the right thing in the kernel, except for
>>> MAP_FIXED, where the user has to know what they're doing again.
>>> 
>>> For the cases the user thinks they know best, we can't avoid giving out
>>> the knowledge somehow, because inequivalent aliases in writeable
>>> mappings will HPMC a system.  We could be more relaxed about
>>> inequivalent aliases in read only mappings (say shared libraries), but
>>> the consequence of that is an explosion in the use of cache space, so we
>>> would want some libraries (like glibc) with many shared copies to obey
>>> SHMLBA.
>> 
>> 
>> I agree with Helge.  We run out of memory too quickly with 4 MB.  This resulted in various
>> userspace applications failing as mentioned by Helge.
>> 
>> MAP_FIXED can fail it the address is a problem.  I believe we check for that.
>> 
>> If you look at the pch implementation in gcc, you will see that the MAP_FIXED problem can be
>> worked around, and this problem is not specific to parisc.  The pch data has to mapped at the same
>> address as it was originally created as there is no way to relocate the data.
>> 
>> SHMLBA is 4096 /* (1 << PGSHIFT) */ on hpux.
>> 
>> The following is in <asm-generic/shmparam.h>:
>> #define SHMLBA PAGE_SIZE	 /* attach addr a multiple of this */
>> 
>> Shared mappings are handled with 
>> asm/shmparam.h:#define SHM_COLOUR 0x00400000	/* shared mappings colouring */
> 
> So how is the sys-v ipc problem fixed?  There the user is told to select
> an address which is a multiple of SHMLBA.  Programs that do this today
> will start to break on writeable mappings if we set SHMLBA to PAGE_SIZE
> because the colour will be wrong.


The code returns -EINVAL.  See arch_get_unmapped_area.

Dave
--
John David Anglin	dave.anglin@bell.net

Re: parisc: fix mmap(MAP_FIXED|MAP_SHARED) to already mmapped address

[James Bottomley]

On Sun, 2015-02-22 at 12:54 -0500, John David Anglin wrote:
> On 2015-02-22, at 12:17 PM, James Bottomley wrote:
> 
> > On Sun, 2015-02-22 at 11:45 -0500, John David Anglin wrote:
> >> On 2015-02-21, at 6:57 PM, James Bottomley wrote:
> >> 
> >>> On Sun, 2015-02-22 at 00:26 +0100, Helge Deller wrote:
> >>>> On 22.02.2015 00:09, James Bottomley wrote:
> >>>>> On Sat, 2015-02-21 at 15:40 -0500, John David Anglin wrote:
> >>>>>> On 2015-02-21, at 3:31 PM, John David Anglin wrote:
> >>>>>> 
> >>>>>>> On 2015-02-20, at 4:36 PM, Carlos O'Donell wrote:
> >>>>>>> 
> >>>>>>>> On Thu, Apr 3, 2014 at 4:26 PM, Helge Deller <deller@gmx.de> wrote:
> >>>>>>>>> In current eglibc it's set to 0x00400000
> >>>>>>>>> That's what my eglibc-patch changes...
> >>>>>>>>> I'm currently building a eglibc on hpviz with SHMLBA set to 4096 (__getpagesize()).
> >>>>>>>> 
> >>>>>>>> Anyone object to me fixing this upstream by making SHMLBA match the kernel?
> >>>>>>>> 
> >>>>>>>> I plan to use a fixed value of 4096, since I never expect hppa
> >>>>>>>> userspace to have to care (even if the kernel uses superpages).
> >>>>>>> 
> >>>>>>> We currently use (__getpagesize ()) in Debian and this seems to be a common definition.
> >>>>>>> Is there a performance advantage in using 4096?
> >>>>>>> 
> >>>>>>>> 
> >>>>>>>> Please correct me if I'm wrong.
> >>>>>>> 
> >>>>>>> 
> >>>>>>> At one time, we thought this value needed to be 4 MB.  Helge was
> >>>>>> working on improving the mmap
> >>>>>>> allocation scheme but this work stalled after some improvement.  I
> >>>>>> can't remember the issues and how
> >>>>>>> they relate to SHMLBA.
> >>>>>> 
> >>>>>> 
> >>>>>> Actually, the number was 4 Mb (bit).
> >>>>> 
> >>>>> No, it was 4MB.  That's the cache equivalency stride on PA processors
> >>>>> because we have a VIPT cache.  The architectural requirement according
> >>>>> to the dreaded appendix F is 16MB but we were assured by the PA
> >>>>> architects that it was 4 because they never planned producing processors
> >>>>> that would require 16.  The actual meaning is it's the number of bits of
> >>>>> the virtual address that are significant in the virtual index.
> >>>>> 
> >>>> 
> >>>> Your following statement:
> >>>> 
> >>>>> The point of SHMLBA is that if the same physical page is mapped into two
> >>>>> different virtual addresses but the two addresses are equal, modulo
> >>>>> SHMLBA, then the L1 cache sees the equivalency and you can't get
> >>>>> inequivalent cache aliases for the page (two writes to the two different
> >>>>> addresses producing two separately dirty cache lines which can never
> >>>>> resolve).  This means that the virtual addresses of all shared mappings
> >>>>> have to be equal modulo SHMLBA for the caches not to alias.
> >>>> 
> >>>> With this you define SHMLBA to be the representative number which defines
> >>>> what the current cache equivalency stride of the kernel is, *and* which then can
> >>>> be used by userspace. I think this is a misinterpretation of SHMLBA (or at
> >>>> least a parisc-specific interpretation of SHMLBA), which is not like how it
> >>>> is used on other architectures with similar limitations.
> >>>> Userspace should not know the kernel/architecture specifics. Instead they
> >>>> should try to mmap() memory somewhere (e.g. 4KB aligned) and if they need shared mappings then
> >>>> kernel/glibc will return a corrected mapping address (modulo 4MB).
> >>>> I think this is important, since most userspace programs usually try to mmap at
> >>>> a multiple of SHMLBA with which we then run very soon out of userspace (with SHMLBA=4MB).
> >>>> This has been the issue with localedef in glibc (a strange coding which tries
> >>>> to be platform-specific with mmap-calculation). Because of that in the end
> >>>> it turned out to be best for parisc to have SHMLBA defined to 4kb (and not 4MB).
> >>>> 
> >>>> So, your statement above is correct, I would just not use "SHMLBA" in this term,
> >>>> but maybe "KERNEL_SHMLBA" instead.
> >>> 
> >>> Um, no, SHMLBA comes from the SYS-V IPC primitives.  They were stupid
> >>> enough to allow the user pick the address of the region of shared
> >>> memory, so the user had to know these architectural details and SHMLBA
> >>> encodes them (man shmat will give you the gory details).
> >>> 
> >>> For mmap, we can mostly do the right thing in the kernel, except for
> >>> MAP_FIXED, where the user has to know what they're doing again.
> >>> 
> >>> For the cases the user thinks they know best, we can't avoid giving out
> >>> the knowledge somehow, because inequivalent aliases in writeable
> >>> mappings will HPMC a system.  We could be more relaxed about
> >>> inequivalent aliases in read only mappings (say shared libraries), but
> >>> the consequence of that is an explosion in the use of cache space, so we
> >>> would want some libraries (like glibc) with many shared copies to obey
> >>> SHMLBA.
> >> 
> >> 
> >> I agree with Helge.  We run out of memory too quickly with 4 MB.  This resulted in various
> >> userspace applications failing as mentioned by Helge.
> >> 
> >> MAP_FIXED can fail it the address is a problem.  I believe we check for that.
> >> 
> >> If you look at the pch implementation in gcc, you will see that the MAP_FIXED problem can be
> >> worked around, and this problem is not specific to parisc.  The pch data has to mapped at the same
> >> address as it was originally created as there is no way to relocate the data.
> >> 
> >> SHMLBA is 4096 /* (1 << PGSHIFT) */ on hpux.
> >> 
> >> The following is in <asm-generic/shmparam.h>:
> >> #define SHMLBA PAGE_SIZE	 /* attach addr a multiple of this */
> >> 
> >> Shared mappings are handled with 
> >> asm/shmparam.h:#define SHM_COLOUR 0x00400000	/* shared mappings colouring */
> > 
> > So how is the sys-v ipc problem fixed?  There the user is told to select
> > an address which is a multiple of SHMLBA.  Programs that do this today
> > will start to break on writeable mappings if we set SHMLBA to PAGE_SIZE
> > because the colour will be wrong.
> 
> 
> The code returns -EINVAL.  See arch_get_unmapped_area.

But that's not a solution.  Let me try to illustrate: I have an existing
application, it uses sys-v ipc and selects a shmat address based on the
multiple of SHMLBA for a writeable mapping.  Today it works.  Tomorrow
when you make this change, it fails with -EINVAL.  That's breaking an
existing application because chances are the app will just report the
failure and exit.

James

Re: parisc: fix mmap(MAP_FIXED|MAP_SHARED) to already mmapped address

[Helge Deller]

On 22.02.2015 18:58, James Bottomley wrote:
> On Sun, 2015-02-22 at 12:54 -0500, John David Anglin wrote:
>> On 2015-02-22, at 12:17 PM, James Bottomley wrote:
>>
>>> On Sun, 2015-02-22 at 11:45 -0500, John David Anglin wrote:
>>>> On 2015-02-21, at 6:57 PM, James Bottomley wrote:
>>>>
>>>>> On Sun, 2015-02-22 at 00:26 +0100, Helge Deller wrote:
>>>>>> On 22.02.2015 00:09, James Bottomley wrote:
>>>>>>> On Sat, 2015-02-21 at 15:40 -0500, John David Anglin wrote:
>>>>>>>> On 2015-02-21, at 3:31 PM, John David Anglin wrote:
>>>>>>>>
>>>>>>>>> On 2015-02-20, at 4:36 PM, Carlos O'Donell wrote:
>>>>>>>>>
>>>>>>>>>> On Thu, Apr 3, 2014 at 4:26 PM, Helge Deller <deller@gmx.de> wrote:
>>>>>>>>>>> In current eglibc it's set to 0x00400000
>>>>>>>>>>> That's what my eglibc-patch changes...
>>>>>>>>>>> I'm currently building a eglibc on hpviz with SHMLBA set to 4096 (__getpagesize()).
>>>>>>>>>>
>>>>>>>>>> Anyone object to me fixing this upstream by making SHMLBA match the kernel?
>>>>>>>>>>
>>>>>>>>>> I plan to use a fixed value of 4096, since I never expect hppa
>>>>>>>>>> userspace to have to care (even if the kernel uses superpages).
>>>>>>>>>
>>>>>>>>> We currently use (__getpagesize ()) in Debian and this seems to be a common definition.
>>>>>>>>> Is there a performance advantage in using 4096?
>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Please correct me if I'm wrong.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> At one time, we thought this value needed to be 4 MB.  Helge was
>>>>>>>> working on improving the mmap
>>>>>>>>> allocation scheme but this work stalled after some improvement.  I
>>>>>>>> can't remember the issues and how
>>>>>>>>> they relate to SHMLBA.
>>>>>>>>
>>>>>>>>
>>>>>>>> Actually, the number was 4 Mb (bit).
>>>>>>>
>>>>>>> No, it was 4MB.  That's the cache equivalency stride on PA processors
>>>>>>> because we have a VIPT cache.  The architectural requirement according
>>>>>>> to the dreaded appendix F is 16MB but we were assured by the PA
>>>>>>> architects that it was 4 because they never planned producing processors
>>>>>>> that would require 16.  The actual meaning is it's the number of bits of
>>>>>>> the virtual address that are significant in the virtual index.
>>>>>>>
>>>>>>
>>>>>> Your following statement:
>>>>>>
>>>>>>> The point of SHMLBA is that if the same physical page is mapped into two
>>>>>>> different virtual addresses but the two addresses are equal, modulo
>>>>>>> SHMLBA, then the L1 cache sees the equivalency and you can't get
>>>>>>> inequivalent cache aliases for the page (two writes to the two different
>>>>>>> addresses producing two separately dirty cache lines which can never
>>>>>>> resolve).  This means that the virtual addresses of all shared mappings
>>>>>>> have to be equal modulo SHMLBA for the caches not to alias.
>>>>>>
>>>>>> With this you define SHMLBA to be the representative number which defines
>>>>>> what the current cache equivalency stride of the kernel is, *and* which then can
>>>>>> be used by userspace. I think this is a misinterpretation of SHMLBA (or at
>>>>>> least a parisc-specific interpretation of SHMLBA), which is not like how it
>>>>>> is used on other architectures with similar limitations.
>>>>>> Userspace should not know the kernel/architecture specifics. Instead they
>>>>>> should try to mmap() memory somewhere (e.g. 4KB aligned) and if they need shared mappings then
>>>>>> kernel/glibc will return a corrected mapping address (modulo 4MB).
>>>>>> I think this is important, since most userspace programs usually try to mmap at
>>>>>> a multiple of SHMLBA with which we then run very soon out of userspace (with SHMLBA=4MB).
>>>>>> This has been the issue with localedef in glibc (a strange coding which tries
>>>>>> to be platform-specific with mmap-calculation). Because of that in the end
>>>>>> it turned out to be best for parisc to have SHMLBA defined to 4kb (and not 4MB).
>>>>>>
>>>>>> So, your statement above is correct, I would just not use "SHMLBA" in this term,
>>>>>> but maybe "KERNEL_SHMLBA" instead.
>>>>>
>>>>> Um, no, SHMLBA comes from the SYS-V IPC primitives.  They were stupid
>>>>> enough to allow the user pick the address of the region of shared
>>>>> memory, so the user had to know these architectural details and SHMLBA
>>>>> encodes them (man shmat will give you the gory details).
>>>>>
>>>>> For mmap, we can mostly do the right thing in the kernel, except for
>>>>> MAP_FIXED, where the user has to know what they're doing again.
>>>>>
>>>>> For the cases the user thinks they know best, we can't avoid giving out
>>>>> the knowledge somehow, because inequivalent aliases in writeable
>>>>> mappings will HPMC a system.  We could be more relaxed about
>>>>> inequivalent aliases in read only mappings (say shared libraries), but
>>>>> the consequence of that is an explosion in the use of cache space, so we
>>>>> would want some libraries (like glibc) with many shared copies to obey
>>>>> SHMLBA.
>>>>
>>>>
>>>> I agree with Helge.  We run out of memory too quickly with 4 MB.  This resulted in various
>>>> userspace applications failing as mentioned by Helge.
>>>>
>>>> MAP_FIXED can fail it the address is a problem.  I believe we check for that.
>>>>
>>>> If you look at the pch implementation in gcc, you will see that the MAP_FIXED problem can be
>>>> worked around, and this problem is not specific to parisc.  The pch data has to mapped at the same
>>>> address as it was originally created as there is no way to relocate the data.
>>>>
>>>> SHMLBA is 4096 /* (1 << PGSHIFT) */ on hpux.
>>>>
>>>> The following is in <asm-generic/shmparam.h>:
>>>> #define SHMLBA PAGE_SIZE	 /* attach addr a multiple of this */
>>>>
>>>> Shared mappings are handled with
>>>> asm/shmparam.h:#define SHM_COLOUR 0x00400000	/* shared mappings colouring */
>>>
>>> So how is the sys-v ipc problem fixed?  There the user is told to select
>>> an address which is a multiple of SHMLBA.  Programs that do this today
>>> will start to break on writeable mappings if we set SHMLBA to PAGE_SIZE
>>> because the colour will be wrong.
>>
>>
>> The code returns -EINVAL.  See arch_get_unmapped_area.
>
> But that's not a solution.  Let me try to illustrate: I have an existing
> application, it uses sys-v ipc and selects a shmat address based on the
> multiple of SHMLBA for a writeable mapping.  Today it works.

It will work as well with SHMBLA=4096, if you just use SHM_RND too
(and most applications do have SHM_RND).
man shmat says:
  * If shmaddr isn't NULL and SHM_RND is specified in shmflg, the attach occurs at the address equal to shmaddr rounded down to the nearest multiple of SHMLBA.
* Otherwise, shmaddr must be a page-aligned address at which the attach occurs.

So, even here shmaddr is mentioned to be page-aligned (4k), not SHMLBA-aligned (4M in your case).

>Tomorrow
> when you make this change, it fails with -EINVAL.  That's breaking an
> existing application because chances are the app will just report the
> failure and exit.

Tomorrow?  This change is *already* implemented in eglibc since a year or
so and I don't see any applications which break because of SHMBLA=4096...

Helge

Re: parisc: fix mmap(MAP_FIXED|MAP_SHARED) to already mmapped address

[James Bottomley]

On Sun, 2015-02-22 at 19:07 +0100, Helge Deller wrote:
> >>>> SHMLBA is 4096 /* (1 << PGSHIFT) */ on hpux.
> >>>>
> >>>> The following is in <asm-generic/shmparam.h>:
> >>>> #define SHMLBA PAGE_SIZE	 /* attach addr a multiple of this */
> >>>>
> >>>> Shared mappings are handled with
> >>>> asm/shmparam.h:#define SHM_COLOUR 0x00400000	/* shared mappings colouring */
> >>>
> >>> So how is the sys-v ipc problem fixed?  There the user is told to select
> >>> an address which is a multiple of SHMLBA.  Programs that do this today
> >>> will start to break on writeable mappings if we set SHMLBA to PAGE_SIZE
> >>> because the colour will be wrong.
> >>
> >>
> >> The code returns -EINVAL.  See arch_get_unmapped_area.
> >
> > But that's not a solution.  Let me try to illustrate: I have an existing
> > application, it uses sys-v ipc and selects a shmat address based on the
> > multiple of SHMLBA for a writeable mapping.  Today it works.
> 
> It will work as well with SHMBLA=4096, if you just use SHM_RND too
> (and most applications do have SHM_RND).
> man shmat says:
>   * If shmaddr isn't NULL and SHM_RND is specified in shmflg, the attach occurs at the address equal to shmaddr rounded down to the nearest multiple of SHMLBA.
> * Otherwise, shmaddr must be a page-aligned address at which the attach occurs.
> 
> So, even here shmaddr is mentioned to be page-aligned (4k), not
> SHMLBA-aligned (4M in your case).

I think that part is x86.  All the other VI architectures impose their
VI colour constrainst in SHMLBA.  We're the odd one out because we have
a huge stride (everyone else is small multiples of pages).

But agree if no applications are affected, we can make the ABI change.

> >Tomorrow
> > when you make this change, it fails with -EINVAL.  That's breaking an
> > existing application because chances are the app will just report the
> > failure and exit.
> 
> Tomorrow?  This change is *already* implemented in eglibc since a year or
> so and I don't see any applications which break because of SHMBLA=4096...

Is eglibc a big enough sample to make the claim that no applications
will be broken?

James

Re: parisc: fix mmap(MAP_FIXED|MAP_SHARED) to already mmapped address

[Helge Deller]

On 22.02.2015 20:13, James Bottomley wrote:
> On Sun, 2015-02-22 at 19:07 +0100, Helge Deller wrote:
>>>>>> SHMLBA is 4096 /* (1 << PGSHIFT) */ on hpux.
>>>>>>
>>>>>> The following is in <asm-generic/shmparam.h>:
>>>>>> #define SHMLBA PAGE_SIZE	 /* attach addr a multiple of this */
>>>>>>
>>>>>> Shared mappings are handled with
>>>>>> asm/shmparam.h:#define SHM_COLOUR 0x00400000	/* shared mappings colouring */
>>>>>
>>>>> So how is the sys-v ipc problem fixed?  There the user is told to select
>>>>> an address which is a multiple of SHMLBA.  Programs that do this today
>>>>> will start to break on writeable mappings if we set SHMLBA to PAGE_SIZE
>>>>> because the colour will be wrong.
>>>>
>>>>
>>>> The code returns -EINVAL.  See arch_get_unmapped_area.
>>>
>>> But that's not a solution.  Let me try to illustrate: I have an existing
>>> application, it uses sys-v ipc and selects a shmat address based on the
>>> multiple of SHMLBA for a writeable mapping.  Today it works.
>>
>> It will work as well with SHMBLA=4096, if you just use SHM_RND too
>> (and most applications do have SHM_RND).
>> man shmat says:
>>    * If shmaddr isn't NULL and SHM_RND is specified in shmflg, the attach occurs at the address equal to shmaddr rounded down to the nearest multiple of SHMLBA.
>> * Otherwise, shmaddr must be a page-aligned address at which the attach occurs.
>>
>> So, even here shmaddr is mentioned to be page-aligned (4k), not
>> SHMLBA-aligned (4M in your case).
>
> I think that part is x86.  All the other VI architectures impose their
> VI colour constrainst in SHMLBA.  We're the odd one out because we have
> a huge stride (everyone else is small multiples of pages).
>
> But agree if no applications are affected, we can make the ABI change.
>
>>> Tomorrow
>>> when you make this change, it fails with -EINVAL.  That's breaking an
>>> existing application because chances are the app will just report the
>>> failure and exit.
>>
>> Tomorrow?  This change is *already* implemented in eglibc since a year or
>> so and I don't see any applications which break because of SHMBLA=4096...
>
> Is eglibc a big enough sample to make the claim that no applications
> will be broken?

Try yourself:
https://parisc.wiki.kernel.org/index.php/Debian_Ports_Installation
Just install debian 8.0 (aka unstable for hppa). KDE, Xfce, libreoffice,
and a all others packages just work out of the box.

Helge

Re: parisc: fix mmap(MAP_FIXED|MAP_SHARED) to already mmapped address

[James Bottomley]

On Sun, 2015-02-22 at 20:16 +0100, Helge Deller wrote:
> On 22.02.2015 20:13, James Bottomley wrote:
> > On Sun, 2015-02-22 at 19:07 +0100, Helge Deller wrote:
> >>>>>> SHMLBA is 4096 /* (1 << PGSHIFT) */ on hpux.
> >>>>>>
> >>>>>> The following is in <asm-generic/shmparam.h>:
> >>>>>> #define SHMLBA PAGE_SIZE	 /* attach addr a multiple of this */
> >>>>>>
> >>>>>> Shared mappings are handled with
> >>>>>> asm/shmparam.h:#define SHM_COLOUR 0x00400000	/* shared mappings colouring */
> >>>>>
> >>>>> So how is the sys-v ipc problem fixed?  There the user is told to select
> >>>>> an address which is a multiple of SHMLBA.  Programs that do this today
> >>>>> will start to break on writeable mappings if we set SHMLBA to PAGE_SIZE
> >>>>> because the colour will be wrong.
> >>>>
> >>>>
> >>>> The code returns -EINVAL.  See arch_get_unmapped_area.
> >>>
> >>> But that's not a solution.  Let me try to illustrate: I have an existing
> >>> application, it uses sys-v ipc and selects a shmat address based on the
> >>> multiple of SHMLBA for a writeable mapping.  Today it works.
> >>
> >> It will work as well with SHMBLA=4096, if you just use SHM_RND too
> >> (and most applications do have SHM_RND).
> >> man shmat says:
> >>    * If shmaddr isn't NULL and SHM_RND is specified in shmflg, the attach occurs at the address equal to shmaddr rounded down to the nearest multiple of SHMLBA.
> >> * Otherwise, shmaddr must be a page-aligned address at which the attach occurs.
> >>
> >> So, even here shmaddr is mentioned to be page-aligned (4k), not
> >> SHMLBA-aligned (4M in your case).
> >
> > I think that part is x86.  All the other VI architectures impose their
> > VI colour constrainst in SHMLBA.  We're the odd one out because we have
> > a huge stride (everyone else is small multiples of pages).
> >
> > But agree if no applications are affected, we can make the ABI change.
> >
> >>> Tomorrow
> >>> when you make this change, it fails with -EINVAL.  That's breaking an
> >>> existing application because chances are the app will just report the
> >>> failure and exit.
> >>
> >> Tomorrow?  This change is *already* implemented in eglibc since a year or
> >> so and I don't see any applications which break because of SHMBLA=4096...
> >
> > Is eglibc a big enough sample to make the claim that no applications
> > will be broken?
> 
> Try yourself:
> https://parisc.wiki.kernel.org/index.php/Debian_Ports_Installation
> Just install debian 8.0 (aka unstable for hppa). KDE, Xfce, libreoffice,
> and a all others packages just work out of the box.

I know, I do, but that's sort of expected: most modern linux apps use
mmap.  It's the older stuff that uses sys-v ipc, but perhaps for us this
doesn't matter.

James

Re: parisc: fix mmap(MAP_FIXED|MAP_SHARED) to already mmapped address

[Carlos O'Donell]

On Sun, Feb 22, 2015 at 2:42 PM, James Bottomley
<James.Bottomley@hansenpartnership.com> wrote:
> I know, I do, but that's sort of expected: most modern linux apps use
> mmap.  It's the older stuff that uses sys-v ipc, but perhaps for us this
> doesn't matter.

And that's the real truth.

The usage patterns that appear to matter are:
(a) An initial mmap, followed by an mmap with MAP_FIXED at the same
previous address.
(b) shmat with SHM_RND, in which case we get to round the resulting
address or return EINVAL.

Both of (a) and (b) work today if glibc sets SHMLBA to 4kb.

On parisc we simply can't deal with the user selecting an arbitrary address.

Cheers,
Carlos.

Re: parisc: fix mmap(MAP_FIXED|MAP_SHARED) to already mmapped address - optimized patches

[Helge Deller]

On 22.02.2015 18:54, John David Anglin wrote:
> At one time, we thought this value needed to be 4 MB.
> Helge was working on improving the mmap allocation scheme but this
> work stalled after some improvement.

The patches are still available:
http://git.kernel.org/cgit/linux/kernel/git/deller/parisc-linux.git/commit/?h=parisc-mmap&id=34ae0a4620b50d27ce2f1314322275cbea7f2055
and
http://git.kernel.org/cgit/linux/kernel/git/deller/parisc-linux.git/commit/?h=parisc-mmap&id=7a6e51ddfd3ab3b11a4ebdd995e26672e69a8efa

Basically the idea is:
- Currently we have a static calculation where the mapping should happen inside the 4MB range:
   see: arch/parisc/kernel/sys_parisc.c: (filp ? ((unsigned long) filp->f_mapping) >> 8 : 0UL)
- Replace that by a dynamic mapping, which searches best fit address in free mem area *if* the file hasn't been mapped yet, and save this mapping in the struct address_space. If another process then maps the same file again, then just reuse the last calculated "dynamic" mapping (offset).

This helps a lot to prevent userspace memory fragmentation, but Linus didn't liked this approach and proposed instead:
  https://lkml.org/lkml/2014/5/1/368

Sadly his patch didn't worked out of the box. I did tried various ways (I'm sure it can work somehow), but I couldn't solve it yet.
Maybe some Linux mm expert could help here?

Helge   

Re: parisc: fix mmap(MAP_FIXED|MAP_SHARED) to already mmapped address

[James Bottomley]

On Sun, 2015-02-22 at 11:45 -0500, John David Anglin wrote:
> On 2015-02-21, at 6:57 PM, James Bottomley wrote:
> 
> > On Sun, 2015-02-22 at 00:26 +0100, Helge Deller wrote:
> >> On 22.02.2015 00:09, James Bottomley wrote:
> >>> On Sat, 2015-02-21 at 15:40 -0500, John David Anglin wrote:
> >>>> On 2015-02-21, at 3:31 PM, John David Anglin wrote:
> >>>> 
> >>>>> On 2015-02-20, at 4:36 PM, Carlos O'Donell wrote:
> >>>>> 
> >>>>>> On Thu, Apr 3, 2014 at 4:26 PM, Helge Deller <deller@gmx.de> wrote:
> >>>>>>> In current eglibc it's set to 0x00400000
> >>>>>>> That's what my eglibc-patch changes...
> >>>>>>> I'm currently building a eglibc on hpviz with SHMLBA set to 4096 (__getpagesize()).
> >>>>>> 
> >>>>>> Anyone object to me fixing this upstream by making SHMLBA match the kernel?
> >>>>>> 
> >>>>>> I plan to use a fixed value of 4096, since I never expect hppa
> >>>>>> userspace to have to care (even if the kernel uses superpages).
> >>>>> 
> >>>>> We currently use (__getpagesize ()) in Debian and this seems to be a common definition.
> >>>>> Is there a performance advantage in using 4096?
> >>>>> 
> >>>>>> 
> >>>>>> Please correct me if I'm wrong.
> >>>>> 
> >>>>> 
> >>>>> At one time, we thought this value needed to be 4 MB.  Helge was
> >>>> working on improving the mmap
> >>>>> allocation scheme but this work stalled after some improvement.  I
> >>>> can't remember the issues and how
> >>>>> they relate to SHMLBA.
> >>>> 
> >>>> 
> >>>> Actually, the number was 4 Mb (bit).
> >>> 
> >>> No, it was 4MB.  That's the cache equivalency stride on PA processors
> >>> because we have a VIPT cache.  The architectural requirement according
> >>> to the dreaded appendix F is 16MB but we were assured by the PA
> >>> architects that it was 4 because they never planned producing processors
> >>> that would require 16.  The actual meaning is it's the number of bits of
> >>> the virtual address that are significant in the virtual index.
> >>> 
> >> 
> >> Your following statement:
> >> 
> >>> The point of SHMLBA is that if the same physical page is mapped into two
> >>> different virtual addresses but the two addresses are equal, modulo
> >>> SHMLBA, then the L1 cache sees the equivalency and you can't get
> >>> inequivalent cache aliases for the page (two writes to the two different
> >>> addresses producing two separately dirty cache lines which can never
> >>> resolve).  This means that the virtual addresses of all shared mappings
> >>> have to be equal modulo SHMLBA for the caches not to alias.
> >> 
> >> With this you define SHMLBA to be the representative number which defines
> >> what the current cache equivalency stride of the kernel is, *and* which then can
> >> be used by userspace. I think this is a misinterpretation of SHMLBA (or at
> >> least a parisc-specific interpretation of SHMLBA), which is not like how it
> >> is used on other architectures with similar limitations.
> >> Userspace should not know the kernel/architecture specifics. Instead they
> >> should try to mmap() memory somewhere (e.g. 4KB aligned) and if they need shared mappings then
> >> kernel/glibc will return a corrected mapping address (modulo 4MB).
> >> I think this is important, since most userspace programs usually try to mmap at
> >> a multiple of SHMLBA with which we then run very soon out of userspace (with SHMLBA=4MB).
> >> This has been the issue with localedef in glibc (a strange coding which tries
> >> to be platform-specific with mmap-calculation). Because of that in the end
> >> it turned out to be best for parisc to have SHMLBA defined to 4kb (and not 4MB).
> >> 
> >> So, your statement above is correct, I would just not use "SHMLBA" in this term,
> >> but maybe "KERNEL_SHMLBA" instead.
> > 
> > Um, no, SHMLBA comes from the SYS-V IPC primitives.  They were stupid
> > enough to allow the user pick the address of the region of shared
> > memory, so the user had to know these architectural details and SHMLBA
> > encodes them (man shmat will give you the gory details).
> > 
> > For mmap, we can mostly do the right thing in the kernel, except for
> > MAP_FIXED, where the user has to know what they're doing again.
> > 
> > For the cases the user thinks they know best, we can't avoid giving out
> > the knowledge somehow, because inequivalent aliases in writeable
> > mappings will HPMC a system.  We could be more relaxed about
> > inequivalent aliases in read only mappings (say shared libraries), but
> > the consequence of that is an explosion in the use of cache space, so we
> > would want some libraries (like glibc) with many shared copies to obey
> > SHMLBA.
> 
> 
> I agree with Helge.  We run out of memory too quickly with 4 MB.  This resulted in various
> userspace applications failing as mentioned by Helge.
> 
> MAP_FIXED can fail it the address is a problem.  I believe we check for that.
> 
> If you look at the pch implementation in gcc, you will see that the MAP_FIXED problem can be
> worked around, and this problem is not specific to parisc.  The pch data has to mapped at the same
> address as it was originally created as there is no way to relocate the data.
> 
> SHMLBA is 4096 /* (1 << PGSHIFT) */ on hpux.

Do we know what tricks hpux does in shmat() to pull this off?  Making
writeable mappings uncacheable might be one way.  Using the space bits
as part of the VI index generation would be another.  I think there were
also some space bit quadrant tricks hpux pulls, aren't there?

James

Re: parisc: fix mmap(MAP_FIXED|MAP_SHARED) to already mmapped address

[John David Anglin]

On 2015-02-22, at 12:28 PM, James Bottomley wrote:

> On Sun, 2015-02-22 at 11:45 -0500, John David Anglin wrote:
>> On 2015-02-21, at 6:57 PM, James Bottomley wrote:
>> 
>>> On Sun, 2015-02-22 at 00:26 +0100, Helge Deller wrote:
>>>> On 22.02.2015 00:09, James Bottomley wrote:
>>>>> On Sat, 2015-02-21 at 15:40 -0500, John David Anglin wrote:
>>>>>> On 2015-02-21, at 3:31 PM, John David Anglin wrote:
>>>>>> 
>>>>>>> On 2015-02-20, at 4:36 PM, Carlos O'Donell wrote:
>>>>>>> 
>>>>>>>> On Thu, Apr 3, 2014 at 4:26 PM, Helge Deller <deller@gmx.de> wrote:
>>>>>>>>> In current eglibc it's set to 0x00400000
>>>>>>>>> That's what my eglibc-patch changes...
>>>>>>>>> I'm currently building a eglibc on hpviz with SHMLBA set to 4096 (__getpagesize()).
>>>>>>>> 
>>>>>>>> Anyone object to me fixing this upstream by making SHMLBA match the kernel?
>>>>>>>> 
>>>>>>>> I plan to use a fixed value of 4096, since I never expect hppa
>>>>>>>> userspace to have to care (even if the kernel uses superpages).
>>>>>>> 
>>>>>>> We currently use (__getpagesize ()) in Debian and this seems to be a common definition.
>>>>>>> Is there a performance advantage in using 4096?
>>>>>>> 
>>>>>>>> 
>>>>>>>> Please correct me if I'm wrong.
>>>>>>> 
>>>>>>> 
>>>>>>> At one time, we thought this value needed to be 4 MB.  Helge was
>>>>>> working on improving the mmap
>>>>>>> allocation scheme but this work stalled after some improvement.  I
>>>>>> can't remember the issues and how
>>>>>>> they relate to SHMLBA.
>>>>>> 
>>>>>> 
>>>>>> Actually, the number was 4 Mb (bit).
>>>>> 
>>>>> No, it was 4MB.  That's the cache equivalency stride on PA processors
>>>>> because we have a VIPT cache.  The architectural requirement according
>>>>> to the dreaded appendix F is 16MB but we were assured by the PA
>>>>> architects that it was 4 because they never planned producing processors
>>>>> that would require 16.  The actual meaning is it's the number of bits of
>>>>> the virtual address that are significant in the virtual index.
>>>>> 
>>>> 
>>>> Your following statement:
>>>> 
>>>>> The point of SHMLBA is that if the same physical page is mapped into two
>>>>> different virtual addresses but the two addresses are equal, modulo
>>>>> SHMLBA, then the L1 cache sees the equivalency and you can't get
>>>>> inequivalent cache aliases for the page (two writes to the two different
>>>>> addresses producing two separately dirty cache lines which can never
>>>>> resolve).  This means that the virtual addresses of all shared mappings
>>>>> have to be equal modulo SHMLBA for the caches not to alias.
>>>> 
>>>> With this you define SHMLBA to be the representative number which defines
>>>> what the current cache equivalency stride of the kernel is, *and* which then can
>>>> be used by userspace. I think this is a misinterpretation of SHMLBA (or at
>>>> least a parisc-specific interpretation of SHMLBA), which is not like how it
>>>> is used on other architectures with similar limitations.
>>>> Userspace should not know the kernel/architecture specifics. Instead they
>>>> should try to mmap() memory somewhere (e.g. 4KB aligned) and if they need shared mappings then
>>>> kernel/glibc will return a corrected mapping address (modulo 4MB).
>>>> I think this is important, since most userspace programs usually try to mmap at
>>>> a multiple of SHMLBA with which we then run very soon out of userspace (with SHMLBA=4MB).
>>>> This has been the issue with localedef in glibc (a strange coding which tries
>>>> to be platform-specific with mmap-calculation). Because of that in the end
>>>> it turned out to be best for parisc to have SHMLBA defined to 4kb (and not 4MB).
>>>> 
>>>> So, your statement above is correct, I would just not use "SHMLBA" in this term,
>>>> but maybe "KERNEL_SHMLBA" instead.
>>> 
>>> Um, no, SHMLBA comes from the SYS-V IPC primitives.  They were stupid
>>> enough to allow the user pick the address of the region of shared
>>> memory, so the user had to know these architectural details and SHMLBA
>>> encodes them (man shmat will give you the gory details).
>>> 
>>> For mmap, we can mostly do the right thing in the kernel, except for
>>> MAP_FIXED, where the user has to know what they're doing again.
>>> 
>>> For the cases the user thinks they know best, we can't avoid giving out
>>> the knowledge somehow, because inequivalent aliases in writeable
>>> mappings will HPMC a system.  We could be more relaxed about
>>> inequivalent aliases in read only mappings (say shared libraries), but
>>> the consequence of that is an explosion in the use of cache space, so we
>>> would want some libraries (like glibc) with many shared copies to obey
>>> SHMLBA.
>> 
>> 
>> I agree with Helge.  We run out of memory too quickly with 4 MB.  This resulted in various
>> userspace applications failing as mentioned by Helge.
>> 
>> MAP_FIXED can fail it the address is a problem.  I believe we check for that.
>> 
>> If you look at the pch implementation in gcc, you will see that the MAP_FIXED problem can be
>> worked around, and this problem is not specific to parisc.  The pch data has to mapped at the same
>> address as it was originally created as there is no way to relocate the data.
>> 
>> SHMLBA is 4096 /* (1 << PGSHIFT) */ on hpux.
> 
> Do we know what tricks hpux does in shmat() to pull this off?  Making
> writeable mappings uncacheable might be one way.  Using the space bits
> as part of the VI index generation would be another.  I think there were
> also some space bit quadrant tricks hpux pulls, aren't there?


I don't know anything about the internal details.  I believed that shared mappings of shared libraries normally in
one quadrant and private mappings in another quadrant.  One can't place a break point in a shared library when
the mapping is shared.

Dave
--
John David Anglin	dave.anglin@bell.net

Re: parisc: fix mmap(MAP_FIXED|MAP_SHARED) to already mmapped address

[Helge Deller]

On 21.02.2015 21:31, John David Anglin wrote:
> On 2015-02-20, at 4:36 PM, Carlos O'Donell wrote:
>
>> On Thu, Apr 3, 2014 at 4:26 PM, Helge Deller <deller@gmx.de> wrote:
>>> In current eglibc it's set to 0x00400000
>>> That's what my eglibc-patch changes...
>>> I'm currently building a eglibc on hpviz with SHMLBA set to 4096 (__getpagesize()).
>>
>> Anyone object to me fixing this upstream by making SHMLBA match the kernel?

I think to remember that that would be best.
Didn't I pushed this (SHMLBA=4k) to eglibc?

>> I plan to use a fixed value of 4096, since I never expect hppa
>> userspace to have to care (even if the kernel uses superpages).

Yes.

> We currently use (__getpagesize ()) in Debian and this seems to be a common definition.
> Is there a performance advantage in using 4096?

Yes, we have:
Index: eglibc-2.18/ports/sysdeps/unix/sysv/linux/hppa/bits/shm.h
===================================================================
--- eglibc-2.18.orig/ports/sysdeps/unix/sysv/linux/hppa/bits/shm.h      2013-01-09 19:28:48.000000000 -0500
+++ eglibc-2.18/ports/sysdeps/unix/sysv/linux/hppa/bits/shm.h   2014-04-09 21:08:06.516202583 -0400
@@ -36,7 +36,7 @@
  #define SHM_UNLOCK     12              /* unlock segment (root only) */

  /* Segment low boundary address multiple.  */
-#define SHMLBA 0x00400000              /* address needs to be 4 Mb aligned */
+#define SHMLBA         (__getpagesize ())

I prefer this for glibc as well.

> At one time, we thought this value needed to be 4 MB.  Helge was working on improving the mmap
> allocation scheme but this work stalled after some improvement.

They are just on hold, but not that needed any more after we have the flexmap implementation.

> I can't remember the issues and how they relate to SHMLBA.

I would need to recall too. IIRC, SHMLBA and the 4MB cache alignments do not really relate to each
other. Userspace should just mmap(X) where X is aligned to SHMLBA. The mmap will then
mmap to the next possible address (starting somewhere above/equal to X) and still fulfilling the 4MB cache
alignment (if it's a shared mmap).

Helge

Re: parisc: fix mmap(MAP_FIXED|MAP_SHARED) to already mmapped address

[John David Anglin]

On 4/3/2014 3:41 PM, Helge Deller wrote:
> And the attached patch for kernel isn't necessary, but makes it clear that the colouring is important.
Regarding the kernel patch, I see you have used the British/Canadian 
spelling
of "colour" :-)

I thinking changing SHMLBA does affect things in other parts of the kernel.
That's why I was uncertain whether it would work.

Dave

-- 
John David Anglin    dave.anglin@bell.net

Re: parisc: fix mmap(MAP_FIXED|MAP_SHARED) to already mmapped address

[Helge Deller]

On 04/03/2014 10:12 PM, John David Anglin wrote:
> On 4/3/2014 3:41 PM, Helge Deller wrote:
>> And the attached patch for kernel isn't necessary, but makes it clear that the colouring is important.
> Regarding the kernel patch, I see you have used the British/Canadian spelling
> of "colour" :-)

Yeah - seems to be commonly used in similiar parts of the kernel for other arches.
If you have a better idea/naming, please let me know.
 
> I thinking changing SHMLBA does affect things in other parts of the kernel.
> That's why I was uncertain whether it would work.

I scanned it. It's used very little, and shouldn't affect us.

Helge

Re: parisc: fix mmap(MAP_FIXED|MAP_SHARED) to already mmapped address

[Jeroen Roovers]

On Thu, 3 Apr 2014 16:12:03 -0400
John David Anglin <dave.anglin@bell.net> wrote:

> On 4/3/2014 3:41 PM, Helge Deller wrote:
> Regarding the kernel patch, I see you have used the British/Canadian 
> spelling of "colour" :-)

Also:

+#define SHM_COLOUR 0x00400000	/* shared mappings coulouring */

coulouring => colouring


     jer

Re: parisc: fix mmap(MAP_FIXED|MAP_SHARED) to already mmapped address

[Helge Deller]

Hello Aaro,

On 12/19/2013 08:17 PM, Aaro Koskinen wrote:
> This commit (0576da2c08e3d332f1b0653030d28ab804585ab6) and the current
> mainline kernel (3.13-rc4) gives me the following with GLIBC 2.18:
> 
> $ localedef -c -i en_US -f UTF-8 en_US.UTF-8
> cannot map archive header: Invalid argument
> 
> strace looks like this:
> 
> mmap2(NULL, 536870912, PROT_NONE, MAP_SHARED, 3, 0) = 0x42f34000
> mmap2(0x43000000, 1607632, PROT_READ|PROT_WRITE, MAP_SHARED|MAP_FIXED, 3, 0) = -1 EINVAL (Invalid argument)
> 
> With the patch reverted, it works:
> 
> mmap2(NULL, 536870912, PROT_NONE, MAP_SHARED, 3, 0) = 0x42d74000
> mmap2(0x43000000, 1607632, PROT_READ|PROT_WRITE, MAP_SHARED|MAP_FIXED, 3, 0) = 0x43000000

It maybe doesn't fail now, but it's not correctly mapped either.
So, even if you don't get the EINVAL error, it might not work properly.
The patch I send just corrected this wrong behaviour.

> BTW, note that for GLIBC 2.18 some changes were done regarding this:
> https://sourceware.org/bugzilla/show_bug.cgi?id=10283

Mike Frysinger already mentioned something like that some time ago:
http://www.spinics.net/lists/linux-parisc/msg05204.html

I think we need to take a look at the glibc again...

Helge

Re: parisc: fix mmap(MAP_FIXED|MAP_SHARED) to already mmapped address

[Aaro Koskinen]

Hi,

On Thu, Dec 19, 2013 at 09:28:41PM +0100, Helge Deller wrote:
> On 12/19/2013 08:17 PM, Aaro Koskinen wrote:
> > This commit (0576da2c08e3d332f1b0653030d28ab804585ab6) and the current
> > mainline kernel (3.13-rc4) gives me the following with GLIBC 2.18:
> > 
> > $ localedef -c -i en_US -f UTF-8 en_US.UTF-8
> > cannot map archive header: Invalid argument
> > 
> > strace looks like this:
> > 
> > mmap2(NULL, 536870912, PROT_NONE, MAP_SHARED, 3, 0) = 0x42f34000
> > mmap2(0x43000000, 1607632, PROT_READ|PROT_WRITE, MAP_SHARED|MAP_FIXED, 3, 0) = -1 EINVAL (Invalid argument)
> > 
> > With the patch reverted, it works:
> > 
> > mmap2(NULL, 536870912, PROT_NONE, MAP_SHARED, 3, 0) = 0x42d74000
> > mmap2(0x43000000, 1607632, PROT_READ|PROT_WRITE, MAP_SHARED|MAP_FIXED, 3, 0) = 0x43000000
> 
> It maybe doesn't fail now, but it's not correctly mapped either.
> So, even if you don't get the EINVAL error, it might not work properly.
> The patch I send just corrected this wrong behaviour.

The generated locale-archive seemed to be correct (md5sum matched my other
machines). Anyway, I'll avoid running localedef on parisc for time being.

Thanks for info,

A.

Re: parisc: fix mmap(MAP_FIXED|MAP_SHARED) to already mmapped address

[Rolf Eike Beer]

[-- Attachment #1: Type: text/plain, Size: 697 bytes --]

Am Donnerstag, 19. Dezember 2013, 21:17:51 schrieb Aaro Koskinen:
> Hi,
> 
> This commit (0576da2c08e3d332f1b0653030d28ab804585ab6) and the current
> mainline kernel (3.13-rc4) gives me the following with GLIBC 2.18:

I also suspect this patch of causing trouble. My C8000 regularly submits to 
the libarchive dashboard (http://my.cdash.org/index.php?project=libarchive). 
Since I upgraded to 3.12.5 one test (libarchive_test_compat_mac) keeps 
failing. The day after compile errors and other problems started (just browse 
through the last 2 weeks and you will see it). Then I reverted that change and 
rebootet. The compat_mac test still fails, but the other failures are gone.

Greetings,

Eike

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: parisc: fix mmap(MAP_FIXED|MAP_SHARED) to already mmapped address

[John David Anglin]

On 23-Dec-13, at 3:34 PM, Rolf Eike Beer wrote:

> Am Donnerstag, 19. Dezember 2013, 21:17:51 schrieb Aaro Koskinen:
>> Hi,
>>
>> This commit (0576da2c08e3d332f1b0653030d28ab804585ab6) and the  
>> current
>> mainline kernel (3.13-rc4) gives me the following with GLIBC 2.18:
>
> I also suspect this patch of causing trouble. My C8000 regularly  
> submits to
> the libarchive dashboard (http://my.cdash.org/index.php?project=libarchive 
> ).
> Since I upgraded to 3.12.5 one test (libarchive_test_compat_mac) keeps
> failing. The day after compile errors and other problems started  
> (just browse
> through the last 2 weeks and you will see it). Then I reverted that  
> change and
> rebootet. The compat_mac test still fails, but the other failures  
> are gone.

The libarchive_test_compat_mac test also appears to fail on x86.  It  
didn't fail
in my 3.1.2 build with kernel change.

I believe the GLIBC 2.18 change was wrong but I don't see why it would  
cause
compile errors.  I have compiled many packages with the kernel change  
and not
seen problems that I would attribute to it.

I did a test build of libarchive and saw following fails:

Failing tests:
   20: test_archive_read_close_twice_open_fd (3 failures)
   21: test_archive_read_close_twice_open_filename (3 failures)

This was with 3.1.2.  Looking at the logs, it didn't appear that these  
were mmap issues.

We need more data.

Dave
--
John David Anglin	dave.anglin@bell.net

Re: parisc: fix mmap(MAP_FIXED|MAP_SHARED) to already mmapped address

[Rolf Eike Beer]

[-- Attachment #1: Type: text/plain, Size: 1142 bytes --]

Am Montag, 23. Dezember 2013, 21:39:54 schrieb John David Anglin:
> On 23-Dec-13, at 3:34 PM, Rolf Eike Beer wrote:
> > Am Donnerstag, 19. Dezember 2013, 21:17:51 schrieb Aaro Koskinen:
> >> Hi,
> >> 
> >> This commit (0576da2c08e3d332f1b0653030d28ab804585ab6) and the
> >> current
> > 
> >> mainline kernel (3.13-rc4) gives me the following with GLIBC 2.18:
> > I also suspect this patch of causing trouble. My C8000 regularly
> > submits to
> > the libarchive dashboard (http://my.cdash.org/index.php?project=libarchive
> > ).
> > Since I upgraded to 3.12.5 one test (libarchive_test_compat_mac) keeps
> > failing. The day after compile errors and other problems started
> > (just browse
> > through the last 2 weeks and you will see it). Then I reverted that
> > change and
> > rebootet. The compat_mac test still fails, but the other failures
> > are gone.
> 
> The libarchive_test_compat_mac test also appears to fail on x86.  It
> didn't fail
> in my 3.1.2 build with kernel change.
> 
> I believe the GLIBC 2.18 change was wrong but I don't see why it would
> cause compile errors.

I am on glibc 2.17, I only changed the kernel.

Eike

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: parisc: fix mmap(MAP_FIXED|MAP_SHARED) to already mmapped address

[Rolf Eike Beer]

Am 23.12.2013 21:34, schrieb Rolf Eike Beer:
> Am Donnerstag, 19. Dezember 2013, 21:17:51 schrieb Aaro Koskinen:
>> Hi,
>> 
>> This commit (0576da2c08e3d332f1b0653030d28ab804585ab6) and the current
>> mainline kernel (3.13-rc4) gives me the following with GLIBC 2.18:
> 
> I also suspect this patch of causing trouble. My C8000 regularly 
> submits to
> the libarchive dashboard 
> (http://my.cdash.org/index.php?project=libarchive).
> Since I upgraded to 3.12.5 one test (libarchive_test_compat_mac) keeps
> failing. The day after compile errors and other problems started (just 
> browse
> through the last 2 weeks and you will see it). Then I reverted that 
> change and
> rebootet. The compat_mac test still fails, but the other failures are 
> gone.

Just for the records: the compat_mac test fails also on my x86_64.

At least the compile error in archive_read_support_format_zip.c is real, 
i.e. I can reproduce it on a 32 bit build on x86. No idea why it did not 
happen before.

Eike