[parisc-linux] mmap and GCC precompiled headers

[parisc-linux] mmap and GCC precompiled headers

[John David Anglin]

Precompiled headers (PCH) seem to be broken at the moment on hppa-linux
on at least certain kernel versions (2.6.6-pa4) because of the following
assumption in the PCH implementation:

   This version assumes that the kernel honors the START operand of mmap
   even without MAP_FIXED if START through START+SIZE are not currently
   mapped with something.  */

int
mmap_gt_pch_use_address (void *base, size_t size, int fd, size_t offset)
{

Anybody have any ideas as to whether parisc-linux can support the
above?  It seems to break down for large PCH files.  START isn't
honored under hpux either.

Dave
-- 
J. David Anglin                                  dave.anglin@nrc-cnrc.gc.ca
National Research Council of Canada              (613) 990-0752 (FAX: 952-6602)
_______________________________________________
parisc-linux mailing list
parisc-linux@lists.parisc-linux.org
http://lists.parisc-linux.org/mailman/listinfo/parisc-linux

Re: [parisc-linux] mmap and GCC precompiled headers

[Matthew Wilcox]

On Thu, Jul 15, 2004 at 04:25:01PM -0400, John David Anglin wrote:
> Precompiled headers (PCH) seem to be broken at the moment on hppa-linux
> on at least certain kernel versions (2.6.6-pa4) because of the following
> assumption in the PCH implementation:
> 
>    This version assumes that the kernel honors the START operand of mmap
>    even without MAP_FIXED if START through START+SIZE are not currently
>    mapped with something.  */
> 
> int
> mmap_gt_pch_use_address (void *base, size_t size, int fd, size_t offset)
> {
> 
> Anybody have any ideas as to whether parisc-linux can support the
> above?  It seems to break down for large PCH files.  START isn't
> honored under hpux either.

That's an unwarranted assumption.  If they want that, they need to
specify MAP_FIXED.

-- 
"Next the statesmen will invent cheap lies, putting the blame upon 
the nation that is attacked, and every man will be glad of those
conscience-soothing falsities, and will diligently study them, and refuse
to examine any refutations of them; and thus he will by and by convince 
himself that the war is just, and will thank God for the better sleep 
he enjoys after this process of grotesque self-deception." -- Mark Twain
_______________________________________________
parisc-linux mailing list
parisc-linux@lists.parisc-linux.org
http://lists.parisc-linux.org/mailman/listinfo/parisc-linux

Re: [parisc-linux] mmap and GCC precompiled headers

[John David Anglin]

> On Thu, Jul 15, 2004 at 04:25:01PM -0400, John David Anglin wrote:
> > Precompiled headers (PCH) seem to be broken at the moment on hppa-linux
> > on at least certain kernel versions (2.6.6-pa4) because of the following
> > assumption in the PCH implementation:
> > 
> >    This version assumes that the kernel honors the START operand of mmap
> >    even without MAP_FIXED if START through START+SIZE are not currently
> >    mapped with something.  */
> > 
> > int
> > mmap_gt_pch_use_address (void *base, size_t size, int fd, size_t offset)
> > {
> > 
> > Anybody have any ideas as to whether parisc-linux can support the
> > above?  It seems to break down for large PCH files.  START isn't
> > honored under hpux either.
> 
> That's an unwarranted assumption.  If they want that, they need to
> specify MAP_FIXED.

The problem with MAP_FIXED is that it replaces any previous mappings.
I believe that the code in question doesn't want to do that.  It
simply wants to fallback to not using PCH if the suggested value
for START doesn't work out.

At this point, I don't know if there is another mapping but I suspect
there isn't as the code tries to load the PCH file quite early.

Obviously, this isn't a problem on lots of other ports or everybody
would be screaming.

We can have hppa specific implementations of mmap_gt_pch_get_address
and mmap_gt_pch_use_address.  Under hpux, mmap_gt_pch_use_address was
failing because mmap returned a somewhat smaller memory address.

The address dependencies in the PCH file need to be fixed, but that's
non trivial.

Dave
-- 
J. David Anglin                                  dave.anglin@nrc-cnrc.gc.ca
National Research Council of Canada              (613) 990-0752 (FAX: 952-6602)
_______________________________________________
parisc-linux mailing list
parisc-linux@lists.parisc-linux.org
http://lists.parisc-linux.org/mailman/listinfo/parisc-linux

Re: [parisc-linux] mmap and GCC precompiled headers

[Randolph Chung]

> Obviously, this isn't a problem on lots of other ports or everybody
> would be screaming.

The reason for this is that the PA has strict aliasing requirements 
for virtual memory aliases, so we have to jump through hoops to make 
sure we don't create incorrectly aliased mappings. I'm afriad there's
not much that we can do about this.

randolph
-- 
Randolph Chung
Debian GNU/Linux Developer, hppa/ia64 ports
http://www.tausq.org/
_______________________________________________
parisc-linux mailing list
parisc-linux@lists.parisc-linux.org
http://lists.parisc-linux.org/mailman/listinfo/parisc-linux

Re: [parisc-linux] mmap and GCC precompiled headers

[Matthew Wilcox]

On Thu, Jul 15, 2004 at 05:34:02PM -0400, John David Anglin wrote:
> > That's an unwarranted assumption.  If they want that, they need to
> > specify MAP_FIXED.
> 
> The problem with MAP_FIXED is that it replaces any previous mappings.
> I believe that the code in question doesn't want to do that.  It
> simply wants to fallback to not using PCH if the suggested value
> for START doesn't work out.

Oh, that's easy enough.  mmap() returns the address it got mapped at,
so just compare it to the address you asked for and decline to use PCH
in that case.

> Obviously, this isn't a problem on lots of other ports or everybody
> would be screaming.

Sparc and MIPS should have difficulties too...

-- 
"Next the statesmen will invent cheap lies, putting the blame upon 
the nation that is attacked, and every man will be glad of those
conscience-soothing falsities, and will diligently study them, and refuse
to examine any refutations of them; and thus he will by and by convince 
himself that the war is just, and will thank God for the better sleep 
he enjoys after this process of grotesque self-deception." -- Mark Twain
_______________________________________________
parisc-linux mailing list
parisc-linux@lists.parisc-linux.org
http://lists.parisc-linux.org/mailman/listinfo/parisc-linux

Re: [parisc-linux] mmap and GCC precompiled headers

[John David Anglin]

> On Thu, Jul 15, 2004 at 05:34:02PM -0400, John David Anglin wrote:
> > > That's an unwarranted assumption.  If they want that, they need to
> > > specify MAP_FIXED.
> > 
> > The problem with MAP_FIXED is that it replaces any previous mappings.
> > I believe that the code in question doesn't want to do that.  It
> > simply wants to fallback to not using PCH if the suggested value
> > for START doesn't work out.
> 
> Oh, that's easy enough.  mmap() returns the address it got mapped at,
> so just compare it to the address you asked for and decline to use PCH
> in that case.

Well, that's exactly what it does.  However, it doesn't work.  The
start address is selected as follows:

/* Default version of HOST_HOOKS_GT_PCH_GET_ADDRESS when mmap is present.
   We temporarily allocate SIZE bytes, and let the kernel place the data
   wherever it will.  If it worked, that's our spot, if not we're likely
   to be in trouble.  */

However, when GCC tries to remap the PCH file to this location, it
sometimes fails.  This is currently a fatal error (i.e., it does not
fallback as described above).  This needs looking at.  Almost all
the libstdc++ tests failed in the last build that I did.  I think if
it successfully loads the PCH file and finds that it's not valid,
it does fallback.

>>From what you and Randolph have said, PCH in its current incarnation
isn't going to work on hppa.  If it works, I belive that it provides
a significant performance enhancement in compiling c++ code.

Dave
-- 
J. David Anglin                                  dave.anglin@nrc-cnrc.gc.ca
National Research Council of Canada              (613) 990-0752 (FAX: 952-6602)
_______________________________________________
parisc-linux mailing list
parisc-linux@lists.parisc-linux.org
http://lists.parisc-linux.org/mailman/listinfo/parisc-linux

Re: [parisc-linux] mmap and GCC precompiled headers

[Randolph Chung]

> However, when GCC tries to remap the PCH file to this location, it
> sometimes fails.  This is currently a fatal error (i.e., it does not
> fallback as described above).  This needs looking at.  Almost all
> the libstdc++ tests failed in the last build that I did.  I think if
> it successfully loads the PCH file and finds that it's not valid,
> it does fallback.

do you mean that, when the pch file is created, it was mmaped to some
address, and that address is written to the file. when the pch file is
read later on for another compile, it tries to mmap it to the same
location, and if it cannot, then it fails?

there's obviously a performance penalty, but is this an option (not very
sure it will work, but i think it might)?

1. mmap the file to addr1, where addr1 is any address returned by the
   kernel
2. do a MAP_PRIVATE anonymous mmap to the desired location addr2 (this 
   is more likely to be honoured by the kernel....)
3. memcpy the data from addr1 to addr2

obviously this assumes you are not changing the pch data...

randolph
-- 
Randolph Chung
Debian GNU/Linux Developer, hppa/ia64 ports
http://www.tausq.org/
_______________________________________________
parisc-linux mailing list
parisc-linux@lists.parisc-linux.org
http://lists.parisc-linux.org/mailman/listinfo/parisc-linux

Re: [parisc-linux] mmap and GCC precompiled headers

[John David Anglin]

> > However, when GCC tries to remap the PCH file to this location, it
> > sometimes fails.  This is currently a fatal error (i.e., it does not
> > fallback as described above).  This needs looking at.  Almost all
> > the libstdc++ tests failed in the last build that I did.  I think if
> > it successfully loads the PCH file and finds that it's not valid,
> > it does fallback.
> 
> do you mean that, when the pch file is created, it was mmaped to some
> address, and that address is written to the file. when the pch file is
> read later on for another compile, it tries to mmap it to the same
> location, and if it cannot, then it fails?

Yes.  I believe that it has to map it to the same address as when
it was created because the file contains pointers to addresses in
the PCH data that otherwise need adjustment.

The initial address is one selected by the kernel in a call to mmap
with START=0.  This is used to generate the PCH file.  Then, when
the program tries to read the pch file using the START address
returned by the kernel when the PCH file was built, the kernel
sometimes doesn't honor this start address.

I'm thinking that the current code should use or fallback to
trying MAP_FIXED when it tries to read the PCH file as it
currently fails with a fatal error if the region doesn't map
successfully.  If it's possible for there to be pages mapped
in the region needed for the PCH data, then the mmap call to
map the PCH file just isn't going to be reliable.  I doubt
that's actually happening.

What I think is happening is that the kernel doesn't try very
hard to honor START without MAP_FIXED.  There may be some subtle
issues dependent on the size of the file.  There is a configure
test in libstdc++ which attempts to see whether to enable
generating a PCH header file for libstdc++.  This started in
GCC 3.4.  The configure test always passes on both linux and
hpux.  However, I recently had on v3 testsuite run in which
all the tests failed because the libstcd++ PCH file couldn't
be read back.

As I noted previously, the downside to using MAP_FIXED is any
previous mapping get replaced.  Replacing a previous mapping
could lead to all kinds of wierd problems.  GCC makes rather
extensive use of mmap, so I can understand why the code tries
to avoid using MAP_FIXED.  It would be a significant overhead
to keep track of all mmapped regions.

> there's obviously a performance penalty, but is this an option (not very
> sure it will work, but i think it might)?
> 
> 1. mmap the file to addr1, where addr1 is any address returned by the
>    kernel
> 2. do a MAP_PRIVATE anonymous mmap to the desired location addr2 (this 
>    is more likely to be honoured by the kernel....)

It already uses MAP_PRIVATE.  Are anonymous mmaps more likely to
honor the START parameter?  If so, then we could use mmap to allocate
the region and just read the data into the region.

Dave
-- 
J. David Anglin                                  dave.anglin@nrc-cnrc.gc.ca
National Research Council of Canada              (613) 990-0752 (FAX: 952-6602)
_______________________________________________
parisc-linux mailing list
parisc-linux@lists.parisc-linux.org
http://lists.parisc-linux.org/mailman/listinfo/parisc-linux