[parisc-linux] unreliable mmap on 720

[parisc-linux] unreliable mmap on 720

[Jochen Friedrich]

Hi,

i recognized that mmap operations seem unreliable on HP720 platform. The 
following test program (taken from cyrus imap configure) demonstrates the 
problem:

#include <sys/types.h>
#include <sys/mman.h>
#include <fcntl.h>

main()
{
  char *base;
  int fd = open("testmmap", O_RDWR|O_CREAT|O_TRUNC, 0666);

  if (fd == -1)
  {
    printf("can't open testmmap\n");
    exit(1);
  }
  if (write(fd, "test", 4) != 4)
  {
    printf("can't write testmmap\n");
    exit(1);
  }
  fsync(fd);
  base = mmap((caddr_t)0, 100, PROT_READ, MAP_SHARED
#ifdef MAP_FILE
  | MAP_FILE
#endif
#ifdef MAP_VARIABLE
  | MAP_VARIABLE
#endif
  , fd, 0L);

  if (base == (caddr_t)-1)
  {
    printf("can't mmap testmmap\n");
    exit(1);
  }
  if (strncmp(base, "test", 4) != 0)
  {
    printf("memory not test\n");
    exit(1);
  }
  if (write(fd, "test", 4) != 4) 
  {
    printf("can't append testmmap\n");
    exit(1);
  }
  fsync(fd);
  if (strncmp(base+4, "test", 4) != 0)
  {
    printf("memory not testtest: %s\n", base);
    exit(1);
  }
  exit(0);
}

# ./testmmap
memory not testtest: test
# ./testmmap
# ./testmmap
# ./testmmap
# ./testmmap
# ./testmmap
# ./testmmap
# ./testmmap
# ./testmmap
# ./testmmap
# ./testmmap
memory not testtest: testtest

This only seems to happen on 720 (i couldn't replicate the bug on a 715 
with the same kernel booted up). Things seem to get worse if the load on 
the box rises (if the test is done on an idle machine, testmmap works 
without problems). Weird...

This is 2.4.19-pa3.

Cheers,
--jochen

Re: [parisc-linux] unreliable mmap

[Jochen Friedrich]

Hi,

> i recognized that mmap operations seem unreliable on HP720 platform. The
> following test program (taken from cyrus imap configure) demonstrates the
> problem:
> [...]
>
> This only seems to happen on 720 (i couldn't replicate the bug on a 715
> with the same kernel booted up).

While debugging cyrus21-imapd, i experienced pretty fishy behaviour
of the skiplist database backend. It turned out that the 715 is affected
by the mmap problem, as well.

Here is a new test program which performs random writes to a file and
verifies the mmapped memory afterwards which demonstrates the problem:

Cheers,
--jochen

-------------------------------------8<-------------------------------------
#include <stdio.h>
#include <stdlib.h>
#include <sys/types.h>
#include <sys/mman.h>
#include <sys/stat.h>
#include <fcntl.h>

int main()
{
  char *base;
  int fd;
  char buf[2000];
  int i, j;
  char c;

  for (i=0; i<2000; i++)
    {
      c = (char) ((float) rand() / (float) (RAND_MAX) * 26) + 'a';
      buf[i] = c;
    }

  fd = open("mmap.test", O_CREAT|O_TRUNC|O_RDWR, 0664);
  write(fd, buf, 2000);

  base = (char *)mmap((caddr_t)0, 2000, PROT_READ, MAP_SHARED
#ifdef MAP_FILE
| MAP_FILE
#endif
#ifdef MAP_VARIABLE
| MAP_VARIABLE
#endif
  , fd, 0L);

  for (i=0; i<2000; i++)
    {
      j = (int) ((float) rand() / (float) (RAND_MAX) * 2000);
      c = (char) ((float) rand() / (float) (RAND_MAX) * 26) + 'a';
      lseek(fd, j, SEEK_SET);
      write(fd, &c, 1);
      if (*(base+j) != c)
        printf("Error: mmap inconsistent at %d\n", j);
    }
  close(fd);
}

Re: [parisc-linux] unreliable mmap

[Randolph Chung]

> While debugging cyrus21-imapd, i experienced pretty fishy behaviour
> of the skiplist database backend. It turned out that the 715 is affected
> by the mmap problem, as well.

as are c3k, a500, etc....

no one has managed to really track this down yet. i can insert a dcache
flush into the kernel to 'fix' this, but it is not the right place to be
doing the flush. 

my understanding of what's happening is that the open(2) and mmap()
creates two different "views" of the underlying page. when a write()
flushes the contents, only the open() view is flushd but not the mmap
view.

unfortunately i don't know nearly enough about vm to be able to track
this down and fix it properly :(

randolph
--  
Randolph Chung
Debian GNU/Linux Developer, hppa/ia64 ports
http://www.tausq.org/

Re: [parisc-linux] unreliable mmap

[Jochen Friedrich]

Hi Randolph,

> no one has managed to really track this down yet. i can insert a dcache
> flush into the kernel to 'fix' this, but it is not the right place to be
> doing the flush.

I can confirm this. The following patch "fixes" the problem for me. Please
do NOT apply this dirty hack to CVS!

cvs server: Diffing .
Index: filemap.c
===================================================================
RCS file: /var/cvs/linux/mm/filemap.c,v
retrieving revision 1.27
diff -u -r1.27 filemap.c
--- filemap.c   4 Aug 2002 23:00:15 -0000       1.27
+++ filemap.c   3 Nov 2002 11:37:52 -0000
@@ -3075,6 +3075,7 @@
                        goto sync_failure;
                page_fault = __copy_from_user(kaddr+offset, buf, bytes);
                flush_dcache_page(page);
+               flush_cache_all();
                status = mapping->a_ops->commit_write(file, page, offset,
offset+bytes);
                if (page_fault)
                        goto fail_write;


--jochen