[OT] Wrapping memory.

[OT] Wrapping memory.

[Maciej Zenczykowski]

Hi All,

I have a pseudo-on-topic question:

I would like to have a 64 KBarray (of char), that's trivial, however what
I would like is for the last 4 KB [yes thankfully this is exactly one
page... (assume i386)] to reference the same physical memory as the first
four.

I.e. 16 4KB pages referencing physical 4 KB pages number 0..14, 0.

Is this at all possible? If so, how would I do this in user space (and
could it be done without root priv?)?

Thanks a lot,

Maciej Zenczykowski.

P.S. Yes, this is necessary, otherwise I have to give up on 32-bit access
(switch to 8-bit) and include mod 60KB in every memory access (very random
and I don't think I could predict when no to do this...]

Re: [OT] Wrapping memory.

[Alan Cox]

> I would like to have a 64 KBarray (of char), that's trivial, however what
> I would like is for the last 4 KB [yes thankfully this is exactly one
> page... (assume i386)] to reference the same physical memory as the first
> four.

Yep you can do that.

> Is this at all possible? If so, how would I do this in user space (and
> could it be done without root priv?)?

mmap will do what you need. Create a 60K object on disk and mmap it
at the base address and then 60K further on for 4K. 

Re: [OT] Wrapping memory.

[Benjamin LaHaise]

On Sat, Dec 01, 2001 at 10:24:58PM +0000, Alan Cox wrote:
> > Is this at all possible? If so, how would I do this in user space (and
> > could it be done without root priv?)?
> 
> mmap will do what you need. Create a 60K object on disk and mmap it
> at the base address and then 60K further on for 4K. 

And try to use /dev/shm/ first...

		-ben

Re: [OT] Wrapping memory.

[Christoph Rohland]

Hi Benjamin,

On Sat, 1 Dec 2001, Benjamin LaHaise wrote:
>> mmap will do what you need. Create a 60K object on disk and mmap it
>> at the base address and then 60K further on for 4K. 
> 
> And try to use /dev/shm/ first...

If you can live with glibc >= 2.2 and kernel 2.4 you should use
shm_open, shm_unlink for object creation and deletion.

Greetings
		Christoph

Re: [OT] Wrapping memory.

[David Woodhouse]

alan@lxorguk.ukuu.org.uk said:
> > I would like to have a 64 KBarray (of char), that's trivial, however
> > what I would like is for the last 4 KB [yes thankfully this is exactly
> > one page... (assume i386)] to reference the same physical memory as the
> > first four.

> mmap will do what you need. Create a 60K object on disk and mmap it at
> the base address and then 60K further on for 4K.  

You said 'assume i386', but just to make it clear - this is likely to break
horribly on some non-i386 platforms, due to dcache aliasing. You may find
that the second mmap(MAP_FIXED) fails, or if it succeeds then changes made
with one virtual address won't be instantly visible through the other
mapping. About the best case on such hardware is that Linux will just map
the offending page uncached.

--
dwmw2

Re: [OT] Wrapping memory.

[Christoph Rohland]

Hi David,

On Sun, 02 Dec 2001, David Woodhouse wrote:
> You said 'assume i386', but just to make it clear - this is likely
> to break horribly on some non-i386 platforms, due to dcache
> aliasing. 

Which platforms are affected by this?

Greetings
		Christoph

Re: [OT] Wrapping memory.

[David Woodhouse]

[-- Attachment #1: Type: text/plain, Size: 2186 bytes --]


cr@sap.com said:
> On Sun, 02 Dec 2001, David Woodhouse wrote:
> > You said 'assume i386', but just to make it clear - this is likely
> > to break horribly on some non-i386 platforms, due to dcache
> > aliasing. 
> Which platforms are affected by this? 

Anything with a virtually-indexed cache and a data capacity per way that's 
more than your page size - basically anything where it's possible for two 
different _virtual_ addresses for the same physical address to end up on 
different cache lines.

On SH3 you'll get away with it because the range of address space covered by
the cache is only 2KiB anyway - it's a 8KiB cache but 4-way associative, so
only the bottom 11 bits of the address affect the cache line used - and in
fact it doesn't matter that it's virtually indexed because unless you start
using 1KiB pages, those bits aren't changed by the MMU - only the offset
within the page defined the cache line you look in, and it doesn't matter
about the translation.

On SH4, the data capacity per way of the cache is 8KiB, and because we use 
4KiB pages we have two colours. See arch_get_unmapped_area() in 
arch/sh/kernel/sys_sh.c, and its usage in sys_mmap() and sys_mremap(). 
Basically we refuse to allow a mmap of a page to the 'wrong' colour, to 
avoid getting aliases.

ARM used to just break, but I pointed it out to Russell a while ago and I 
believe he fixed it. I don't remember what his fix was - it may have been 
just to map the offending page uncached, which is also a fairly effective 
was of avoiding cache aliasing :)

The MIPS kernel doesn't deal with this at all. Ralf tells me that the RM7000
and R[236]000 processors should be fine for the same reason as the SH3, but 
R[45]000 are likely to break.

I don't know about other architectures. 

A third possible approach other than just refusing the wrong-colour mapping
or disabling the cache entirely is to use the MMU to make sure only one
colour is accessible at a time - when you get a fault on the virtual address
of another colour, you flush the caches and swap over. You're not going to
do that without a physical->virtual lookup though, so it's basically not an
option for Linux atm.

--
dwmw2


[-- Attachment #2: aliastest.c --]
[-- Type: text/plain , Size: 1714 bytes --]

#include <sys/types.h>
#include <sys/stat.h>
#include <sys/mman.h>
#include <fcntl.h>
#include <unistd.h>
#include <stdlib.h>
#include <string.h>
#include <stdio.h>

#define PAGE_SIZE 4096

int main(int argc, char **argv)
{
	int fd, err;
	int towrite = PAGE_SIZE;
	char *p;
	volatile char *map1, *map2;
	unsigned char b[4];

	fd = open (argc>2?argv[1]:"/tmp/testfile", O_CREAT|O_TRUNC|O_RDWR, 0644);
	if (!fd<0) {
		perror("open");
		close(fd);
		return 1;
	}
	
	p = malloc(PAGE_SIZE);
	if (!p) {
		fprintf(stderr, "malloc failed\n");
		return 2;
	}
	memset(p, 0, PAGE_SIZE);

	while(towrite) {
		err = write(fd, p, PAGE_SIZE);

		if (err < 0) {
			perror("write");
			close(fd);
			return 3;
		}
		towrite -= err;
	}

	map1 = mmap(0, PAGE_SIZE, PROT_READ|PROT_WRITE, MAP_SHARED, fd, 0);
	if (!map1) {
		perror("mmap 1");
		close(fd);
		return 4;
	}
	/* Make sure the second mapping is _immediately_ after the first */
	map2 = mmap((void *)map1+PAGE_SIZE, PAGE_SIZE, PROT_READ|PROT_WRITE, MAP_SHARED|MAP_FIXED, fd, 0);
	if (!map2) {
		perror("mmap 2");
		munmap((void *)map1, PAGE_SIZE);
		close(fd);
		return 5;
	}
	
	printf("Mapped page twice, at 0x%08lx and 0x%08lx\n", 
	       (unsigned long)map1, (unsigned long) map2);

	/* Some arches (ARM) allocate cache lines only on read */
	(void)map1[0];
	(void)map2[0];

	map1[0] = 0x55;
	map2[1] = 0xaa;
	map1[2] = 0xa5;
	map2[3] = 0x5a;

	b[0] = map2[0];
	b[1] = map1[1];
	b[2] = map2[2];
	b[3] = map1[3];

	if (b[0] != 0x55 || b[1] != 0xaa || b[2] != 0xa5 || b[3] != 0x5a) {
		printf("Your cache is broken\n");
		return 6;
	} else {
		printf("Your cache appears to be OK\n");
	}

	munmap((void *)map1, PAGE_SIZE);
	munmap((void *)map2, PAGE_SIZE);
	close(fd);	
}

Re: [OT] Wrapping memory.

[Russell King]

On Mon, Dec 03, 2001 at 09:11:18AM +0000, David Woodhouse wrote:
> ARM used to just break, but I pointed it out to Russell a while ago and I 
> believe he fixed it. I don't remember what his fix was - it may have been 
> just to map the offending page uncached, which is also a fairly effective 
> was of avoiding cache aliasing :)

We actually still map the pages as cached, but when update_mmu_cache
detects that a page has been mmapped more than once, we ensure that
the other mappings in the current mm will fault when accessed.

--
Russell King (rmk@arm.linux.org.uk)                The developer of ARM Linux
             http://www.arm.linux.org.uk/personal/aboutme.html

Re: [OT] Wrapping memory.

[Jamie Lokier]

Russell King wrote:
> On Mon, Dec 03, 2001 at 09:11:18AM +0000, David Woodhouse wrote:
> > ARM used to just break, but I pointed it out to Russell a while ago and I 
> > believe he fixed it. I don't remember what his fix was - it may have been 
> > just to map the offending page uncached, which is also a fairly effective 
> > was of avoiding cache aliasing :)
> 
> We actually still map the pages as cached, but when update_mmu_cache
> detects that a page has been mmapped more than once, we ensure that
> the other mappings in the current mm will fault when accessed.

It should be possible, in a "portable" program, to map the two pages you
want and then test to see whether they are aliasing correctly.

Write to one and read from the other page, and vice versa.  Repeat a few
thousand times just in case you were interrupted in the middle.

That is my approach to creating circular buffers (which is the question
which started this thread).

Unfortunately, the update_mmu_cache makes aliasing work properly while
ruining performence, so then it's better to not to use the mapping trick
at all in that case.  To check for this, I have to call gettimeofday()
between pairs of accesses, to check whether they are slow.  I don't know
for sure if this works because I don't have an ARM to try it on.

-- Jamie

Re: [OT] Wrapping memory.

[Russell King]

On Tue, Dec 04, 2001 at 04:39:50PM +0000, Jamie Lokier wrote:
> Unfortunately, the update_mmu_cache makes aliasing work properly while
> ruining performence, so then it's better to not to use the mapping trick
> at all in that case.  To check for this, I have to call gettimeofday()
> between pairs of accesses, to check whether they are slow.  I don't know
> for sure if this works because I don't have an ARM to try it on.

Why not create a program and email it to someone with an ARM machine?

--
Russell King (rmk@arm.linux.org.uk)                The developer of ARM Linux
             http://www.arm.linux.org.uk/personal/aboutme.html

Re: [OT] Wrapping memory.

[David Woodhouse]

rmk@arm.linux.org.uk said:
>  We actually still map the pages as cached, but when update_mmu_cache
> detects that a page has been mmapped more than once, we ensure that
> the other mappings in the current mm will fault when accessed.

Oooh. Can you do that without having phys->virt lookup? And what about 
mappings in other mms? Or are the ARM caches so broken that you have to 
flush the whole damn thing on mm switch anyway?

VIVT. Urgh.

--
dwmw2

Re: [OT] Wrapping memory.

[Russell King]

On Tue, Dec 04, 2001 at 10:45:04AM +0000, David Woodhouse wrote:
> Oooh. Can you do that without having phys->virt lookup?

Yep.  But only because we have page->mapping->i_mmap_shared.

> And what about mappings in other mms? Or are the ARM caches so broken
> that you have to flush the whole damn thing on mm switch anyway?
> VIVT. Urgh.

Correct. ;(

--
Russell King (rmk@arm.linux.org.uk)                The developer of ARM Linux
             http://www.arm.linux.org.uk/personal/aboutme.html