mmap question

mmap question

[Der Herr Hofrat]

Hi !

 trying to write up a simple mmap for a pseudo device that accesses a 
 kmalloc'ed area.

 The driver is a character driver that only has mmap implemented - the kmalloc
 is done in init module and the pointer to the buffer is in global context.
 I expected to be able to write to the mmap'ed area from user-space but it
 never shows up in kernel space (the printk in driver_mmap always shows the
 init_msg passed in init_module). 

 the basic framework I'm using is below - can anybody point me to an obvious
 error or to some docs that would explain how to share an kmalloc'ed area
 with user-space via mmap ? 

thx !
hofrat
 
---driver---
char *kmalloc_area;
...
static int
driver_mmap(struct file *file,
	struct vm_area_struct *vma)
{
	vma->vm_flags |= VM_LOCKED|VM_SHARED;

	printk("message buffer: %s\",kmalloc_area); 
	remap_page_range(vma->vm_start,
		virt_to_phys(kmalloc_area),
		LEN,
		PAGE_SHARED);
	return 0;
}
	
static struct file_operations simple_fops={
    mmap:	driver_mmap,
};

int
init_module(void){
	...
	kmalloc_area=kmalloc(LEN,GFP_USER);
	strncpy(kmalloc_area,init_msg,sizeof(init_msg));
	...
}

---user-app---

int main(void)
{
	int fd;
	char msg[]="some message - should appear in kernel space";
	unsigned int *addr;

	if((fd=open("/dev/simple-device", O_RDWR))<0)
	addr = mmap(0, LEN, PROT_READ|PROT_WRITE, MAP_SHARED, fd, 0);
	memset(addr,0,LEN); 
	strncpy(addr,msg,sizeof(msg));
	return 0;
}

Re: mmap question

[Der Herr Hofrat]

> 
> int
> init_module(void){
> 	...
> 	kmalloc_area=kmalloc(LEN,GFP_USER);
> 	strncpy(kmalloc_area,init_msg,sizeof(init_msg));
> 	...
> }

found the problem (naturally after posting....) - forgot to mark 
the page as reserved...

	struct page *page;
	kmalloc_area=kmalloc(LEN,GFP_USER);
	page = virt_to_page(kmalloc_area); 
	mem_map_reserve(page);
	memcpy(kmalloc_area,msg,sizeof(msg));

hofrat

Re: mmap question

[Zhonghua Dai]

On Tuesday 24 September 2002 15:26, Der Herr Hofrat wrote:
> Hi !
>
>  trying to write up a simple mmap for a pseudo device that accesses a
>  kmalloc'ed area.
>
>  The driver is a character driver that only has mmap implemented - the
> kmalloc is done in init module and the pointer to the buffer is in global
> context. I expected to be able to write to the mmap'ed area from user-space
> but it never shows up in kernel space (the printk in driver_mmap always
> shows the init_msg passed in init_module).
>
>  the basic framework I'm using is below - can anybody point me to an
> obvious error or to some docs that would explain how to share an kmalloc'ed
> area with user-space via mmap ?
>
> thx !
> hofrat
>
> ---driver---
> char *kmalloc_area;
> ...
> static int
> driver_mmap(struct file *file,
> 	struct vm_area_struct *vma)
> {
> 	vma->vm_flags |= VM_LOCKED|VM_SHARED;
>
> 	printk("message buffer: %s\",kmalloc_area);
> 	remap_page_range(vma->vm_start,
> 		virt_to_phys(kmalloc_area),
> 		LEN,
> 		PAGE_SHARED);

Before memory can be exported into userspace, the reserved bit must be set. 
Call mem_map_reserve()  prior to remap_page_range().

good luck.


> 	return 0;
> }
>
> static struct file_operations simple_fops={
>     mmap:	driver_mmap,
> };
>
> int
> init_module(void){
> 	...
> 	kmalloc_area=kmalloc(LEN,GFP_USER);
> 	strncpy(kmalloc_area,init_msg,sizeof(init_msg));
> 	...
> }
>
> ---user-app---
>
> int main(void)
> {
> 	int fd;
> 	char msg[]="some message - should appear in kernel space";
> 	unsigned int *addr;
>
> 	if((fd=open("/dev/simple-device", O_RDWR))<0)
> 	addr = mmap(0, LEN, PROT_READ|PROT_WRITE, MAP_SHARED, fd, 0);
> 	memset(addr,0,LEN);
> 	strncpy(addr,msg,sizeof(msg));
> 	return 0;
> }
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/

mmap question

[Ruslan Nikolaev]

[-- Attachment #1: Type: text/plain, Size: 1151 bytes --]

OK. Possible I'll get free time next week and can do something :)

I have some questions:

1. Memory limit that was argued not long ago. I think that memory size
variable should be 64-bit size and also full mmap support would come... I
can try to do it.

2. Also I have a question about disk offsets. Now it is 32-bit and it's
OK for dos and apple maps but doesn't enough for another partitions map
such as LDM. Of course LDM doesn't available for GRUB now but perhaps it
will come in future. Replacing to 64-bit offsets seems to be not very
hard but a complex work. Also it seems that block_list stores block
number as 32-bit.

3. amd64 compilation. It seems to be not available now. One more question
I have... Linux gets device geometry througth the ioctl call. Does the
"long" type for "start
 field in "hd_geometry" struct (that used is util/i386/pc/biosdisk.c) is
always long type (that means it is 64-bit for amd64) or just 32-bit type?
I'm not Linux specialist and don't know...

-- 
___________________________________________________________
Sign-up for Ads Free at Mail.com
http://promo.mail.com/adsfreejump.htm


[-- Attachment #2: Type: text/html, Size: 1321 bytes --]

mmap question

[Eric Van Hensbergen]

I'm trying to implement a completely synchronous mmap in my
filesystem, but am running into some difficulty and was wondering if
someone could give me some insight/clue.

I want all my file system's operations to be complete uncached and
synchronous, but I also want to support mmap.  The problem is that
mmap writes don't seem to register right away.  If I have a test app:

        start = mmap(0, 8192, PROT_READ|PROT_WRITE, MAP_SHARED, fd, 0);
        if((int)start==-1) {
                perror("mmap:");
                exit(-1);
        }
        memcpy(buffer, start, 10);
        buffer[10] = 0;
        printf("reply: (%s)\n", buffer);
        sprintf(start, "hello world\n");
        memcpy(buffer, start, 10);
        buffer[10] = 0;
        printf("reply2: (%s)\n", buffer);
        munmap(start, 8192);
        close(fd);

I get the right results (because the read is also an mmap through the
buffer, but it looks like I don't see the file's set_page_dirty method
called until after I see the second read.  If I do use normal reads
(instead of mmap reads) I get the wrong results because my normal file
read method doesn't go through the page cache.

What am I doing wrong?  Is what I'm trying to do impossible, and if
so, how can I get as close as possible?

Thanks in advance for any help.

      -eric

Re: mmap question

[Bryan Henderson]

>I want all my file system's operations to be complete uncached and
>synchronous, but I also want to support mmap.
>...
>What am I doing wrong?  Is what I'm trying to do impossible, and if
>so, how can I get as close as possible?

It looks to me like you're running into the fundamental limitation that 
the CPU doesn't notify Linux every time you store into a memory location. 
It does, though, set the dirty flag in the page table, and Linux 
eventually inspects that flag and finds out that you have stored in the 
past.  At that time, it can call set_page_dirty.

Without knowing what properties of not having a cache you were hoping for, 
I couldn't say what alternative would be closest to this.

Hypothetically, if you had a backing storage device that could do memory 
mapped I/O, you could have mmapped direct I/O.

--
Bryan Henderson                          IBM Almaden Research Center
San Jose CA                              Filesystems

Re: mmap question

[Matthew Wilcox]

On Mon, Mar 21, 2005 at 01:33:55PM -0800, Bryan Henderson wrote:
> It looks to me like you're running into the fundamental limitation that 
> the CPU doesn't notify Linux every time you store into a memory location. 
> It does, though, set the dirty flag in the page table, and Linux 
> eventually inspects that flag and finds out that you have stored in the 
> past.  At that time, it can call set_page_dirty.

well, we *could* know ... never map this page writable.  have a per-vma
flag that says "emulate writes", and call the filesystem to update
backing storage before returning to the application.

the price of doing this would be exorbitant.

-- 
"Next the statesmen will invent cheap lies, putting the blame upon 
the nation that is attacked, and every man will be glad of those
conscience-soothing falsities, and will diligently study them, and refuse
to examine any refutations of them; and thus he will by and by convince 
himself that the war is just, and will thank God for the better sleep 
he enjoys after this process of grotesque self-deception." -- Mark Twain

Re: mmap question

[Bryan Henderson]

>well, we *could* know ... never map this page writable.  have a per-vma
>flag that says "emulate writes", and call the filesystem to update
>backing storage before returning to the application.

Ah yes, you mean, I take it, that the page fault handler would look at the 
user's program and emulate the faulting store instruction and return to 
the instruction after it.  Very clever.

And as long as we're going down that path, we should also consider 
changing exec() so instead of branching into the program, it just 
interprets it!

mmap question

[Eric Van Hensbergen]

On Mon, 21 Mar 2005 13:33:55 -0800, Bryan Henderson <hbryan@us.ibm.com> wrote:
> >I want all my file system's operations to be complete uncached and
> >synchronous, but I also want to support mmap.
> >...
> >What am I doing wrong?  Is what I'm trying to do impossible, and if
> >so, how can I get as close as possible?
>
> It looks to me like you're running into the fundamental limitation that
> the CPU doesn't notify Linux every time you store into a memory location.
> It does, though, set the dirty flag in the page table, and Linux
> eventually inspects that flag and finds out that you have stored in the
> past.  At that time, it can call set_page_dirty.
>

Is there an existing interface to force it to check if the page is
dirty -- for example, one way to maintain coherency would be for my
normal read/write path to check if the page is dirty and write it back
and invalidate the page before executing the normal-path read/write.
This may be an unlikely scenerio, but I think it is one of the corner
cases fsx tickles.

    -eric

Re: mmap question

[Bryan Henderson]

>Is there an existing interface to force it to check if the page is
>dirty

The msync() system call and libc function does that.  And then it does the 
same thing as fsync().

--
Bryan Henderson                          IBM Almaden Research Center
San Jose CA                              Filesystems

Re: mmap question

[Bryan Henderson]

I forgot you were talking about code inside the kernel.  In that case, 
filemap_sync().

Re: mmap question

[Eric Van Hensbergen]

Just wanted to thank everybody for their help.  I ended up backing off
the lofty goals and worked towards just passing fsx for now.  The main
thing needed was an invalidate_inode_pages2(inode->i_mapping) when I
wrote the file through the uncached method (in order to invalidate any
read buffers).  It turns out fsx was msync'ing the mmap writes so that
path was already clear.

I've got plenty of other fires to fight before I go after the more
extreme case, and perhaps direct I/O is the appropriate way to go
about it.

Thanks again.

         -eric

Re: mmap question

[Martin Jambor]

On Mon, 21 Mar 2005 11:59:51 -0600, Eric Van Hensbergen
<ericvh@gmail.com> wrote:
> 
> I want all my file system's operations to be complete uncached and
> synchronous, but I also want to support mmap.  

I am only a kernel newbie but I belive that you need a pagefault every
time the  memory is accessed. Don't know how to do that or if you
would need to be platform specific, though.

HTH

Martin Jambor

mmap question

[Frederic Gobry]

[-- Attachment #1: Type: text/plain, Size: 999 bytes --]

Hi,

I need to port an in-memory database so that it can use mtd
devices. The program currently uses mmap in order to access a file as
permanent storage. I discovered that using this capability is not
possible on jffs2, as mmap does not accept MAP_SHARED.

In my framework, write operations are performed by explicit calls, but
the data must be readable as if it were in a direct-access
memory. Would it be possible to do that on jffs2 (for instance,
open/write/close sessions to modify the data, but with immediate
update on the mmapped version of the file)

Alternatively, would it possible to use a raw MTD device to provide
the equivalent service ? I don't need filesystem semantic, as the
program already considers the data as a sequence of pages on a
flash-like device.

Thanks for any suggestion,

Frédéric

-- 
Frédéric Gobry    SMARTDATA    	  
      ---         http://www.smartdata.ch
Software Engineer Lausanne - Switzerland
                  +41 21 693 84 98

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: mmap question

[Frederic Gobry]

[-- Attachment #1: Type: text/plain, Size: 907 bytes --]

I follow-up to my own question: I've managed to use mmap with jffs2
quite successfully, but now I would like to directly use MTD (so that
I can handle the way sectors are erased / written to in order to
ensure good atomicity).

I've seen that the MTD char driver does not provide mmap support. In
order to implement it, I would like to know if I can use the 'point'
function of the mtd_info structure to get the physical address of the
flash device. From the documentation

  "For devices which are entirely memory-mapped and which can be
   mapped directly into user-space page tables, we may support
   execute-in-place (XIP) mapping of data on the flash. The precise
   semantics of this are yet to be defined, so it's probably best just
   not to either implement or attempt to use these two functions right
   at the moment."

So, is it the correct way to go ?

Thanks,

Frédéric

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: mmap question

[Frederic Gobry]

[-- Attachment #1: Type: text/plain, Size: 798 bytes --]

Hi,

I don't despair having an answer to at least one of my questions :-)

I implemented mmap on a mtd device (via a simple remap_page_range). I
have a userland program that opens and mmap a mtd device with success,
and can read-access it correctly. But once I perform a MEMERASE ioctl
or a simple write, the process only reads series of 0x80 bytes. If I
restart the process, it continues reading this value (even before any
erase/write operation) unless I reboot.

Do anybody have a clue about this behavior ?

My setup: ARM process (OMAP1510 board),  MTD device uses cfi_probe 

Thanks in advance,
Frédéric

-- 
Frédéric Gobry    SMARTDATA    	  
      ---         http://www.smartdata.ch
Software Engineer Lausanne - Switzerland
                  +41 21 693 84 98
				  

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: mmap question

[Jörn Engel]

Hi!

> I don't despair having an answer to at least one of my questions :-)

Hm. Maybe I should read/reread them. :-)

> I implemented mmap on a mtd device (via a simple remap_page_range). I
> have a userland program that opens and mmap a mtd device with success,
> and can read-access it correctly. But once I perform a MEMERASE ioctl
> or a simple write, the process only reads series of 0x80 bytes. If I
> restart the process, it continues reading this value (even before any
> erase/write operation) unless I reboot.
> 
> Do anybody have a clue about this behavior ?

0x80 is the status register, you are reading. The command set (Intel
or Amd, I suppose) can read it, handle any errors (0x80 mean no errors
on intel) and return to read mode by writing 0xff to the flash.

My guess is that you command set has a bug. Fix it or provide a spec
and some beer and pizza. ;-)

Jörn

-- 
Often, the most striking and innovative solutions come from realizing
that your concept of the problem was wrong.
-- Eric Raymond in "The cathedral and the bazaar"

Re: mmap question

[Frederic Gobry]

[-- Attachment #1: Type: text/plain, Size: 1958 bytes --]

> 0x80 is the status register, you are reading. The command set (Intel
> or Amd, I suppose) can read it, handle any errors (0x80 mean no errors
> on intel) and return to read mode by writing 0xff to the flash.

Mmmm, this enlightens the situation. I was fearing some cache problem...

> My guess is that you command set has a bug. Fix it or provide a spec
> and some beer and pizza. ;-)

The base code uses cfi_probe: it seemed to me the command set is
determined and provided automatically, isn't it ?

I think it's more because I interfere with the "canonical" way MTD
handles the flash: I access directly the memory after the erase or
write operation (via mmap, *not* via a read system call), but from the
code in do_read_onechip (cfi_cmdset_0001.c), it seems the flash is set
to read mode not when the erase or write operation is finished, but
before an actual read must be performed (which makes sense).

I can work around this for my program (I just tested, performing a
simple read just after the write restores what is seen in the mmapped
memory), but I still would like to know if the MTD API could be
augmented in order to handle read-only memory mapped areas (when
available) in a cleaner way as what I currently do?  I don't think
this would imply lots of changes:

      - a flag indicating if the flash can be mmapped

      - the implementation  of mmap (which probably would  need a call
        similar  to  'point'  but  with an  explicit  semantic  toward
        mmapping.

      - possibly another  flag to indicate  that an area  is currently
        mmapped, so that the erase / write operations set the flash in
        read mode once they have finished their duty

Frédéric

PS : where can I send the beer and pizza ? :-)

-- 
Frédéric Gobry    SMARTDATA    	  
      ---         http://www.smartdata.ch
Software Engineer Lausanne - Switzerland
                  +41 21 693 84 98
				  

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: mmap question

[Jörn Engel]

On Fri, 12 July 2002 09:57:16 +0200, Frederic Gobry wrote:
> The base code uses cfi_probe: it seemed to me the command set is
> determined and provided automatically, isn't it ?

Nyes, kind of. You configure the kernel to use one or more command
sets and the flash probing code picks the correct one out of those, if
present.

> I think it's more because I interfere with the "canonical" way MTD
> handles the flash: I access directly the memory after the erase or
> write operation (via mmap, *not* via a read system call), but from the
> code in do_read_onechip (cfi_cmdset_0001.c), it seems the flash is set
> to read mode not when the erase or write operation is finished, but
> before an actual read must be performed (which makes sense).

This helps performance a little, somewhere in the range of 5% for
large writes on the flashes I know. I have seen other code though,
that writes a sequence of 0xff 0x50 0xff into the flash before and
after every write operation, just to be sure the flash is in a sane
state. Slow, but effective.

> I can work around this for my program (I just tested, performing a
> simple read just after the write restores what is seen in the mmapped
> memory), but I still would like to know if the MTD API could be
> augmented in order to handle read-only memory mapped areas (when
> available) in a cleaner way as what I currently do?  I don't think
> this would imply lots of changes:
> 
>       - a flag indicating if the flash can be mmapped
> 
>       - the implementation  of mmap (which probably would  need a call
>         similar  to  'point'  but  with an  explicit  semantic  toward
>         mmapping.
> 
>       - possibly another  flag to indicate  that an area  is currently
>         mmapped, so that the erase / write operations set the flash in
>         read mode once they have finished their duty

BTW: Did you already try to mmap /dev/mtd* or /dev/mtdblock* ?

> PS : where can I send the beer and pizza ? :-)

>From Thursday on, I will be in Wedel, writing exams:
Joern Engel
Tinsdaler Weg 125
22880 Wedel
Germany

One possibility to order the pizza is Joeys:
http://www.joeys.de/joeys/65/index.htm

And if you actually do it, remember to cc David Woodhouse.

Jörn

-- 
"Translations are and will always be problematic. They inflict violence 
upon two languages." (translation from German)

Re: mmap question

[Frederic Gobry]

[-- Attachment #1: Type: text/plain, Size: 1846 bytes --]

Jörn said:
> BTW: Did you already try to mmap /dev/mtd* or /dev/mtdblock* ?

Sure, but neither does implement the mmap method.

> One possibility to order the pizza is Joeys:
> http://www.joeys.de/joeys/65/index.htm

Gargl, I'll need to find my french/german dictionnary first (I'm in
the french speaking part of switzerland !) Should I drop you an email
before, or is a pizza always welcome ?

David said:
> Er, ditch the crap 'point' semantics and implement something sane for 
> mmapping, because that's about the only sane use of point() anyway.

Ok. What was the idea of unpoint, BTW ?

> That's not sufficient. If you let the erase or write operations happen 
> while it's mapped, what prevents your app from reading from the flash 
> _during_ the erase/write operation? You need to either prevent the erases/
> writes from happening while the flash is mapped (bad) or arrange for the 
> pages mapped to userspace to be marked invalid during the erase/write 
> operation, and faults on those pages have to wait until the flash is back 
> in read mode again.

I admit I looked at it from the narrow point of view of my single
process which is safe by construction... I do agree that the only
acceptable solution is the second (disallow page access during write /
erase).

> We want some kind of mtd_remap() call which has its own struct vm_operations,
> with a nopage() which sets a flag in the state of the underlying mtd_info, 
> and ensure that the page can be found again and made invalid before any 
> other MTD operations. With the rmap VM the latter is quite simple, without 
> it we'd need to keep track of all vmas ourselves.

I'm quite a newcomer to VM management (my only experience comes from
"Linux Device Drivers" and what I tested for this driver...). What is
the rmap VM ?

Frédéric

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: mmap question

[David Woodhouse]

frederic.gobry@smartdata.ch said:
>  I can work around this for my program (I just tested, performing a
> simple read just after the write restores what is seen in the mmapped
> memory), but I still would like to know if the MTD API could be
> augmented in order to handle read-only memory mapped areas (when
> available) in a cleaner way as what I currently do?  I don't think
> this would imply lots of changes:

>       - a flag indicating if the flash can be mmapped

>       - the implementation  of mmap (which probably would  need a call
>         similar  to  'point'  but  with an  explicit  semantic  toward
>         mmapping.

Er, ditch the crap 'point' semantics and implement something sane for 
mmapping, because that's about the only sane use of point() anyway.

>       - possibly another  flag to indicate  that an area  is currently
>         mmapped, so that the erase / write operations set the flash in
>         read mode once they have finished their duty 

That's not sufficient. If you let the erase or write operations happen 
while it's mapped, what prevents your app from reading from the flash 
_during_ the erase/write operation? You need to either prevent the erases/
writes from happening while the flash is mapped (bad) or arrange for the 
pages mapped to userspace to be marked invalid during the erase/write 
operation, and faults on those pages have to wait until the flash is back 
in read mode again.

We want some kind of mtd_remap() call which has its own struct vm_operations,
with a nopage() which sets a flag in the state of the underlying mtd_info, 
and ensure that the page can be found again and made invalid before any 
other MTD operations. With the rmap VM the latter is quite simple, without 
it we'd need to keep track of all vmas ourselves.

This can be used from userspace via the mtdchar device's mmap() method, and 
also from kernelspace so that things like jffs2 can just remap the flash 
and read from it directly instead of having to make repeated mtd->read() 
calls. 

--
dwmw2