truncate implementation

truncate implementation

[Jan Koss]

Hello.

I have question about "truncate" function implementation.

Let's suppose that we have file with 8K size.

One process (let's call it 'A') open file 'X' for read and write,
and mmap'ed it to memory with "MAP_SHARED" flag. After
that it read content of file, so all data of file go to "page cache".
And after that 'A' wait something.
At this moment another process (let's call it 'B')
call "truncate" function, and truncate 'X' file to 4K length.
After that process 'C' create file 'Y' and because of
second block of file 'X' now is free, kernel give it to file 'Y'.
And at last, process 'A' write to second page of file 'X' and call
'munmap' and 'close'. And 'C' also close file 'Y'.

Hence instead of writing to file 'X' 'A' will write to 'Y'.

In short
process 'A':

fd=open('X', O_RDWR)
p=mmap(MAP_SHARED, fd)
for (i=0; i<length(fd); ++i)
  putchar(p[i])
sleep(1)
process 'B': truncate('X')
process 'C': open('Y'), write('Y')

Process 'A':
	p[length(fd)-1]='Z'
	munmap(p)
process 'C' close('Y')<=here we MAY get wrong data in 'Y'.



I look at 'ext2_truncate' and 'sys_msync',
but still haven't see whole picture.

in ext2_truncate we call
sync_mapping_buffers(inode->i_mapping);

so as I understand all pages belong to file will be moved
from 'cache' to disk?
but what happend in:
>process 'A':
>	p[length(fd)-1]='Z'
>	munmap(p)

will be data wroten to disk or not, and if not where this
sitation is handled?

--
Kernelnewbies: Help each other learn about the Linux kernel.
Archive:       http://mail.nl.linux.org/kernelnewbies/
FAQ:           http://kernelnewbies.org/faq/

Re: truncate implementation

[Arjan van de Ven]

On Tue, 2006-01-17 at 11:18 +0300, Jan Koss wrote:
> Hello.
> 
> I have question about "truncate" function implementation.
> 
> Let's suppose that we have file with 8K size.
> 
> One process (let's call it 'A') open file 'X' for read and write,
> and mmap'ed it to memory with "MAP_SHARED" flag. After
> that it read content of file, so all data of file go to "page cache".
> And after that 'A' wait something.
> At this moment another process (let's call it 'B')
> call "truncate" function, and truncate 'X' file to 4K length.

.. and removes the page from the pagetable.

> After that process 'C' create file 'Y' and because of
> second block of file 'X' now is free, kernel give it to file 'Y'.
> And at last, process 'A' write to second page of file 'X' 

.. which isn't in the pagetable, and then the kernel notices it's
outside the file and causes a SIGBUS.

Re: truncate implementation

[Jan Koss]

Thanks for reply.

> .. which isn't in the pagetable, and then the kernel notices it's
> outside the file and causes a SIGBUS.
But if length of file after truncate is 4 byte, for example,
and I do p[5]=.., this didn't cause any error, so I can append some garbage
after end of file, and this is not error?

>
> .. and removes the page from the pagetable.
>
I'm sorry for stupid question, but where we remove from pagetable,
I see two places in ext2_truncate:

block_truncate_page
and
sync_mapping_buffers
but sync_mapping_buffer called only if inode_needs_sync(inode),

so p=mmap and reading file using p cause setting of "inode_need_sync" flag?

Re: truncate implementation

[Arjan van de Ven]

On Tue, 2006-01-17 at 12:21 +0300, Jan Koss wrote:
> Thanks for reply.
> 
> > .. which isn't in the pagetable, and then the kernel notices it's
> > outside the file and causes a SIGBUS.
> But if length of file after truncate is 4 byte, for example,
> and I do p[5]=.., this didn't cause any error, so I can append some garbage
> after end of file, and this is not error?

this is an artifact of mmap... those writes are not valid code
obviously, and the kernel just throws them away. Which afaik is allowed
by all standards.

> 
> >
> > .. and removes the page from the pagetable.
> >
> I'm sorry for stupid question, but where we remove from pagetable,
> I see two places in ext2_truncate:
> 

there is truncate_inode_pages.

remember this sort of stuff is done on the vfs level not on the
filesystem level in linux. the filesystem gets told in the end to adjust
it's own metadata, not to do stuff that belongs in higher layers

Re: truncate implementation

[Anton Altaparmakov]

On Tue, 2006-01-17 at 12:21 +0300, Jan Koss wrote:
> Thanks for reply.
> 
> > .. which isn't in the pagetable, and then the kernel notices it's
> > outside the file and causes a SIGBUS.
>
> But if length of file after truncate is 4 byte, for example,
> and I do p[5]=.., this didn't cause any error, so I can append some garbage
> after end of file, and this is not error?

No you can't.  You only think you can.  The dirty page is written using
->writepage of the file system which will "memset()" everything between
the end of file if it is in the page and the end of the page to zero
thus your write of p[5]=... will disappear every time the page is
written.

> > .. and removes the page from the pagetable.

> I'm sorry for stupid question, but where we remove from pagetable,
> I see two places in ext2_truncate:

It is not done in the file system.  It is done by the core kernel before
the file system ->truncate is called.  The function you want is
linux/mm/memory.c::vmtruncate().  The relevant lines are:

        i_size_write(inode, offset);
        unmap_mapping_range(mapping, offset + PAGE_SIZE - 1, 0, 1);
        truncate_inode_pages(mapping, offset);

Best regards,

        Anton
-- 
Anton Altaparmakov <aia21 at cam.ac.uk> (replace at with @)
Unix Support, Computing Service, University of Cambridge, CB2 3QH, UK
Linux NTFS maintainer / IRC: #ntfs on irc.freenode.net
WWW: http://linux-ntfs.sf.net/ & http://www-stu.christs.cam.ac.uk/~aia21/