Re: truncate head of file?

[Howard Chu <hyc@symas.com>]

Lukáš Czerner wrote:
> On Wed, 20 Aug 2014, Howard Chu wrote:
>
>> Date: Wed, 20 Aug 2014 00:00:56 -0700
>> From: Howard Chu <hyc@symas.com>
>> To: Lukáš Czerner <lczerner@redhat.com>
>> Cc: linux-fsdevel <linux-fsdevel@vger.kernel.org>
>> Subject: Re: truncate head of file?
>>
>> Lukáš Czerner wrote:
>>> On Tue, 19 Aug 2014, Howard Chu wrote:
>>>
>>>> Date: Tue, 19 Aug 2014 22:45:16 -0700
>>>> From: Howard Chu <hyc@symas.com>
>>>> To: linux-fsdevel <linux-fsdevel@vger.kernel.org>
>>>> Subject: truncate head of file?
>>>>
>>>> Was thinking it would be very handy to have a truncate() variant that
>>>> deletes
>>>> pages from the head of a file. This could be leveraged to make logfiles
>>>> easier
>>>> to maintain, as an example. Anyone else interested, think this would be
>>>> nice
>>>> to have?
>>>>
>>>> (Note - not the same as just punching holes in the beginning of the file -
>>>> we
>>>> want the beginning of the file to advance as well, past the deleted
>>>> pages.)
>>>
>>> I am not really sure I understand the behaviour you'd like to see.
>>> Can you please explain the behaviour including more concrete use
>>> case ?
>>
>> For example - we have a logfile (opened O_APPEND) that grows continuously. We
>> want to delete some old log info from the head of the file. We could use "hole
>> punching" to cause a specific range of data to be freed, but that just leaves
>> a sparse file. If we were to cat this file the read() would have to advance
>> thru all of that empty space before arriving at actual log data. We want both
>> the data to be freed and for the logical beginning of the file to be moved
>> forward, to match the location of where the remaining data begins.
>>
>> Freeing the space would be simplest if we just deallocate X pages from the
>> file, and then the beginning of the file becomes the beginning of the first
>> remaining page of the file.
>
> Ok, now I understand. It is exactly what FALLOC_FL_COLLAPSE_RANGE is
> for. It has been merged into mainline with commit v3.14-rc1-1-g00f5e61

Ah, thanks for the tip.

> However it will not work with O_APPEND nor does any other fallocate
> flag except pure fallocate, so not even punch hole would have
> worked.

Hm, that's a little bit inconvenient. What if the process calling fallocate() 
is different from the one appending to the log (i.e., using two separate file 
descriptors)? Also, when a range is collapsed, are all processes with 
descriptors opened on the file updated - are their seek positions correctly 
decremented? I dug up the patches and didn't see any code to handle this but 
may have missed it.

-- 
   -- Howard Chu
   CTO, Symas Corp.           http://www.symas.com
   Director, Highland Sun     http://highlandsun.com/hyc/
   Chief Architect, OpenLDAP  http://www.openldap.org/project/
--
To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html