Re: understanding speculative preallocation

Ben Myers
2013-07-26T21:42:42Z

Hi Jason,

On Fri, Jul 26, 2013 at 05:11:55PM -0400, Jason Rosenberg wrote:
> Is it safe to say that speculative preallocation will not be used if a file
> is opened read-only?

The blocks will only be reserved on an appending write.

> It turns out that the kafka server does indeed write lots of log files, and
> rotate them after they reach a max size, but never closes the files until
> the app exits, or until it deletes the files. This is because it needs to
> make them available for reading, etc. So, an obvious change for kafka
> might be to close each log file after rotating, and then re-open it
> read-only for consumers of the data. Does that sound like a solution that
> would pro-actively release pre-allocated storage?

An interesting idea, and I'm not quite sure. The blocks past EOF are freed in
xfs_release on close in some circumstances, and it looks like you have a chance
to call xfs_free_eofblocks (at least in the most uptodate codebase) if you did
not use explicit preallocation (e.g. fallocate or an xfs ioctl) and did not
open it append-only. You could reopen with read-write flags and it wouldn't
make a difference vs read-only, so long as you don't do an appending write.

Seems like it's worth a try. Another possibility is to look into what would
happen if you do a truncate up to i_size when you're ready to stop appending to
the file. I haven't checked that out though.

Regards,
Ben