Re: [PATCH v2 0/7] large atomic writes for xfs

John Garry
2024-12-13T17:43:24Z

On 13/12/2024 17:22, Christoph Hellwig wrote:
> On Fri, Dec 13, 2024 at 05:15:55PM +0000, John Garry wrote:
>> Sure, so some background is that we are using atomic writes for innodb
>> MySQL so that we can stop relying on the double-write buffer for crash
>> protection. MySQL is using an internal 16K page size (so we want 16K atomic
>> writes).
>
> Make perfect sense so far.
>
>>
>> MySQL has what is known as a REDO log - see
>> https://dev.mysql.com/doc/dev/mysql-server/9.0.1/PAGE_INNODB_REDO_LOG.html
>>
>> Essentially it means that for any data page we write, ahead of time we do a
>> buffered 512B log update followed by a periodic fsync. I think that such a
>> thing is common to many apps.
>
> So it's actually using buffered I/O for that and not direct I/O?

Right

> >> When we tried just using 16K FS blocksize, we found for low thread
count
>> testing that performance was poor - even worse baseline of 4K FS blocksize
>> and double-write buffer. We put this down to high write latency for REDO
>> log. As you can imagine, mostly writing 16K for only a 512B update is not
>> efficient in terms of traffic generated and increased latency (versus 4K FS
>> block size). At higher thread count, performance was better. We put that
>> down to bigger log data portions to be written to REDO per FS block write.
>
> So if the redo log uses buffered I/O I can see how that would bloat writes.
> But then again using buffered I/O for a REDO log seems pretty silly
> to start with.
>

Yeah, at the low end, it may make sense to do the 512B write via DIO.
But OTOH sync'ing many redo log FS blocks at once at the high end can be
more efficient.

From what I have heard, this was attempted before (using DIO) by some
vendor, but did not come to much.

So it seems that we are stuck with this redo log limitation.

Let me know if you have any other ideas to avoid large atomic writes...

Cheers,
John