[SPAM] Database performance is worse with (DIRECT IO)O_DIRECT

[SPAM] Database performance is worse with (DIRECT IO)O_DIRECT

[Yongkun Wang]

[-- Attachment #1.1: Type: text/plain, Size: 836 bytes --]

hi, guys

 

I have been using this awesome file system for a while. 

I built a database server on it. The transaction throughput on NILFS2 is about several times faster than that on EXT2.

 

However, when I set the flush method to DIRECT IO (O_DIRECT), the database performance on NILFS2 is even worse than that on EXT2 with DIRECT IO.

 

My conjecture is that probably the log-structured file system need a lot of buffer to apply changes and group the writes.

 

Is it also possible that the implementation of NILFS2 on DIRECT IO is  not very efficient?

 

Before I read the code, could you give me some hints why the performance is not good with DIRECT IO?

Or could you please introduce the implementation relevant to the DIRECT IO?

 

Thank you very much.

 

Best regards,

Yongkun

 


[-- Attachment #1.2: Type: text/html, Size: 3514 bytes --]

[-- Attachment #2: Type: text/plain, Size: 158 bytes --]

_______________________________________________
users mailing list
users-JrjvKiOkagjYtjvyW6yDsg@public.gmane.org
https://www.nilfs.org/mailman/listinfo/users

Re: [SPAM] Database performance is worse with (DIRECT IO)O_DIRECT

[Ryusuke Konishi]

Hi!
On Wed, 5 Aug 2009 22:59:43 +0900, "Yongkun Wang" wrote:
> hi, guys
> 
> I have been using this awesome file system for a while.
>
> I built a database server on it. The transaction throughput on
> NILFS2 is about several times faster than that on EXT2.
>
> However, when I set the flush method to DIRECT IO (O_DIRECT), the
> database performance on NILFS2 is even worse than that on EXT2 with
> DIRECT IO.

Interesting results.

Nilfs2 is actually not yet supporting O_DIRECT write because the core
routine of direct IO is not available for copy-on-write filesystems.
It is fallen back to a buffered write for nilfs.

O_DIRECT read is OK, but I took O_DIRECT write is implied in your
case.  Do you use postgresql ?

> My conjecture is that probably the log-structured file system need a
> lot of buffer to apply changes and group the writes.

Likely reason.  You can see what's happing on the partition by using
``dumpseg'' command; it will show summary of logs for a specified
segment.  ( You can track recently written segments with ``lssu''
command )

If your conjecture is right, you will see many files and meta data
files with inode number 4, 5, and 3, are written very frequently.

> Is it also possible that the implementation of NILFS2 on DIRECT IO
> is not very efficient?
>
> Before I read the code, could you give me some hints why the
> performance is not good with DIRECT IO?

Yes, as I mentioned above.

I think we can add native support of O_DIRECT to nilfs, but I suspect
there is a influential difference in how to flush out data between the
native direct-io and the fall-backed buffered write & sync; we may be
able to improve performance without the native o-direct support.


> Or could you please introduce the implementation relevant to the
> DIRECT IO?

Roughly speaking, I think the current nilfs implementation of O_DIRECT
leads to the same action as O_DSYNC.  To be exact, we need to analyze
the disk with the dumpseg or trace internal functions because the
behavior depends on the environment.

Cheers,
Ryusuke Konishi