Postmark on NTFS

Postmark on NTFS

[Charles P. Wright]

Hello,

I'm running Postmark on NTFS and Ext2, and NTFS is surprisingly slow.  
(200k transactions, 20k concurrent files takes 28 minutes with 3 
minutes of system time).  I am trying to explain why.

Ext3 with journalled data takes around 20 minutes (compared to 2 for Ext2)
with only 20 seconds of system time.  This leads me to believe that at
least part of the issue is journalling.

However, when I increase the number of concurrent processes to 32 and
equally divide the work among them Ext3 drops to 13 minutes, but NTFS
stays the same (actually the elapsed time increases by 35 seconds).

I've done some reading on NTFS and it seems that since postmark is mostly
stressing file creation/deletion there will be a lot of activity to the
MFT, which is located in the first 12% of the disk.  On Ext2/3 the inode
tables are spread out in each cylinder group, so this could mean that NTFS
seeks more.

Can someone who is more familiar with NTFS internals propose other
alternatives, orvalidate some of this reasoning?

Thanks,
Charles

Re: Postmark on NTFS

[Szakacsits Szabolcs]

On Thu, 7 Aug 2003, Charles P. Wright wrote:

> I'm running Postmark on NTFS and Ext2, and NTFS is surprisingly slow.

Not surprising but it really depends on a lot of things,

- what OS, SP? Since the dog slow NT4, NTFS performance improved a lot.
- hardware: RAM size, disk size.
- how full is the disk
- how fragmented is MFT and the volume overall
- cluster size
- etc

The output of 'ntfsmeta -d device' could be a start. ntfsmeta is at
http://mlf.linux.rulez.org/mlf/ezaz/ntfsmeta.gz

> Ext3 with journalled data takes around 20 minutes (compared to 2 for Ext2)
> with only 20 seconds of system time.  This leads me to believe that at
> least part of the issue is journalling.

IMHO, definitely.

> I've done some reading on NTFS and it seems that since postmark is mostly
> stressing file creation/deletion there will be a lot of activity to the
> MFT, which is located in the first 12% of the disk.

Depends. MFT can be also scattered all over around on the volume.

> On Ext2/3 the inode tables are spread out in each cylinder group, so
> this could mean that NTFS seeks more.

Not necesarily but CPU usage could suggest.

	Szaka

Re: Postmark on NTFS

[Charles P. Wright]

Szaka,

Thanks for the quick reply, here are some answers to your questions.  

> On Thu, 7 Aug 2003, Charles P. Wright wrote:
> - what OS, SP? Since the dog slow NT4, NTFS performance improved a lot.
Windows Xp SP1.
> - hardware: RAM size, disk size.
512MB of RAM.  The disk is 20GB, but I created a 3GB partition at the end
of the disk to run the test.  I'm using a separate partition to minimize 
the seeking in order to achieve more consistent results.

> - how full is the disk
100MB is used.

> - how fragmented is MFT and the volume overall
I would hope that it isn't, since it is a fresh f/s.

> - cluster size
Volume name         :  Test Drive
NTFS major version  :  3
NTFS minor version  :  1
NTFS flags          :  0
Number of clusters  :  765095
Cluster size        :  4096
MFT record size     :  1024
LCN of $MFT         :  262144
LCN of $MFTMirr     :  382547
Num. of $MFT records:  82738

> - etc
> 
> The output of 'ntfsmeta -d device' could be a start. ntfsmeta is at
> http://mlf.linux.rulez.org/mlf/ezaz/ntfsmeta.gz
Where can I find the source for this?	

> > I've done some reading on NTFS and it seems that since postmark is mostly
> > stressing file creation/deletion there will be a lot of activity to the
> > MFT, which is located in the first 12% of the disk.
> 
> Depends. MFT can be also scattered all over around on the volume.
Even on a fresh f/s?

Loading $MFT... ntfs_pread(): Entering for pos 0x40000000, count 0x400.
ntfs_attr_open(): Entering for inode 0x0, attr 0x80.
ntfs_mapping_pairs_decompress(): Entering for attr 0x80.
Mapping pairs array successfully decompressed:
NTFS-fs DEBUG: Dumping runlist (values in hex):
VCN              LCN               Run length
0                40000             1460
1460             23f2a             3c70
50d0             LCN_ENOENT        0                (runlist end)
ntfs_attr_open(): Entering for inode 0x0, attr 0xb0.
OK

> > On Ext2/3 the inode tables are spread out in each cylinder group, so
> > this could mean that NTFS seeks more.
> 
> Not necesarily but CPU usage could suggest.
There is quite a bit more CPU time used by NTFS, but it still doesn't add
up to the difference in I/O times between Ext3 and NTFS, and why NTFS
doesn't interleave the CPU time with the I/O time when there are two
processes.

Does NTFS have more synchronous writes or reads than Ext3?

Thanks,
Charles

Re: Postmark on NTFS

[Szakacsits Szabolcs]

On Thu, 7 Aug 2003, Charles P. Wright wrote:

> > - what OS, SP? Since the dog slow NT4, NTFS performance improved a lot.
> Windows Xp SP1.

Ok, that's desktop OS. I would be surprised if MS used some throttling as
other times but it can't be excluded. However there are the Recycle Bin (or
whatever it's called), XP also has a feature to record how the blocks are
accessed in a file for readahed, etc so several things can go in the
background to help a desktop user.

Running Postmark is not a general usage scenario on XP. An MS server OS
with the recommended settings would be a more appropriate comparison, IMHO.

> > - hardware: RAM size, disk size.
> 512MB of RAM.  The disk is 20GB, but I created a 3GB partition at the end
> of the disk to run the test.

MS says bigger the volume the better (until one doesn't hit a limit). Since
XP/W2K3 they put the MFT at 3 GB from start because they claim/found this
gives 5-8% better overall performance.

BTW, you used the same partition for all fs, right?

> > - how full is the disk
> 100MB is used.

That's nothing and probably the volume was freshly created so it wasn't
overused (no fragmentation).

> > - how fragmented is MFT and the volume overall
> I would hope that it isn't, since it is a fresh f/s.

The MFT isn't fragmented (two fragments, quite normal).

> > The output of 'ntfsmeta -d device' could be a start. ntfsmeta is at
> > http://mlf.linux.rulez.org/mlf/ezaz/ntfsmeta.gz
> Where can I find the source for this?

On linux-ntfs-dev I offered several times but I was told anybody can write
it in the morning before breakfast ;) If you're interested I can put it
somewhere, after I had some time to clean it up a bit.

> > Depends. MFT can be also scattered all over around on the volume.
> Even on a fresh f/s?

No.

> > Not necesarily but CPU usage could suggest.
> There is quite a bit more CPU time used by NTFS,

I would say Windows. Who does really know? Somewhat I doubt you find your
answers here ... (even if one knows the source, not allowed to tell).

I think it's really pity, nobody is working on Linux NTFS support, now when
"everything" is known (except journaling but probably jbd could be used for
that, Linux specific). That would give a fair comparison and NTFS has
several features what none of the main Linux fs has today (here should come
Hans).

Cheers,

	Szaka

Re: Postmark on NTFS

[Anton Altaparmakov]

On Thu, 2003-08-07 at 20:08, Charles P. Wright wrote:
> Does NTFS have more synchronous writes or reads than Ext3?

Yes, loads more.  All metadata writes must either be synchronous OR they
must use double buffering.  Both cases make it slower than a simple
write to disk which can be asynchronous.

This is because metadata in ntfs is not written as it is in memory but
it is modified - fixups are applied which allow checking for
corruption/crashes on read from disk.

In the direct write case the metadata is then written to disk, and then
the modifications are undone.  Obviously, during the time in which the
metadata is modified, you cannot change the metadata as for all intents
and purposes it is corrupt and unusable.  This makes it impossible to
make any changes while a write is in progress and all changes are
stalled till the write is completed and the fixups are undone.

In the double buffered case the metadata is copied, then fixups applied,
then written to disk, and then thrown away.  There is one complication
and that is that the original in-memory metadata must be modified as
well to reflect the fact that it was just written to disk (the fixup
counter is incremented by one) so even here there is a short period of
time during which the original metadata cannot be touched by anyone
else.

I opted for the direct write approach in the Linux ntfs driver and as
far as I have understood the windows ntfs driver, it uses the direct
write approach, too.

Ext3 doesn't do such metadata fixups which allows its writes to be more
flexibly managed.

Best regards,

	Anton
-- 
Anton Altaparmakov <aia21 at cam.ac.uk> (replace at with @)
Unix Support, Computing Service, University of Cambridge, CB2 3QH, UK
Linux NTFS maintainer / IRC: #ntfs on irc.freenode.net
WWW: http://linux-ntfs.sf.net/ &
http://www-stu.christs.cam.ac.uk/~aia21/