ANN: New NTFS driver (2.0.0/TNG) now finished.

ANN: New NTFS driver (2.0.0/TNG) now finished.

[Anton Altaparmakov]

Hi,

This is to announce that the new NTFS Linux kernel driver 2.0.0 (formerly 
NTFS TNG) is now finished (read-only). It is for kernel 2.5.7 only and will 
be submitted to Linus for inclusion in the 2.5 kernel series when Linus 
returns from his holiday.

The driver has been tested extensively and has survived all tests so far.

It is fully compatible with kernel preemption and SMP. And it should work 
on both little endian and big endian architectures, and both on 32 and 64 
bit architectures. Note, only ia32 has actually been tested and there may 
be problems with architectures not supporting unaligned accesses. Any 
volunteers for non-ia32 architectures?

The new driver is significantly faster than the old driver (~20% in my 
tests), uses less CPU time and generally is superior to the old driver. (-:

The driver can be compiled both with gcc-2.95 and gcc-2.96. gcc-3.x has not 
been tested. (If anyone experiences compilation problems especially with 
gcc-2.95 please let me know and they will be fixed ASAP!)

To try the driver either use BitKeeper to obtain a clone from the repository:

         bk clone -q http://linux-ntfs.bkbits.net/ntfs-tng-2.5

Or if you already have a clone derived from an official kernel repository 
you only need to pull in the changes:

         bk pull http://linux-ntfs.bkbits.net/ntfs-tng-2.5

And then checkout all the files using bk -r co -q from within the 
repository directory.

For people not using BitKeeper patches for the standard 2.5.7 kernel are 
available here:

http://www-stu.christs.cam.ac.uk/~aia21/linux/linux-2.5.7-ntfs-2.0.0.patch.bz2 
(151kiB)

http://www-stu.christs.cam.ac.uk/~aia21/linux/linux-2.5.7-ntfs-2.0.0.patch.gz 
(199kiB)

http://www-stu.christs.cam.ac.uk/~aia21/linux/linux-2.5.7-ntfs-2.0.0.patch 
(796kiB)

Finally, for people wanting to browse the source code on-line, point your 
web browser at:

         http://linux-ntfs.bkbits.net:8080/ntfs-tng-2.5

Please everyone courageous enough to use a bleeding edge kernel and who is 
also an NTFS user give this a try and let me know if you encounter any 
problems! - Thanks!

Best regards,

Anton


-- 
   "I've not lost my mind. It's backed up on tape somewhere." - Unknown
-- 
Anton Altaparmakov <aia21 at cam.ac.uk> (replace at with @)
Linux NTFS Maintainer / WWW: http://linux-ntfs.sf.net/
ICQ: 8561279 / WWW: http://www-stu.christs.cam.ac.uk/~aia21/

Re: ANN: New NTFS driver (2.0.0/TNG) now finished.

[David Woodhouse]

aia21@cam.ac.uk said:
>  And it should work  on both little endian and big endian
> architectures, and both on 32 and 64  bit architectures. Note, only
> ia32 has actually been tested and there may  be problems with
> architectures not supporting unaligned accesses. Any  volunteers for
> non-ia32 architectures?

All architectures should support unaligned accesses.

--
dwmw2

Re: ANN: New NTFS driver (2.0.0/TNG) now finished.

[Christoph Hellwig]

On Mon, Mar 25, 2002 at 02:26:41AM +0000, Anton Altaparmakov wrote:
> The driver can be compiled both with gcc-2.95 and gcc-2.96. gcc-3.x has not 
> been tested. (If anyone experiences compilation problems especially with 
> gcc-2.95 please let me know and they will be fixed ASAP!)

gcc 2.95.2 works fine here.

Re: ANN: New NTFS driver (2.0.0/TNG) now finished.

[Brad Boyer]

David Woodhouse wrote:
> All architectures should support unaligned accesses.

Perhaps all architectures "should", but I can assure
you that many of them do no such thing. I didn't
look at every current architecture, but some notable
ones like the brand new IA64, as well as some older
chips such as MIPS have some relatively complicated
code in get_unaligned(), which can be found in the
appropriate include/asm-<arch> directory in the
file unaligned.h. I suspect that at least some of
these allow for an exception handler to fake the
capability in user space programs, but that isn't
something you can allow inside the kernel.

	Brad Boyer
	flar@allandria.com

Re: ANN: New NTFS driver (2.0.0/TNG) now finished.

[Alan Cox]

> file unaligned.h. I suspect that at least some of
> these allow for an exception handler to fake the
> capability in user space programs, but that isn't
> something you can allow inside the kernel.

The Linux kernel assumes and requires that a processor handles alignment
faults and fixups in kernel space. 

This is done because there are many cases where an object is almost always
aligned and it is faster to assume alignment than to mess around with every
single chunk of data. Its tuned so those alignment traps should not be
occurring at a high rate.

Alan

Re: ANN: New NTFS driver (2.0.0/TNG) now finished.

[Jeff Garzik]

David Woodhouse wrote:

>aia21@cam.ac.uk said:
>
>> And it should work  on both little endian and big endian
>>architectures, and both on 32 and 64  bit architectures. Note, only
>>ia32 has actually been tested and there may  be problems with
>>architectures not supporting unaligned accesses. Any  volunteers for
>>non-ia32 architectures?
>>
>
>All architectures should support unaligned accesses.
>

Yeah well... I continue to do get_unaligned() in my net drivers where 
prudent :)

    Jeff

Re: ANN: New NTFS driver (2.0.0/TNG) now finished.

[Grogan]

On Mon, 25 Mar 2002 02:26:41 +0000
Anton Altaparmakov <aia21@cam.ac.uk> wrote:

> Please everyone courageous enough to use a bleeding edge kernel and who is 
> also an NTFS user give this a try and let me know if you encounter any 
> problems! - Thanks!

Hello everyone

   I have been lurking here for a while on and off (to read the interesting discussions and learn). Since I've got a Windows XP installation on an NTFS partition on one of my hard disks, I went and got the 2.5.7 source and applied this patch last night.

It compiled cleanly with gcc 2.95.3 and is considerably faster than any of the old NTFS drivers. Before, there was a bit of a CPU intensive delay browsing directories in a file manager, like system32 and i386 with alot of files when they have to be initially read from disk. The perceived difference between the driver in 2.4.19-pre4 is certainly noticable on an older system like this one. (PII @266 with 128 Mb RAM). I figured I'd best measure something here, so I tried a bit of a test.

Now, I realize this test is flawed because it also tests file creation and buffers and cache and everything between the two kernels but it sure shows that someone is on the right track, somewhere.

On a fresh boot with 2.4.19-pre4:

bash-2.05$ time cp -r /mnt/windows3/windows/system32 /home/grogan/test

real    8m45.256s
user    0m0.730s
sys     6m27.030s

On a fresh boot with 2.5.7 with the new NTFS driver:

bash-2.05$ time cp -r /mnt/windows3/windows/system32 /home/grogan/test

real    3m13.190s
user    0m0.610s
sys     0m51.660s

This "test" was repeated twice under the same conditions, with negligible difference in the result (couple of seconds). Both of these disks are on the same IDE controller (/home is /dev/hda8 and the NTFS partition is /dev/hdb2). I must say I wasn't expecting such a drastic difference. The data appears to be intact. (correct size and number of files, anyway)

I wanted to send  a "thumbs up". Thanks, Anton, and everyone else that does what they do around here : - )

Grogan

On Mon, 25 Mar 2002 02:26:41 +0000
Anton Altaparmakov <aia21@cam.ac.uk> wrote:

> Please everyone courageous enough to use a bleeding edge kernel and who is 
> also an NTFS user give this a try and let me know if you encounter any 
> problems! - Thanks!
> 
> Best regards,
> 
> Anton
> 
> 
>

Re: ANN: New NTFS driver (2.0.0/TNG) now finished.

[Anton Altaparmakov]

At 17:17 25/03/02, Grogan wrote:
>On Mon, 25 Mar 2002 02:26:41 +0000
>Anton Altaparmakov <aia21@cam.ac.uk> wrote:
> > Please everyone courageous enough to use a bleeding edge kernel and who is
> > also an NTFS user give this a try and let me know if you encounter any
> > problems! - Thanks!
>
>Hello everyone
>
>    I have been lurking here for a while on and off (to read the 
> interesting discussions and learn). Since I've got a Windows XP 
> installation on an NTFS partition on one of my hard disks, I went and got 
> the 2.5.7 source and applied this patch last night.
>
>It compiled cleanly with gcc 2.95.3 and is considerably faster than any of 
>the old NTFS drivers. Before, there was a bit of a CPU intensive delay 
>browsing directories in a file manager, like system32 and i386 with alot 
>of files when they have to be initially read from disk. The perceived 
>difference between the driver in 2.4.19-pre4 is certainly noticable on an 
>older system like this one. (PII @266 with 128 Mb RAM). I figured I'd best 
>measure something here, so I tried a bit of a test.

Great to hear! (-:

>Now, I realize this test is flawed because it also tests file creation and 
>buffers and cache and everything between the two kernels but it sure shows 
>that someone is on the right track, somewhere.

Yes it is flawed indeed but even if you would compare 2.5.7/old ntfs with 
2.5.7/new ntfs you would notice significant speed improvements.

>On a fresh boot with 2.4.19-pre4:
>
>bash-2.05$ time cp -r /mnt/windows3/windows/system32 /home/grogan/test
>
>real    8m45.256s
>user    0m0.730s
>sys     6m27.030s
>
>On a fresh boot with 2.5.7 with the new NTFS driver:
>
>bash-2.05$ time cp -r /mnt/windows3/windows/system32 /home/grogan/test
>
>real    3m13.190s
>user    0m0.610s
>sys     0m51.660s
>
>This "test" was repeated twice under the same conditions, with negligible 
>difference in the result (couple of seconds). Both of these disks are on 
>the same IDE controller (/home is /dev/hda8 and the NTFS partition is 
>/dev/hdb2). I must say I wasn't expecting such a drastic difference. The 
>data appears to be intact. (correct size and number of files, anyway)

The new driver has very significant advances in the way it accesses 
metadata as it is completely stored in the page cache which means that your 
directory contents are cached in memory so directory lookups take fractions 
of a second instead of seconds during which the device has to be accessed 
again and again.

Memory mapping and read ahead are further features which impact streaming 
performance such as copying files dramatically.

>I wanted to send  a "thumbs up". Thanks, Anton, and everyone else that 
>does what they do around here : - )

Thanks. (-:

Best regards,

         Anton


-- 
   "I've not lost my mind. It's backed up on tape somewhere." - Unknown
-- 
Anton Altaparmakov <aia21 at cam.ac.uk> (replace at with @)
Linux NTFS Maintainer / WWW: http://linux-ntfs.sf.net/
ICQ: 8561279 / WWW: http://www-stu.christs.cam.ac.uk/~aia21/

Re: ANN: New NTFS driver (2.0.0/TNG) now finished.

[Bill Davidsen]

On Mon, 25 Mar 2002, Grogan wrote:

> On a fresh boot with 2.4.19-pre4:
> 
> bash-2.05$ time cp -r /mnt/windows3/windows/system32 /home/grogan/test
> 
> real    8m45.256s
> user    0m0.730s
> sys     6m27.030s
> 
> On a fresh boot with 2.5.7 with the new NTFS driver:
> 
> bash-2.05$ time cp -r /mnt/windows3/windows/system32 /home/grogan/test
> 
> real    3m13.190s
> user    0m0.610s
> sys     0m51.660s
> 
> This "test" was repeated twice under the same conditions, with
> negligible difference in the result (couple of seconds). Both of these
> disks are on the same IDE controller (/home is /dev/hda8 and the NTFS
> partition is /dev/hdb2). I must say I wasn't expecting such a drastic
> difference. The data appears to be intact. (correct size and number of
> files, anyway) 

  Looks great, one of the few things in 2.5 I see as a reason upgrade in
the future. This could be really useful to people who need to pull NT
data.

-- 
bill davidsen <davidsen@tmr.com>
  CTO, TMR Associates, Inc
Doing interesting things with little computers since 1979.