[ANNOUNCE] new HFS(+) driver

[ANNOUNCE] new HFS(+) driver

[Roman Zippel]

Hi,

This is a rather big update to the HFS+ driver. It includes now also an 
updated HFS driver. Both drivers use now almost the same btree code and 
the general structure is very similiar, so one day it will be possible to 
merge both drivers. The HFS driver got a major cleanup and a lot of broken 
options were removed, most notably if you want to continue using netatalk 
with this driver, you have to fix netatalk first.

The HFS+ driver has a number of improvements and fixes:
- blocks are now preallocated.
- allocation file is now in the page cache too
- better extent caching
- btrees are now able to grow arbitrarily
- allocation block size can now be larger than a page
- actual fs block size is adjusted to avoid alignment problems
- cdrom session/partition support (note that this is a crutch and has 
  problems)

This is basically the version I'd liked to get merged into 2.6 (minus lots 
of ifdefs and some debug prints). You can find the driver at
http://www.ardistech.com/hfsplus/

bye, Roman

Re: [ANNOUNCE] new HFS(+) driver

[Dave O]

Hi Roman

 I get this when building your driver (20030930) against 2.6.0-test6:

*** Warning: "get_gendisk" [fs/hfsplus/hfsplus.ko] undefined!
*** Warning: "get_gendisk" [fs/hfs/hfs.ko] undefined!

any idea how to resolve this?

On Thu, 2 Oct 2003, Roman Zippel wrote:

> Hi,
>
> This is a rather big update to the HFS+ driver. It includes now also an
> updated HFS driver. Both drivers use now almost the same btree code and
> the general structure is very similiar, so one day it will be possible to
> merge both drivers. The HFS driver got a major cleanup and a lot of broken
> options were removed, most notably if you want to continue using netatalk
> with this driver, you have to fix netatalk first.
>
> The HFS+ driver has a number of improvements and fixes:
> - blocks are now preallocated.
> - allocation file is now in the page cache too
> - better extent caching
> - btrees are now able to grow arbitrarily
> - allocation block size can now be larger than a page
> - actual fs block size is adjusted to avoid alignment problems
> - cdrom session/partition support (note that this is a crutch and has
>   problems)
>
> This is basically the version I'd liked to get merged into 2.6 (minus lots
> of ifdefs and some debug prints). You can find the driver at
> http://www.ardistech.com/hfsplus/
>
> bye, Roman
>
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/
>
>
>

Re: [ANNOUNCE] new HFS(+) driver

[viro]

On Thu, Oct 02, 2003 at 10:37:32AM +0200, Roman Zippel wrote:
> The HFS+ driver has a number of improvements and fixes:
> - blocks are now preallocated.
> - allocation file is now in the page cache too
> - better extent caching
> - btrees are now able to grow arbitrarily
> - allocation block size can now be larger than a page
> - actual fs block size is adjusted to avoid alignment problems
> - cdrom session/partition support (note that this is a crutch and has 
>   problems)
> 
> This is basically the version I'd liked to get merged into 2.6 (minus lots 
> of ifdefs and some debug prints). You can find the driver at
> http://www.ardistech.com/hfsplus/

What the devil are you doing with get_gendisk() in there?  Neither 2.4
nor 2.6 should be messing with it.

Re: [ANNOUNCE] new HFS(+) driver

[Roman Zippel]

Hi,

On Thu, 2 Oct 2003 viro@parcelfarce.linux.theplanet.co.uk wrote:

> What the devil are you doing with get_gendisk() in there?  Neither 2.4
> nor 2.6 should be messing with it.

In 2.6 it's only a debugging check and will be removed. In 2.4 it's 
currently needed to get the correct size. The backup volume sector is in 
the second last (512 byte) sector, but bd_inode->i_size is truncated to 
BLOCK_SIZE.

bye, Roman

Re: [ANNOUNCE] new HFS(+) driver

[Roman Zippel]

Hi,

On Thu, 2 Oct 2003, Dave O wrote:

>  I get this when building your driver (20030930) against 2.6.0-test6:
> 
> *** Warning: "get_gendisk" [fs/hfsplus/hfsplus.ko] undefined!
> *** Warning: "get_gendisk" [fs/hfs/hfs.ko] undefined!
> 
> any idea how to resolve this?

You can simply remove the code block from get_gendisk to put_disk in 
hfsplus/wrapper.c and hfs/mdb.c.

bye, Roman

Re: [ANNOUNCE] new HFS(+) driver

[Brad Boyer]

On Thu, Oct 02, 2003 at 07:06:45PM +0100, viro@parcelfarce.linux.theplanet.co.uk wrote:
> What the devil are you doing with get_gendisk() in there?  Neither 2.4
> nor 2.6 should be messing with it.

Since this topic has come up, I'd like to ask about something that
apparently only affects HFS/HFS+. For some reason, Apple decided
that a Mac style CD-ROM should be a partitioned device. However,
the Linux kernel is quite insistent that a CD-ROM is not able to
be partitioned. Because of this, there's a hack to manually read
a partition map and find the correct part of the block device.

Would it be possible to have a way to use the gendisk and partitioning
code that is already in the kernel for regular disks to read these
CDs? It also might be useful for the loopback device.

Just as an example of worst case, the main A/UX install CD had
not only an HFS partition, but multiple UFS partitions. If you
really want a view of the extent of Apple hackery, take a look
at arch/m68k/mac and groan.  :)

	Brad Boyer
	flar@allandria.com

Re: [ANNOUNCE] new HFS(+) driver

[Dave O]

On Thu, 2 Oct 2003, Roman Zippel wrote:
> On Thu, 2 Oct 2003, Dave O wrote:
>
> >  I get this when building your driver (20030930) against 2.6.0-test6:
> >
> > *** Warning: "get_gendisk" [fs/hfsplus/hfsplus.ko] undefined!
> > *** Warning: "get_gendisk" [fs/hfs/hfs.ko] undefined!
> >
> > any idea how to resolve this?
>
> You can simply remove the code block from get_gendisk to put_disk in
> hfsplus/wrapper.c and hfs/mdb.c.
>

This works, however du(1) seems to get the block size wrong:

meatloop:/cdrom# ls -l
total 393244
-rwxr-xr-x    1 501      dialout  341952833 Sep 22 17:24 else.zip
-rwxr-xr-x    1 501      dialout  450701627 Sep 22 20:07 outlook.zip
-rwxr-xr-x    1 501      dialout  607534655 Sep 22 17:26 quick1.zip
-rwxr-xr-x    1 501      dialout  431279243 Sep 22 17:26 quick2.zip
-rwxr-xr-x    1 501      dialout  605501959 Sep 22 17:27 quick3.zip
-rwxr-xr-x    1 501      dialout  403836898 Sep 22 17:28 quick4.zip
-rwxr-xr-x    1 501      dialout  380636073 Sep 22 17:28 quick5.zip

meatloop:/cdrom# du -sh
385M    .

Re: [ANNOUNCE] new HFS(+) driver

[Mike Fedyk]

On Thu, Oct 02, 2003 at 02:00:25PM -0500, Dave O wrote:
> This works, however du(1) seems to get the block size wrong:
> 
> meatloop:/cdrom# ls -l
> total 393244
> -rwxr-xr-x    1 501      dialout  341952833 Sep 22 17:24 else.zip
> -rwxr-xr-x    1 501      dialout  450701627 Sep 22 20:07 outlook.zip
> -rwxr-xr-x    1 501      dialout  607534655 Sep 22 17:26 quick1.zip
> -rwxr-xr-x    1 501      dialout  431279243 Sep 22 17:26 quick2.zip
> -rwxr-xr-x    1 501      dialout  605501959 Sep 22 17:27 quick3.zip
> -rwxr-xr-x    1 501      dialout  403836898 Sep 22 17:28 quick4.zip
> -rwxr-xr-x    1 501      dialout  380636073 Sep 22 17:28 quick5.zip
> 
> meatloop:/cdrom# du -sh
> 385M    .

Are you sure?  I haven't done the math, but you are comparing base 1000 to
base 1024 numbers.  Try comparing:

du -sh .
ls -lh

Re: [ANNOUNCE] new HFS(+) driver

[Dave O]

On Thu, 2 Oct 2003, Mike Fedyk wrote:

> On Thu, Oct 02, 2003 at 02:00:25PM -0500, Dave O wrote:
> > This works, however du(1) seems to get the block size wrong:
> >
> > meatloop:/cdrom# ls -l
> > total 393244
> > -rwxr-xr-x    1 501      dialout  341952833 Sep 22 17:24 else.zip
> > -rwxr-xr-x    1 501      dialout  450701627 Sep 22 20:07 outlook.zip
> > -rwxr-xr-x    1 501      dialout  607534655 Sep 22 17:26 quick1.zip
> > -rwxr-xr-x    1 501      dialout  431279243 Sep 22 17:26 quick2.zip
> > -rwxr-xr-x    1 501      dialout  605501959 Sep 22 17:27 quick3.zip
> > -rwxr-xr-x    1 501      dialout  403836898 Sep 22 17:28 quick4.zip
> > -rwxr-xr-x    1 501      dialout  380636073 Sep 22 17:28 quick5.zip
> >
> > meatloop:/cdrom# du -sh
> > 385M    .
>
> Are you sure?  I haven't done the math, but you are comparing base 1000 to
> base 1024 numbers.  Try comparing:
>
> du -sh .
> ls -lh
>

There's not that much to be sure about.  A single file there (else.zip) is
over 300 megs itself, and the total of all the files is about 3 gigs

Re: [ANNOUNCE] new HFS(+) driver

[J.A. Magallon]

On 10.02, Roman Zippel wrote:
> Hi,
> 
> This is a rather big update to the HFS+ driver. It includes now also an 
> updated HFS driver. Both drivers use now almost the same btree code and 
> the general structure is very similiar, so one day it will be possible to 
> merge both drivers. The HFS driver got a major cleanup and a lot of broken 
> options were removed, most notably if you want to continue using netatalk 
> with this driver, you have to fix netatalk first.
> 

Two notes:
- You should give a patch or at least give a notice that linux/include/hfs* have
  to be killed (or move hfs_fs.h there).
- I had to include linux/sched.h in hfs/sysdep.c to get the definition for
  'current', that was neded in some subinclude of linux/smp_lock. This can be
  caused by any other of my patches, but it doesn't hurt.

TIA 

-- 
J.A. Magallon <jamagallon()able!es>     \                 Software is like sex:
werewolf!able!es                         \           It's better when it's free
Mandrake Linux release 9.2 (Cooker) for i586
Linux 2.4.23-pre6-jam1 (gcc 3.3.1 (Mandrake Linux 9.2 3.3.1-2mdk))

Re: [ANNOUNCE] new HFS(+) driver

[Roman Zippel]

Hi,

On Fri, 3 Oct 2003, J.A. Magallon wrote:

> Two notes:
> - You should give a patch or at least give a notice that linux/include/hfs* have
>   to be killed (or move hfs_fs.h there).

You did apply linux-2.4.hfs.diff? I don't understand why you had to move 
hfs_fs.h, it should be picked up from the current directory.

> - I had to include linux/sched.h in hfs/sysdep.c to get the definition for
>   'current', that was neded in some subinclude of linux/smp_lock. This can be
>   caused by any other of my patches, but it doesn't hurt.

Simply move <linux/smp_lock.h> past "hfs_fs.h".

bye, Roman

Re: [ANNOUNCE] new HFS(+) driver

[J.A. Magallon]

On 10.03, Roman Zippel wrote:
> Hi,
> 
> On Fri, 3 Oct 2003, J.A. Magallon wrote:
> 
> > Two notes:
> > - You should give a patch or at least give a notice that linux/include/hfs* have
> >   to be killed (or move hfs_fs.h there).
> 
> You did apply linux-2.4.hfs.diff? I don't understand why you had to move 
> hfs_fs.h, it should be picked up from the current directory.
> 

I applied it, it just keeps mainstream files from including old hfs* files
in include/linux, but the old files stay around (you end with two
versions of hfs_fs.h, one in include/linux and other in fs/hfs...)
I did not move anything, just deleted those old files. But as other
filesystems put their xxxx_fs.h in include/linux, I thought that
perhaps hfs(plus) should do the same.

> > - I had to include linux/sched.h in hfs/sysdep.c to get the definition for
> >   'current', that was neded in some subinclude of linux/smp_lock. This can be
> >   caused by any other of my patches, but it doesn't hurt.
> 
> Simply move <linux/smp_lock.h> past "hfs_fs.h".
> 

Thanks.

-- 
J.A. Magallon <jamagallon()able!es>     \                 Software is like sex:
werewolf!able!es                         \           It's better when it's free
Mandrake Linux release 9.2 (Cooker) for i586
Linux 2.4.23-pre6-jam1 (gcc 3.3.1 (Mandrake Linux 9.2 3.3.1-2mdk))

Re: [ANNOUNCE] new HFS(+) driver

[Matthew Wilcox]

On Sat, Oct 04, 2003 at 12:39:04AM +0200, J.A. Magallon wrote:
> I applied it, it just keeps mainstream files from including old hfs* files
> in include/linux, but the old files stay around (you end with two
> versions of hfs_fs.h, one in include/linux and other in fs/hfs...)
> I did not move anything, just deleted those old files. But as other
> filesystems put their xxxx_fs.h in include/linux, I thought that
> perhaps hfs(plus) should do the same.

actually more of these should move out of include/linux/

-- 
"It's not Hollywood.  War is real, war is primarily not about defeat or
victory, it is about death.  I've seen thousands and thousands of dead bodies.
Do you think I want to have an academic debate on this subject?" -- Robert Fisk

Re: [ANNOUNCE] new HFS(+) driver

[Mike Fedyk]

On Thu, Oct 02, 2003 at 11:52:48AM -0700, Brad Boyer wrote:
> On Thu, Oct 02, 2003 at 07:06:45PM +0100, viro@parcelfarce.linux.theplanet.co.uk wrote:
> > What the devil are you doing with get_gendisk() in there?  Neither 2.4
> > nor 2.6 should be messing with it.
> 
> Since this topic has come up, I'd like to ask about something that
> apparently only affects HFS/HFS+. For some reason, Apple decided
> that a Mac style CD-ROM should be a partitioned device. However,
> the Linux kernel is quite insistent that a CD-ROM is not able to
> be partitioned. Because of this, there's a hack to manually read
> a partition map and find the correct part of the block device.
> 

Where can I find this hack?  I just had three CDs from cleints that have
partition maps on them...

I've read them on our Mac (OS9.2), but I'd love to know how to read them
under linux!

Re: [ANNOUNCE] new HFS(+) driver

[Brad Boyer]

On Mon, Oct 06, 2003 at 12:38:41PM -0700, Mike Fedyk wrote:
> Where can I find this hack?  I just had three CDs from cleints that have
> partition maps on them...
> 
> I've read them on our Mac (OS9.2), but I'd love to know how to read them
> under linux!

It's in the hfs and hfsplus modules as implemented in the patch mentioned
in the original message in this thread. The hfs code has had it for ages,
but this is the first version of the hfsplus code to support it. The
only issue is that you can only mount the one volume at a time, and it
has to be either HFS or HFS+. If you have a multi-volume CD, you can't
mount all of them. If you have a really stupid CD like the old A/UX
install CD with other filesystem types on it, you're out of luck
without copying it over to a hard drive and mounting with the loop
device and some careful trimming of the file.

	Brad Boyer
	flar@allandria.com