Coda and Ext3

Coda and Ext3

[Sujal Shah]

Hi everyone,

     The Linux Coda drivers and the ext3 patches don't seem to get along
very well, at least in Linux 2.4.7.  I've got a stock 2.4.7 kernel with
a patch applied to the USB drivers (for a sony digital camera; see
http://www.sujal.net/tech/linux/ just a change in unusual_devs.h). 

After I applied the ext3 patches from
http://www.uow.edu.au/~andrewm/linux/ext3/ .  Basically, when an
application tries to write to a file system mounted via coda, the
application terminates with "Memory Fault" returned to the terminal. 
THe file system still thinks it's busy (can't umount).

I'm using the ext3-2.4-0.9.5-247 patch.  I have not yet tried a newer
kernel.

THe funny thing is that mounting/unmount works fine as long as I don't
try to write.  Also, I can create files via touch or mkdir without
causing problems.  The following log snippet shows me mounting then
immediately unmounting the filesystem, then remounting and trying a
write operation (using cp).

Just to let everyone know, I am running my own replacement for venus
(something similar to podfuk/uservfs), and I do have the NVidia drivers
loaded.  I was able to recreate this bug without the NVdriver module
loaded, however.  Also, I backed out the patches for ext3 and the
problem went away.

THe following messages appear in my syslog:

Sep 18 19:02:00 pcsshah kernel: Coda Kernel/Venus communications, v5.3.14, coda@cs.cmu.edu
Sep 18 19:02:16 pcsshah kernel: coda_read_super: Bad mount version
Sep 18 19:02:16 pcsshah kernel: coda_read_super: device index: 0
Sep 18 19:02:16 pcsshah kernel: coda_read_super: rootfid is (0x1234567,0xffffffff,0x1)
Sep 18 19:02:16 pcsshah kernel: coda_read_super: rootinode is 1450180609 dev 7
Sep 18 19:03:04 pcsshah kernel: Coda: Bye bye.
Sep 18 19:03:50 pcsshah gconfd (sujal-1184): 21 items remain in the cache after cleaning already-synced items older than 300 seconds
Sep 18 19:04:34 pcsshah kernel: coda_read_super: Bad mount version
Sep 18 19:04:34 pcsshah kernel: coda_read_super: device index: 0
Sep 18 19:04:34 pcsshah kernel: coda_read_super: rootfid is (0x1234567,0xffffffff,0x1)
Sep 18 19:04:34 pcsshah kernel: coda_read_super: rootinode is 1450180609 dev 7
Sep 18 19:05:02 pcsshah kernel: kernel BUG at file.c:45!
Sep 18 19:05:02 pcsshah kernel: invalid operand: 0000
Sep 18 19:05:02 pcsshah kernel: CPU:    0
Sep 18 19:05:02 pcsshah kernel: EIP:    0010:[<e59a73df>]
Sep 18 19:05:02 pcsshah kernel: EFLAGS: 00210282
Sep 18 19:05:02 pcsshah kernel: eax: 00000019   ebx: d403e1a0   ecx: 00000006   edx: 00000000
Sep 18 19:05:02 pcsshah kernel: esi: ffffffea   edi: d3eaf240   ebp: d3f71de0   esp: d36f3f60
Sep 18 19:05:02 pcsshah kernel: ds: 0018   es: 0018   ss: 0018
Sep 18 19:05:02 pcsshah kernel: Process cp (pid: 1650, stackpage=d36f3000)
Sep 18 19:05:02 pcsshah kernel: Stack: e59acb2c e59acc83 0000002d d3f71de0 ffffffea 00000000 00000400 c0130f96 
Sep 18 19:05:02 pcsshah kernel:        d3f71de0 bfffef10 00000400 d3f71e00 00000000 0009bce9 00000000 d3a78b60 
Sep 18 19:05:02 pcsshah kernel:        d383ede0 00000000 d3f719c0 bffff308 d36f2000 00000400 bfffef10 bfffeef8 
Sep 18 19:05:02 pcsshah kernel: Call Trace: [sys_write+150/208] [system_call+51/56] 
Sep 18 19:05:02 pcsshah kernel: 
Sep 18 19:05:02 pcsshah kernel: Code: 0f 0b 83 c4 0c 8b 43 08 8b 70 08 8d 5e 5c 89 d9 ff 4e 5c 0f 

Just FYI. 

Thanks,

Sujal

-- 
---- Sujal Shah --- sujal@sujal.net ---

        http://www.sujal.net

Now Playing: George Michael - Freedom 90

Re: Coda and Ext3

[Jan Harkes]

On Wed, Sep 19, 2001 at 10:23:36AM -0400, Sujal Shah wrote:
>      The Linux Coda drivers and the ext3 patches don't seem to get along
> very well, at least in Linux 2.4.7.  I've got a stock 2.4.7 kernel with
> a patch applied to the USB drivers (for a sony digital camera; see
> http://www.sujal.net/tech/linux/ just a change in unusual_devs.h). 
> 
> After I applied the ext3 patches from
> http://www.uow.edu.au/~andrewm/linux/ext3/ .  Basically, when an
> application tries to write to a file system mounted via coda, the
> application terminates with "Memory Fault" returned to the terminal. 
> THe file system still thinks it's busy (can't umount).

Yeah, I know, and it will probably work when you don't enable
data-journalling. Coda's kernelmodule currently uses generic_file_read
and generic_file_write on it's containerfiles, which works for many
filesystems. However, ext3fs (and tmpfs and several others) have a
filesystem specific write implementation and don't really like being
called with the generic functions.

The patch is simple, but I haven't made it yet. Basically we need to
wrap the read/write calls and call cii->c_cfile->f_op->file_write or
something.

> loaded, however.  Also, I backed out the patches for ext3 and the
> problem went away.

ext2 uses the generic file read/write functions, so whenever ext2 is the
underlying filesystem it all works fine.

Jan

[PATCH] Re: Coda and Ext3

[Jan Harkes]

On Wed, Sep 19, 2001 at 10:23:36AM -0400, Sujal Shah wrote:
>      The Linux Coda drivers and the ext3 patches don't seem to get along
> very well, at least in Linux 2.4.7.  I've got a stock 2.4.7 kernel with

The attached patch should fix it. I haven't tested it against ext3, but
with tmpfs which used to have the same problem, i.e. not using
generic_file_ functions to access the container files. I'll pass it on
to Linus and Alan once I get some feedback on whether this solves all
problems. I believe this will fix the whole batch of problems that are
have been reported with ext3fs, XFS, and tmpfs.

It should apply fine for kernel versions 2.4.4 and higher (both the AC
and the Linus trees).

btw. my comment about disabling ext3 data journalling fixing it was
bogus, there was a BUG() in Coda that got triggered because f_op->write
wasn't generic_file_write.

Jan

diff -urN --exclude-from=dontdiff linux-2.4.10-pre9/fs/coda/file.c linux/fs/coda/file.c
--- linux-2.4.10-pre9/fs/coda/file.c	Thu Sep  6 20:02:24 2001
+++ linux/fs/coda/file.c	Wed Sep 19 20:26:41 2001
@@ -31,28 +31,65 @@
 int use_coda_close;
 
 static ssize_t
-coda_file_write(struct file *file,const char *buf,size_t count,loff_t *ppos)
+coda_file_read(struct file *file, char *buf, size_t count, loff_t *ppos)
 {
+	struct inode *inode = file->f_dentry->d_inode;
+	struct coda_inode_info *cii = ITOC(inode);
 	struct file *cfile;
+
+	cfile = cii->c_container;
+	if (!cfile) BUG();
+
+	if (!cfile->f_op || !cfile->f_op->read)
+		return -EINVAL;
+
+	return cfile->f_op->read(cfile, buf, count, ppos);
+}
+
+static ssize_t
+coda_file_write(struct file *file,const char *buf,size_t count,loff_t *ppos)
+{
 	struct inode *cinode, *inode = file->f_dentry->d_inode;
 	struct coda_inode_info *cii = ITOC(inode);
-	ssize_t n;
+	struct file *cfile;
+	ssize_t ret;
+	int flags;
 
 	cfile = cii->c_container;
 	if (!cfile) BUG();
 
-	if (!cfile->f_op || cfile->f_op->write != generic_file_write)
-		BUG();
+	if (!cfile->f_op || !cfile->f_op->write)
+		return -EINVAL;
 
 	cinode = cfile->f_dentry->d_inode;
-	down(&cinode->i_sem);
+	down(&inode->i_sem);
+	flags = cfile->f_flags;
+        cfile->f_flags |= file->f_flags & (O_APPEND | O_SYNC);
+
+	ret = cfile->f_op->write(cfile, buf, count, ppos);
 
-	n = generic_file_write(file, buf, count, ppos);
+	cfile->f_flags = flags;
 	inode->i_size = cinode->i_size;
+	up(&inode->i_sem);
+
+	return ret;
+}
+
+static int
+coda_file_mmap(struct file *file, struct vm_area_struct *vma)
+{
+	struct inode *inode = file->f_dentry->d_inode;
+	struct coda_inode_info *cii = ITOC(inode);
+	struct file *cfile;
+
+	cfile = cii->c_container;
+
+	if (!cfile) BUG();
 
-	up(&cinode->i_sem);
+	if (!cfile->f_op || !cfile->f_op->mmap)
+		return -ENODEV;
 
-	return n;
+	return cfile->f_op->mmap(cfile, vma);
 }
 
 int coda_open(struct inode *i, struct file *f)
@@ -237,9 +274,9 @@
 
 struct file_operations coda_file_operations = {
 	llseek:		generic_file_llseek,
-	read:		generic_file_read,
+	read:		coda_file_read,
 	write:		coda_file_write,
-	mmap:		generic_file_mmap,
+	mmap:		coda_file_mmap,
 	open:		coda_open,
 	flush:  	coda_flush,
 	release:	coda_release,
diff -urN --exclude-from=dontdiff linux-2.4.10-pre9/fs/coda/psdev.c linux/fs/coda/psdev.c
--- linux-2.4.10-pre9/fs/coda/psdev.c	Wed Apr 25 19:18:54 2001
+++ linux/fs/coda/psdev.c	Wed Sep 19 18:45:46 2001
@@ -414,7 +414,7 @@
 static int __init init_coda(void)
 {
 	int status;
-	printk(KERN_INFO "Coda Kernel/Venus communications, v5.3.14, coda@cs.cmu.edu\n");
+	printk(KERN_INFO "Coda Kernel/Venus communications, v5.3.15, coda@cs.cmu.edu\n");
 
 	status = init_coda_psdev();
 	if ( status ) {

Re: [PATCH] Re: Coda and Ext3

[Florian Schaefer]

[-- Attachment #1: Type: text/plain, Size: 859 bytes --]

On Wed, 19 Sep 2001 21:37:14 -0400, Jan Harkes said:

>On Wed, Sep 19, 2001 at 10:23:36AM -0400, Sujal Shah wrote:
> >      The Linux Coda drivers and the ext3 patches don't seem to get along
> > very well, at least in Linux 2.4.7.  I've got a stock 2.4.7 kernel with
> 
> The attached patch should fix it. I haven't tested it against ext3, but
> with tmpfs which used to have the same problem, i.e. not using
> generic_file_ functions to access the container files. I'll pass it on
> to Linus and Alan once I get some feedback on whether this solves all
> problems. I believe this will fix the whole batch of problems that are
> have been reported with ext3fs, XFS, and tmpfs.

I just tried your patch together with coda-debug-client-5.3.15-1,
kernel-2.4.9 and ext3-2.4-0.9.6-249 and still get the BUG message. The
ksymoops output is attached.

Ciao
Florian

[-- Attachment #2: BUG.ksymoops --]
[-- Type: application/x-unknown, Size: 9478 bytes --]

Re: [PATCH] Re: Coda and Ext3

[Jan Harkes]

On Thu, Sep 20, 2001 at 01:38:26PM +0100, Florian Schaefer wrote:
> On Wed, 19 Sep 2001 21:37:14 -0400, Jan Harkes said:
> >On Wed, Sep 19, 2001 at 10:23:36AM -0400, Sujal Shah wrote:
> > >      The Linux Coda drivers and the ext3 patches don't seem to get along
> > > very well, at least in Linux 2.4.7.  I've got a stock 2.4.7 kernel with
> > 
> > The attached patch should fix it. I haven't tested it against ext3, but
> > with tmpfs which used to have the same problem, i.e. not using
> > generic_file_ functions to access the container files. I'll pass it on
> > to Linus and Alan once I get some feedback on whether this solves all
> > problems. I believe this will fix the whole batch of problems that are
> > have been reported with ext3fs, XFS, and tmpfs.
> 
> I just tried your patch together with coda-debug-client-5.3.15-1,
> kernel-2.4.9 and ext3-2.4-0.9.6-249 and still get the BUG message. The
> ksymoops output is attached.

No, different bug,

Fixed problem is basically what everyone is seeing when they are using
Coda on top of a filesystem that doesn't use generic_file_write. It was
simply there to avoid corrupting the underlying fs.

Your bug is unusual, because it happens during the mount, everybody
else successfully mounted Coda and could read files, it would just die
when trying to write anything.

The bug is triggered because the inode we just got passed from the VFS
has a superblock that has no pointer to the Coda specific info.

If you look at the trace, parts of it look right, but most of it doesn't
make sense, printk never calls back into FS specific code, coda_iget
does not call coda_cnode make, coda_inocmp isn't calling iget4, etc.

If you look at the logic of the mount operation,

coda_read_super

    Here we allocate the Coda info and link it to the superblock. If the
    allocation fails we return early. An upcall is made to venus to get
    the rootfid. Then we call coda_cnode_make, passing it the rootfid
    and the initialized superblock.

coda_cnode_make

    Makes a second upcall, getting the attributes of the root inode, if
    it fails we return. Else it calls coda_iget, passing the superblock,
    the rootfid and the attributes.

coda_iget
    Calls iget4 passing it the initialized superblock, a hash of the
    fid, and a callback function + rootfid to find the real inode.

iget4
    Should fail to find the inode, and calls get_new_inode again passing
    it the initialized superblock.

get_new_inode
    Allocates an inode, initializes inode->i_sb with the passed in
    superblock and calls the fs-specific read_inode.

coda_read_inode
    Craps out because the inode->i_sb.u.generic_sbp is NULL, wtf?

The only way I can see this happening is if the sb.u.generic_sbp field
is cleared, but it looks like even coda_put_super isn't doing that.

Summary, I'm clueless how this could have happened, and I can't see how it
could happen. Perhaps a patch got botched, or your kernel is mixing up
objects from before the patch with later ones. All those ksyms errors
and the illogical jumps in the trace don't make it look to healthy either.

Try to start from a fresh 2.4.9 tree, apply the ext3 patch and my Coda
patch, add -fs1 to EXTRAVERSION just to force this tree to install it's
modules in a separate directory. Copy the config of your existing tree,
make oldconfig ; make dep ; etc...

Jan

Re: [PATCH] Re: Coda and Ext3

[Florian Schaefer]

On Thu, 20 Sep 2001 12:06:49 -0400, Jan Harkes said:

> If you look at the trace, parts of it look right, but most of it doesn't
> make sense, printk never calls back into FS specific code, coda_iget
> does not call coda_cnode make, coda_inocmp isn't calling iget4, etc.

Thank you for looking at my problem. :-)

> Try to start from a fresh 2.4.9 tree, apply the ext3 patch and my Coda

This is exactly what I did just some hours ago (oh, I had to add an
aditional patch for the latest acpi daemon, but it just patches one
single acpi file).

> patch, add -fs1 to EXTRAVERSION just to force this tree to install it's

Sorry, but I've never heard of this extraversion stuff. I found it in the
main Makefile of the kernel, do I have to add the option there?

> modules in a separate directory. 

Till now, I always tried to compile it directly into the kernel to be
able to boot from an ext3 partition. Does this make a difference? This
way I never had to touch my working modules.

> Copy the config of your existing tree,
> make oldconfig ; make dep ; etc...

make oldconfig? Doesn't make dep use the .config file directly?

As you can see, I have to admit that I'm quite confused. But I'll try to
sum up what I've understood: You want me to take a new copy of the
kernel, patch ext3 and your last patch into it, add some fancy options to
prevent me from ruining my current system and try it again.
But this is quite exactly what I did (aside from not compiling it as a
module).
One more thing: I can reproduce the BUG on my second computer with the a
similar setup.

Ciao
Florian

PS: I know that I haven't made your life easier with this mail, but
that's the way it is. Sorry.