9/10 intermezzos prefer eating memory

9/10 intermezzos prefer eating memory

[Anton Blanchard]

Hi,

Im sure the 4kB stack brigade wont be too happy about this:

presto_get_fileid: 4672
presto_copy_kml_tail: 4272

This is 2.6. The following patch fixes it. There are still more things
to attack (this is on ppc64 but you get the idea):

presto_ioctl: 1424
presto_journal_unlink: 1056
presto_journal_rename: 1040
presto_journal_rmdir: 1008

Anton

--

diff -puN fs/intermezzo/journal.c~intermezzofix fs/intermezzo/journal.c
--- foobar2/fs/intermezzo/journal.c~intermezzofix	2004-05-02 16:58:36.297497086 +1000
+++ foobar2-anton/fs/intermezzo/journal.c	2004-05-02 17:50:31.696425996 +1000
@@ -1235,12 +1235,15 @@ int presto_write_kml_logical_offset(stru
         return izo_rcvd_write(fset, &rec);
 }
 
+#define BUFSIZE 4096
+
 struct file * presto_copy_kml_tail(struct presto_file_set *fset,
                                    unsigned long int start)
 {
         struct file *f;
         int len;
         loff_t read_off, write_off, bytes;
+        char *buf;
 
         ENTRY;
 
@@ -1251,15 +1254,21 @@ struct file * presto_copy_kml_tail(struc
                 return f;
         }
 
+        buf = kmalloc(BUFSIZE, GFP_KERNEL);
+        if (!buf) {
+                EXIT;
+                return -ENOMEM;
+        }
+        memset(buf, 0, BUFSIZE);
+
         write_off = 0;
         read_off = start;
         bytes = fset->fset_kml.fd_offset - start;
         while (bytes > 0) {
-                char buf[4096];
                 int toread;
 
-                if (bytes > sizeof(buf))
-                        toread = sizeof(buf);
+                if (bytes > BUFSIZE)
+                        toread = BUFSIZE;
                 else
                         toread = bytes;
 
@@ -1270,6 +1279,7 @@ struct file * presto_copy_kml_tail(struc
 
                 if (presto_fwrite(f, buf, len, &write_off) != len) {
                         filp_close(f, NULL);
+                        kfree(buf);
                         EXIT;
                         return ERR_PTR(-EIO);
                 }
@@ -1277,6 +1287,7 @@ struct file * presto_copy_kml_tail(struc
                 bytes -= len;
         }
 
+        kfree(buf);
         EXIT;
         return f;
 }
@@ -1580,12 +1591,14 @@ int presto_journal_setattr(struct rec_in
         return error;
 }
 
+#define RECORD_LENGTH 4096
+
 int presto_get_fileid(int minor, struct presto_file_set *fset,
                       struct dentry *dentry)
 {
         int opcode = KML_OPCODE_GET_FILEID;
         struct rec_info rec;
-        char *buffer, *path, *logrecord, record[4096]; /*include path*/
+        char *buffer, *path, *logrecord, *record; /*include path*/
         struct dentry *root;
         __u32 uid, gid, pathlen;
         int error, size;
@@ -1593,6 +1606,11 @@ int presto_get_fileid(int minor, struct 
 
         ENTRY;
 
+        record = kmalloc(RECORD_LENGTH, GFP_KERNEL);
+        if (!record)
+                return -ENOMEM;
+        memset(record, 0, RECORD_LENGTH);
+
         root = fset->fset_dentry;
 
         uid = cpu_to_le32(dentry->d_inode->i_uid);
@@ -1606,7 +1624,7 @@ int presto_get_fileid(int minor, struct 
                 sizeof(struct kml_suffix);
 
         CDEBUG(D_FILE, "kml size: %d\n", size);
-        if ( size > sizeof(record) )
+        if ( size > RECORD_LENGTH )
                 CERROR("InterMezzo: BUFFER OVERFLOW in %s!\n", __FUNCTION__);
 
         memset(&rec, 0, sizeof(rec));
@@ -1628,6 +1646,7 @@ int presto_get_fileid(int minor, struct 
                                    size_round(le32_to_cpu(pathlen)), path,
                                    fset->fset_name);
 
+        kfree(record);
         BUFF_FREE(buffer);
         EXIT;
         return error;

Re: 9/10 intermezzos prefer eating memory

[Arjan van de Ven]

[-- Attachment #1: Type: text/plain, Size: 186 bytes --]

On Sun, 2004-05-02 at 10:00, Anton Blanchard wrote:
> Hi,
> 
> Im sure the 4kB stack brigade wont be too happy about this:

I thought intermezzo would have been rm -rf'd by now...

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: 9/10 intermezzos prefer eating memory

[Jörn Engel]

On Sun, 2 May 2004 10:22:26 +0200, Arjan van de Ven wrote:
> On Sun, 2004-05-02 at 10:00, Anton Blanchard wrote:
> > Hi,
> > 
> > Im sure the 4kB stack brigade wont be too happy about this:
> 
> I thought intermezzo would have been rm -rf'd by now...

Should have.  I've written patches for intermezzo a year ago.
Maintenance is a little slow these days. :)

Jörn

-- 
It does not matter how slowly you go, so long as you do not stop.
-- Confucius

Re: 9/10 intermezzos prefer eating memory

[Muli Ben-Yehuda]

[-- Attachment #1: Type: text/plain, Size: 735 bytes --]

On Mon, May 03, 2004 at 11:37:19AM +0200, Jörn Engel wrote:
> On Sun, 2 May 2004 10:22:26 +0200, Arjan van de Ven wrote:
> > On Sun, 2004-05-02 at 10:00, Anton Blanchard wrote:
> > > Hi,
> > > 
> > > Im sure the 4kB stack brigade wont be too happy about this:
> > 
> > I thought intermezzo would have been rm -rf'd by now...
> 
> Should have.  I've written patches for intermezzo a year ago.
> Maintenance is a little slow these days. :)

Likewise. I have a patch[0] for intermezzo stack lossage from 2.5.73
era that hasn't been applied yet, AFAIK.  

[0] http://www.mulix.org/patches/intermezzo-stack-lossage-2.5.73.diff

Cheers, 
Muli 
-- 
Muli Ben-Yehuda
http://www.mulix.org | http://mulix.livejournal.com/


[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

RE: 9/10 intermezzos prefer eating memory

[Peter J. Braam]

 
Hi Andrew, 

I would just like to say that I have no difficulties with intermezzo being
rm -rf'd.  There are probably only a handful of users.   

In the past 4 years nobody has supported InterMezzo sufficiently for it to
become successful. I have been fortunate to get really good support for the
Lustre project.  So I have focussed on that.  Lustre 1.X has become really
solid.

The disconnected operation, caching and mirroring functionality of
InterMezzo will become available in Lustre as a new feature in version 2.  

So I see no point in keeping InterMezzo if it is a nuisance.  

I am also entirely happy to ask my one part time InterMezzo programmer to do
a better job of repeatedly sending pathes until they are in.

Please guide me along.  Thanks!

- Peter -


> -----Original Message-----
> From: intermezzo-devel-admin@lists.sourceforge.net 
> [mailto:intermezzo-devel-admin@lists.sourceforge.net] On 
> Behalf Of Arjan van de Ven
> Sent: Sunday, May 02, 2004 4:22 PM
> To: Anton Blanchard
> Cc: akpm@osdl.org; intermezzo-devel@lists.sourceforge.net; 
> linux-kernel@vger.kernel.org
> Subject: Re: 9/10 intermezzos prefer eating memory
> 
> On Sun, 2004-05-02 at 10:00, Anton Blanchard wrote:
> > Hi,
> > 
> > Im sure the 4kB stack brigade wont be too happy about this:
> 
> I thought intermezzo would have been rm -rf'd by now...
> 

Re: 9/10 intermezzos prefer eating memory

[Andrew Morton]

"Peter J. Braam" <braam@clusterfs.com> wrote:
>
>  
> Hi Andrew, 
> 
> I would just like to say that I have no difficulties with intermezzo being
> rm -rf'd.  There are probably only a handful of users.   
> 
> In the past 4 years nobody has supported InterMezzo sufficiently for it to
> become successful. I have been fortunate to get really good support for the
> Lustre project.  So I have focussed on that.  Lustre 1.X has become really
> solid.
> 
> The disconnected operation, caching and mirroring functionality of
> InterMezzo will become available in Lustre as a new feature in version 2.  
> 
> So I see no point in keeping InterMezzo if it is a nuisance.  
> 
> I am also entirely happy to ask my one part time InterMezzo programmer to do
> a better job of repeatedly sending pathes until they are in.
> 
> Please guide me along.  Thanks!
> 

Thanks - such opportunities are all too rare.

Are there any distro people out there who have a significant Intermezzo
user base, or who see any problems with this?

Re: 9/10 intermezzos prefer eating memory

[Troy Benjegerdes]

On Sun, May 09, 2004 at 02:36:21AM -0700, Andrew Morton wrote:
> "Peter J. Braam" <braam@clusterfs.com> wrote:
> >
> >  
> > Hi Andrew, 
> > 
> > I would just like to say that I have no difficulties with intermezzo being
> > rm -rf'd.  There are probably only a handful of users.   
> > 
> > In the past 4 years nobody has supported InterMezzo sufficiently for it to
> > become successful. I have been fortunate to get really good support for the
> > Lustre project.  So I have focussed on that.  Lustre 1.X has become really
> > solid.
> > 
> > The disconnected operation, caching and mirroring functionality of
> > InterMezzo will become available in Lustre as a new feature in version 2.  
> > 
> > So I see no point in keeping InterMezzo if it is a nuisance.  
> > 
> > I am also entirely happy to ask my one part time InterMezzo programmer to do
> > a better job of repeatedly sending pathes until they are in.
> > 
> > Please guide me along.  Thanks!
> > 
> 
> Thanks - such opportunities are all too rare.
> 
> Are there any distro people out there who have a significant Intermezzo
> user base, or who see any problems with this?

As one of the apparently 3 or 4 longtime users of Intermezzo, I like the
architecture, but the user-level utilities for manageing it (aka, for
resolving conflicts that occured during disconnected operation) just
aren't there, and nobody (including myself) has really had time to
devote to getting them in working shape.

I'd love to get my hands on Lustre version 2 and start beating up the
disconnected operation. If that's available in CVS somewhere, I'd vote
for rm -rf of intermezzo.

--------------------------------------------------------------------------
Troy Benjegerdes                'da hozer'                hozer@hozed.org  

Somone asked my why I work on this free (http://www.fsf.org/philosophy/)
software stuff and not get a real job. Charles Shultz had the best answer:

"Why do musicians compose symphonies and poets write poems? They do it
because life wouldn't have any meaning for them if they didn't. That's why
I draw cartoons. It's my life." -- Charles Shultz

Re: 9/10 intermezzos prefer eating memory

[Troy Benjegerdes]

On Sun, May 09, 2004 at 03:33:46PM +0200, Daniel Egger wrote:
> On 09.05.2004, at 11:01, Peter J. Braam wrote:
> 
> >I would just like to say that I have no difficulties with intermezzo 
> >being
> >rm -rf'd.  There are probably only a handful of users.
> 
> After my painful experience that it never worked cross-architecture[1]
> I decided to stay completely away from it. I would not consider a
> filesystem that doesn't work at all on some popular platforms to be
> "stable" and thus 2.4 or 2.6 worthy.
> 
> [1] powerpc-linux client, x86-linux server
> 
> Servus,
>       Daniel

That's kinda weird.. I'm running a powerpc-linux server, x86-client 

All my MP3's are on an intermezzo filesystem.