memory management in mlmmj

memory management in mlmmj

[Morten K. Poulsen]

Memory management makes up quite a large portion of the code, especially
freeing memory at error conditions. How about we sacrifice a tiny bit of memory
and performance and automate it?

The basic idea here is to have scopes of dynamic memory, a lot like local
variables. At the beginning of each function we could begin a scope, and at the
end of a function (and at returns) we could end it. If we need to return a
pointer to some memory, we could move that chunk of memory up to the parent
scope. I think that would work well, and make the code a lot more readable.

It is not new, I think PHP does something like this (where a request is the
scope).

I have made a small proof of concept:
http://www.afdelingp.dk/files/memory.c

MEM_BEGIN opens a new scope.
MEM_END closes the current scope.
MEM_SAVE(ptr) moves a chunk of memory to the parent scope.

mymalloc() and myfree() works as usual.

Is it a bad idea?

Morten

-- 
Morten K. Poulsen <morten@afdelingp.dk>
http://www.afdelingp.dk/

Re: memory management in mlmmj

[Christian Laursen]

"Morten K. Poulsen" <morten@afdelingp.dk> writes:

> Memory management makes up quite a large portion of the code, especially
> freeing memory at error conditions. How about we sacrifice a tiny bit of memory
> and performance and automate it?
> 
> The basic idea here is to have scopes of dynamic memory, a lot like local
> variables. At the beginning of each function we could begin a scope, and at the
> end of a function (and at returns) we could end it. If we need to return a
> pointer to some memory, we could move that chunk of memory up to the parent
> scope. I think that would work well, and make the code a lot more readable.
> 
> It is not new, I think PHP does something like this (where a request is the
> scope).

Apache has also done something similar for years. It is probably available in
APR, but it's probably not feasible to link against that monstrosity.

> Is it a bad idea?

Personally I like it, but since I'm not touching much of the code in question
my opinion doesn't count that much.

-- 
Christian Laursen

Re: memory management in mlmmj

[Mads Martin Joergensen]

* Morten K. Poulsen <morten@afdelingp.dk> [Mar 23. 2005 22:29]:
> Memory management makes up quite a large portion of the code, especially
> freeing memory at error conditions. How about we sacrifice a tiny bit of memory
> and performance and automate it?
> 
> The basic idea here is to have scopes of dynamic memory, a lot like local
> variables. At the beginning of each function we could begin a scope, and at the
> end of a function (and at returns) we could end it. If we need to return a
> pointer to some memory, we could move that chunk of memory up to the parent
> scope. I think that would work well, and make the code a lot more readable.
> 
> It is not new, I think PHP does something like this (where a request is the
> scope).
> 
> I have made a small proof of concept:
> http://www.afdelingp.dk/files/memory.c
> 
> MEM_BEGIN opens a new scope.
> MEM_END closes the current scope.
> MEM_SAVE(ptr) moves a chunk of memory to the parent scope.
> 
> mymalloc() and myfree() works as usual.
> 
> Is it a bad idea?

The idea is nice, but why start changing all this code that works
already with something that had no serious testing yet? There's so many
other places that could need some work before this.

Also we should have a look at:

 http://irccrew.org/~cras/security/c-guide.html and

and (from the dovecot imap server source):

 dovecot-0.99.14/src/lib/mempool*

before deciding on how to approach this. There's some really nice ideas
here as well.

-- 
Mads Martin Joergensen, http://mmj.dk
"Why make things difficult, when it is possible to make them cryptic
 and totally illogical, with just a little bit more effort?"
                                -- A. P. J.

Re: memory management in mlmmj

[Mads Toftum]

On Wed, Mar 23, 2005 at 11:44:36PM +0100, Christian Laursen wrote:
> Apache has also done something similar for years. It is probably available in
> APR, but it's probably not feasible to link against that monstrosity.
> 
Correct, it is in apr -
http://apr.apache.org/docs/apr/group__apr__pools.html
There is quite a few things in apr that might come in handy - first of
all there's the main portability issue and then along with the that we
would get things like ldap and database support (dbm, mysql, postgres and
sqllite). 

vh

Mads Toftum
-- 
`Darn it, who spiked my coffee with water?!' - lwall

Re: memory management in mlmmj

[Mads Martin Joergensen]

* Mads Toftum <mads@toftum.dk> [Mar 24. 2005 09:03]:
> > Apache has also done something similar for years. It is probably available in
> > APR, but it's probably not feasible to link against that monstrosity.
> > 
> Correct, it is in apr -
> http://apr.apache.org/docs/apr/group__apr__pools.html
> There is quite a few things in apr that might come in handy - first of
> all there's the main portability issue and then along with the that we

Main portability issue? Can you mention a platform where mlmmj is not
compiling and working as of today? Or are you referring to something
else?

> would get things like ldap and database support (dbm, mysql, postgres and
> sqllite). 

This should probably not go directly into mlmmj, but into wrapper
scripts with helper apps.

-- 
Mads Martin Joergensen, http://mmj.dk
"Why make things difficult, when it is possible to make them cryptic
 and totally illogical, with just a little bit more effort?"
                                -- A. P. J.

Re: memory management in mlmmj

[Mads Toftum]

On Thu, Mar 24, 2005 at 09:07:11AM +0100, Mads Martin Joergensen wrote:
> Main portability issue? Can you mention a platform where mlmmj is not
> compiling and working as of today? Or are you referring to something
> else?

Ah, sorry - that came out wrong - s/issue/focus/ probably. What I meant
to say is that the main goal of APR is portability.

vh

Mads Toftum
-- 
`Darn it, who spiked my coffee with water?!' - lwall

Re: memory management in mlmmj

[Mads Martin Joergensen]

* Mads Toftum <mads@toftum.dk> [Mar 24. 2005 09:11]:
> On Thu, Mar 24, 2005 at 09:07:11AM +0100, Mads Martin Joergensen wrote:
> > Main portability issue? Can you mention a platform where mlmmj is not
> > compiling and working as of today? Or are you referring to something
> > else?
> 
> Ah, sorry - that came out wrong - s/issue/focus/ probably. What I meant
> to say is that the main goal of APR is portability.

Ah, good to know :-)

We always went to quite some trouble to get mlmmj to compile and work
out of the box everywhere.

-- 
Mads Martin Joergensen, http://mmj.dk
"Why make things difficult, when it is possible to make them cryptic
 and totally illogical, with just a little bit more effort?"
                                -- A. P. J.

Re: memory management in mlmmj

[Morten K. Poulsen]

On Thu, Mar 24, 2005 at 01:38:44AM +0100, Mads Martin Joergensen wrote:
> > Is it a bad idea?
> 
> The idea is nice, but why start changing all this code that works already
> with something that had no serious testing yet? There's so many other places
> that could need some work before this.

True, but the code is getting big and relatively difficult to maintain.

> Also we should have a look at:
> 
>  http://irccrew.org/~cras/security/c-guide.html

Dovecot memory pools look very nice. And they are under the MIT license. I will
look at the source code soon.

On Wed, Mar 23, 2005 at 11:44:36PM +0100, Christian Laursen wrote:
> > It is not new, I think PHP does something like this (where a request is the
> > scope).
> 
> Apache has also done something similar for years. It is probably available in
> APR, but it's probably not feasible to link against that monstrosity.

I think you are right ;-)

Morten

-- 
Morten K. Poulsen <morten@afdelingp.dk>
http://www.afdelingp.dk/