lists.gentoo.org powered by mlmmj - bugs and a feature request

lists.gentoo.org powered by mlmmj - bugs and a feature request

[Robin H. Johnson]

[-- Attachment #1: Type: text/plain, Size: 1765 bytes --]

Hi,

I've taken over maintenance of lists.gentoo.org (all powered by mlmmj
:-), some 32k unique recipients!) from the previous maintainer, and
there's a few things we're working at. First of all, two bug reports,
then a feature request.

1.
There is occasionally some mail loss that we haven't gotten to
the bottom of. I can confirm 100% that Sendmail (not my choice, trust
me) definitely piped the email to mlmmj-recieve. There should have been
some 1900 emails that went out from the input, but a lot less than that
did go out.

The splitting in the sendmail logs appear that it only got some 800
recipients from mlmmj-send, but I cannot confirm that.

All suggestions on tracing this are welcome, losing mail sucks.

2.
This was reported by one of our users:
https://bugs.gentoo.org/show_bug.cgi?id=173159

mlmmj appends the footer, without being aware of the encoding of the
email. It is not safe to just directly append the footer if the mail has
Content-Transfer-Encoding: base64!

3.
Now the feature request.

Without going into all of the details, I need a sane way to have
mlmmj-process to call a different application (procmail for the moment)
where it would call mlmmj-send to send out the mass email to all list
recipients (so mlmmj-process has already completed all of it's
processing at this point).
Then I also need a way to call mlmmj-send in the same way later on. 

We're working on some asynchronous processing stuff, so breaking the
chain between mlmmj-process and mlmmj-send is the only sane way of
handling large quantities of messages.

-- 
Robin Hugh Johnson
Gentoo Linux Developer & Council Member
E-Mail     : robbat2@gentoo.org
GnuPG FP   : 11AC BA4F 4778 E3F6 E4ED  F38E B27B 944E 3488 4E85

[-- Attachment #2: Type: application/pgp-signature, Size: 321 bytes --]

Re: lists.gentoo.org powered by mlmmj - bugs and a feature request

[Mads Martin Joergensen]

* Robin H. Johnson <robbat2@gentoo.org> [Apr 11. 2007 11:53]:
> Hi,
> 
> I've taken over maintenance of lists.gentoo.org (all powered by mlmmj
> :-), some 32k unique recipients!) from the previous maintainer, and
> there's a few things we're working at. First of all, two bug reports,
> then a feature request.
> 
> 1.
> There is occasionally some mail loss that we haven't gotten to
> the bottom of. I can confirm 100% that Sendmail (not my choice, trust
> me) definitely piped the email to mlmmj-recieve. There should have been
> some 1900 emails that went out from the input, but a lot less than that
> did go out.
> 
> The splitting in the sendmail logs appear that it only got some 800
> recipients from mlmmj-send, but I cannot confirm that.
> 
> All suggestions on tracing this are welcome, losing mail sucks.

If sendmail is not your choice, why not switch to Postfix which performs
wonderfully. SuSE is running lists.opensuse.org without a hitch with
mlmmj + postfix.

> 2.
> This was reported by one of our users:
> https://bugs.gentoo.org/show_bug.cgi?id\x173159
> 
> mlmmj appends the footer, without being aware of the encoding of the
> email. It is not safe to just directly append the footer if the mail has
> Content-Transfer-Encoding: base64!

There's a patch from Jakob Hirsch for handling this.

> 3.
> Now the feature request.
> 
> Without going into all of the details, I need a sane way to have
> mlmmj-process to call a different application (procmail for the moment)
> where it would call mlmmj-send to send out the mass email to all list
> recipients (so mlmmj-process has already completed all of it's
> processing at this point).
> Then I also need a way to call mlmmj-send in the same way later on. 
> 
> We're working on some asynchronous processing stuff, so breaking the
> chain between mlmmj-process and mlmmj-send is the only sane way of
> handling large quantities of messages.

You can always just replace any mlmmj binary with one of your choice.

For instance mlmmj-process (yours) doing stuff and finally invokes the
mlmmj binaries the same way as the original one would.

-- 
Mads Martin Joergensen, http://mmj.dk
"Why make things difficult, when it is possible to make them cryptic
 and totally illogical, with just a little bit more effort?"
                                 -- A. P. J.

Re: lists.gentoo.org powered by mlmmj - bugs and a feature request

[Andrea Barisani]

On Wed, Apr 11, 2007 at 02:08:24PM +0200, Mads Martin Joergensen wrote:
> * Robin H. Johnson <robbat2@gentoo.org> [Apr 11. 2007 11:53]:
> > Hi,
> > 
> > I've taken over maintenance of lists.gentoo.org (all powered by mlmmj
> > :-), some 32k unique recipients!) from the previous maintainer, and
> > there's a few things we're working at. First of all, two bug reports,
> > then a feature request.
> > 
> > 1.
> > There is occasionally some mail loss that we haven't gotten to
> > the bottom of. I can confirm 100% that Sendmail (not my choice, trust
> > me) definitely piped the email to mlmmj-recieve. There should have been
> > some 1900 emails that went out from the input, but a lot less than that
> > did go out.
> > 
> > The splitting in the sendmail logs appear that it only got some 800
> > recipients from mlmmj-send, but I cannot confirm that.
> > 
> > All suggestions on tracing this are welcome, losing mail sucks.
> 
> If sendmail is not your choice, why not switch to Postfix which performs
> wonderfully. SuSE is running lists.opensuse.org without a hitch with
> mlmmj + postfix.
>

For the record Sendmail has been gentoo infrastructure team choice and
there's nothing wrong with it. There's no evidence that the issues we are
getting are related to the MTA which is working just fine.

</end_mta_flamewar> ;-P

Cheers

-- 
Andrea Barisani <lcars@gentoo.org>                            .*.
Gentoo Linux Infrastructure Developer                          V
                                                             (   )
PGP-Key 0x864C9B9E http://dev.gentoo.org/~lcars/pubkey.asc   (   )
    0A76 074A 02CD E989 CE7F AC3F DA47 578E 864C 9B9E        ^^_^^
      "Pluralitas non est ponenda sine necessitate"