archive directory only has one message in it

archive directory only has one message in it

[Jason Diamond]

Hi.

I just created my first mailing list today using mlmmj. Everything seems 
to be working except for the archive. When I look in the archive 
directory, it contains one file called "0". The contents of that file 
are the most recent message posted to the list. The rest of the messages 
seem to have been lost. Is this normal or might I have something 
misconfigured?

I'm using Exim 4 on a Debian Sarge machine. The version of mlmmj I'm 
running is 1.2.7 since that what was in Debian experimental. The 
ChangeLog didn't say any bug was fixed regarding archives for 1.2.8 so I 
haven't tried doing a manual upgrade to that version.

Any ideas? Thanks.

-- 
Jason

Re: archive directory only has one message in it

[Joel Aelwyn]

[-- Attachment #1: Type: text/plain, Size: 2354 bytes --]

Jason Diamond wrote:
> Hi.
> 
> I just created my first mailing list today using mlmmj. Everything seems 
> to be working except for the archive. When I look in the archive 
> directory, it contains one file called "0". The contents of that file 
> are the most recent message posted to the list. The rest of the messages 
> seem to have been lost. Is this normal or might I have something 
> misconfigured?
> 
> I'm using Exim 4 on a Debian Sarge machine. The version of mlmmj I'm 
> running is 1.2.7 since that what was in Debian experimental. The 
> ChangeLog didn't say any bug was fixed regarding archives for 1.2.8 so I 
> haven't tried doing a manual upgrade to that version.
> 
> Any ideas? Thanks.

My copy, from the same source but hand-built to handle a few other tweaks 
(like my variant of "support characters other than + as a separator", which I 
need to tweak slightly given something pointed out on the list), certainly 
doesn't appear to have this issue. However, there is one thing you should 
probably check, given the description of the problem: make sure the 'index' 
file exists, and is writeable, since that appears to be what mlmmj uses to 
keep track of the archive's last posting number (and thus, if it doesn't 
exist, it probably thinks that it should write out '0', and does so... and if 
it can't create the index file, well... lather, rinse, repeat). Did you check 
your logfile to see if it reported anything?

Wishlist items:

*) Detect if the index file fails to write out, and if so, store the message 
in a temporary area with a guaranteed-unique name (it might start with a hash 
of the entire message, and then append '.1', '.2', etc, if there are other 
copies...)

*) Detect a pre-existing file, and don't write to it if one exists (I think 
O_EXCL is generally available these days?) This one is pretty crucial, 
actually, if there is ever any intent of allowing 'mere mortals' to touch any 
part of the queue system, but delivery runs as mlmmj; otherwise it's prone to 
symlink attacks, etc. Very bad. If the file already exists, something has gone 
horribly wrong in the first place, and the message should probably be handed 
as above; assume this is for the humans to untangle, but throw lots of klaxons 
about it (mail the list admin, log it, etc.) so that they know.
-- 
Joel Aelwyn <joel@lightbearer.com>

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 187 bytes --]

Re: archive directory only has one message in it

[Jason Diamond]

[-- Attachment #1: Type: text/plain, Size: 1532 bytes --]

On 9/9/05, Joel Aelwyn <joel@lightbearer.com> wrote:
> 
> 
> My copy, from the same source but hand-built to handle a few other tweaks
> (like my variant of "support characters other than + as a separator", 
> which I
> need to tweak slightly given something pointed out on the list), certainly
> doesn't appear to have this issue. However, there is one thing you should
> probably check, given the description of the problem: make sure the 
> 'index'
> file exists, and is writeable, since that appears to be what mlmmj uses to
> keep track of the archive's last posting number (and thus, if it doesn't
> exist, it probably thinks that it should write out '0', and does so... and 
> if
> it can't create the index file, well... lather, rinse, repeat). Did you 
> check
> your logfile to see if it reported anything?
> 

I think this is the problem.

The index file does exist but it's owned by root:root (the user I was when I 
ran mlmmj-make-ml) and not writable by anybody else.

I have exim launching the mlmmj-recieve program as nobody:nogroup since 
that's what all of the files in my mailing list directory (except index) 
were owned by. Could this be a bug in mlmmj-make-ml?

I didn't realize that index needed to be written to by the user running 
mlmmj-receive. This is definitely good to know.

I only have two log files in my mailing list directory: 
mlmmj-maintd.lastrun.log and mlmmj.operation.log. Neither of them mentions 
not being able to write to index.

Thanks!

-- 
Jason

[-- Attachment #2: Type: text/html, Size: 1843 bytes --]

Re: archive directory only has one message in it

[Søren Boll Overgaard]

On Fri, Sep 09, 2005 at 07:50:31AM -0700, Jason Diamond wrote:
> 
> The index file does exist but it's owned by root:root (the user I was when I 
> ran mlmmj-make-ml) and not writable by anybody else.
> 
> I have exim launching the mlmmj-recieve program as nobody:nogroup since 
> that's what all of the files in my mailing list directory (except index) 
> were owned by. Could this be a bug in mlmmj-make-ml?

Please note that the mlmmj-make-ml script in Debian is heavily patched to
support Debians file hierachy. Ie. bugs should be reported to the Debian bug
tracking system. I will look into your bug report tonight or tomorrow, as time 
permits.


-- 
Søren O.                                           ,''`.
                                                  : :' :
GPG key id: 0x1EB2DE66                            `. `'
GPG signed mail preferred.                          `-

Re: archive directory only has one message in it

[Søren Boll Overgaard]

Hello,

On Fri, Sep 09, 2005 at 07:50:31AM -0700, Jason Diamond wrote:
> > However, there is one thing you should
> > probably check, given the description of the problem: make sure the 
> > 'index'
> > file exists, and is writeable, since that appears to be what mlmmj uses to
> > keep track of the archive's last posting number (and thus, if it doesn't
> > exist, it probably thinks that it should write out '0', and does so... and 
> > if
> > it can't create the index file, well... lather, rinse, repeat). Did you 
> > check
> > your logfile to see if it reported anything?
> > 
> 
> I think this is the problem.
> 
> The index file does exist but it's owned by root:root (the user I was when I 
> ran mlmmj-make-ml) and not writable by anybody else.
> 
> I have exim launching the mlmmj-recieve program as nobody:nogroup since 
> that's what all of the files in my mailing list directory (except index) 
> were owned by. Could this be a bug in mlmmj-make-ml?

This indeed turned out to be the problem. The patched version of mlmmj-make-ml
in 1.2.5 failed to set proper permissions in $LISTDIR/index. I've fixed this in
version 1.2.8-3 which I have just uploaded to the Debian unstable archive. If
you would like a copy of the package before it hits the mirrors, please let me 
know in private mail.

Also, to avoid Debian specific bugs related to mlmmj-make-ml making it past the
Debian Bug Tracking System, the Debian version of mlmmj-make-ml now tells users
where to report bugs.

> I only have two log files in my mailing list directory: 
> mlmmj-maintd.lastrun.log and mlmmj.operation.log. Neither of them mentions 
> not being able to write to index.

I would like to second the request for this sort of information to appear as a
warning in the logs.

-- 
Søren O.                                           ,''`.
                                                  : :' :
GPG key id: 0x1EB2DE66                            `. `'
GPG signed mail preferred.                          `-

Re: archive directory only has one message in it

[Mads Martin Joergensen]

* Søren Boll Overgaard <boll+mlmmj@fork.dk> [Sep 10. 2005 21:25]:
> > I only have two log files in my mailing list directory: 
> > mlmmj-maintd.lastrun.log and mlmmj.operation.log. Neither of them mentions 
> > not being able to write to index.
> 
> I would like to second the request for this sort of information to
> appear as a warning in the logs.

It does appear in syslog/mail. I know, I know--logging shouldn't be
happening in 3 different places, patches welcome to fix this.

-- 
Mads Martin Joergensen, http://mmj.dk
"Why make things difficult, when it is possible to make them cryptic
 and totally illogical, with just a little bit more effort?"
                                -- A. P. J.