Re: Does anyone have any tips for efficient lei and dovecot integration?

[Eric Wong <e@80x24.org>]

James Bottomley <James.Bottomley@HansenPartnership.com> wrote:
> Thanks.  --no-import-before seems to have stopped store growing ... is
> there any way I can shrink it?

Not easily in a fine-grained way.  I *think*
`rm -rf ~/.local/share/lei/store' (but keeping
~/.local/share/lei/saved-searches) should be OK but I haven't
tested.  Theoretically we could add support for purging
individual messages from it (like public-inbox-purge).

You can also use normal git-gc || SQLite VACUUM ||
xapian-compact to reduce space losslessly.  We can probably add
`lei compact' to wrap xapian-compact similar to what
public-inbox-compact does; and probably add SQLite VACUUM
support to both of those xapian-compact wrappers.

> On Fri, 2025-06-06 at 18:02 +0000, Eric Wong wrote:
> > AFAIK, dovecot uses inotify||kevent to detect changes to its
> > Maildirs.  I never needed to run `doveadm force-resync' after
> > writing with 3rd-party tools (e.g. lei, mutt) to Maildirs used
> > by dovecot.
> 
> Well, it can, yes, but my setup is a fraction of a terrabyte of imap
> running all my subscriptions, which is pretty huge and fanotify can be
> a bit unscalable in something that big, so I have it turned off and I
> simply used dovecot-lda for local delivery via procmail to make sure
> everything got noticed immediately.

Ouch, yeah.

I actually have an idea to flip things by exposing lei/store
data as Maildirs via FUSE3 so there'd be no need for Maildirs
on regular FSes at all.

I'm not sure if the performance of the mainline perl
implementation + FUSE3 shim will be fast enough to handle
lots of small files even w/ readdirplus.  Or just bypass Perl
and just use FUSE3+SQLite+git but there's still syscall
amplification from all those layers.


sidenote: I completely forgot fanotify exists since the
early versions required CAP_SYS_ADMIN :x