Re: NFLOG - missing packets?

[Patrick McHardy <kaber@trash.net>]

Pablo Neira Ayuso wrote:
> Anton wrote:
>> Just some extra results,
>>
>> seems DB drivers does such a difference, since while logging 
>> to LOGEMU (only) target - I've go result which looks true. 
>> But - If I enable DB logger - results in DB and LOGEMU - 
>> are the same. Looks like DB transfers makes ULOG to not 
>> accept packets from kernelspace
> 
> The problem is netlink that cannot back off. Netlink is the underlying
> communication subsystem that we use to communicate kernel with usepace
> space. Since Netlink is unreliable, some log messages can vanish under
> heavy load. I guess that database insertions consumes lots of CPU
> resouces. Thus, doing online database logging in a scalable manner turns
> really hard. Instead, if you need scalability, I'd suggest to use logemu
> or whatever plain text logging facility and then convert it to a
> database *offline* if you really need advanced queries.


Yeah, but what we can do is check whether the message
was successfully transmitted in the kernel and drop
the packet in case it wasn't. That should catch 99.9%
of all error cases since a slow databse effectively
only causes the process to read less often from the
netlink socket.

I still have a very old and unfinished patch for this
somewhere ...