From: Pablo Neira Ayuso <pablo@netfilter.org>
To: Eric Leblond <eric@regit.org>
Cc: netfilter-devel@vger.kernel.org
Subject: Re: [PATCH 09/11] db: add ring buffer for DB query
Date: Tue, 21 May 2013 15:38:02 +0200 [thread overview]
Message-ID: <20130521133802.GA5160@localhost> (raw)
In-Reply-To: <1368354569.22387.48.camel@ice-age.regit.org>
Hi Eric,
On Sun, May 12, 2013 at 12:29:29PM +0200, Eric Leblond wrote:
> Le samedi 11 mai 2013 à 21:29 +0200, Pablo Neira Ayuso a écrit :
> > On Fri, May 10, 2013 at 08:48:56AM +0200, Eric Leblond wrote:
> > > This patch adds an optional ring buffer option which modify
> > > the way database queries are made. The main thread is only handling
> > > kernel message reading and query formatting. The SQL request is made
> > > in a separate dedicated thread.
> > > The idea is to try to avoid buffer overrun by minimizing the time
> > > requested to treat kernel message. Doing synchronous SQL request, as
> > > it was made before was causing a delay which could cause some messages
> > > to be lost in case of burst from kernel side.
> >
> > Would be feasible to make asynchronous SQL requests instead, so you
> > can skip the use of pthread?
[...]
> From a lazy point of view, this would require to update all the database
> backends instead of simply updating the db abstraction layer as it is
> done with the proposed patch.
Fair enough, it provides a generic way to support this and the main
thread should be really taking care of netlink messages coming from
the kernel to avoid the overrun.
If the target is to overcome small bursts, this should help to relief
those peaks for some time, but at some point the intermediate queue
(the ring) will also get full if the situation lasts long.
Regards.
--
To unsubscribe from this list: send the line "unsubscribe netfilter-devel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
next prev parent reply other threads:[~2013-05-21 13:38 UTC|newest]
Thread overview: 15+ messages / expand[flat|nested] mbox.gz Atom feed top
2013-05-10 6:48 [PATCH RFC 0/11] ulogd2 db rework Eric Leblond
2013-05-10 6:48 ` [PATCH 01/11] ulogd: display stack during configuration Eric Leblond
2013-05-10 6:48 ` [PATCH 02/11] Fix automagic support of dbi, pcap and sqlite3 Eric Leblond
2013-05-10 6:48 ` [PATCH 03/11] postgresql: add sanity checking Eric Leblond
2013-05-10 6:48 ` [PATCH 04/11] mysql: " Eric Leblond
2013-05-10 6:48 ` [PATCH 05/11] sqlite3: " Eric Leblond
2013-05-10 6:48 ` [PATCH 06/11] db: store data in memory during database downtime Eric Leblond
2013-05-10 6:48 ` [PATCH 07/11] db: suppress field in db structure Eric Leblond
2013-05-10 6:48 ` [PATCH 08/11] db: use offset instead of direct pointer Eric Leblond
2013-05-10 6:48 ` [PATCH 09/11] db: add ring buffer for DB query Eric Leblond
2013-05-11 19:29 ` Pablo Neira Ayuso
2013-05-12 10:29 ` Eric Leblond
2013-05-21 13:38 ` Pablo Neira Ayuso [this message]
2013-05-10 6:48 ` [PATCH 10/11] db: disable SIGHUP if ring buffer is used Eric Leblond
2013-05-10 6:48 ` [PATCH 11/11] db: db ring has precedence over backlog Eric Leblond
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20130521133802.GA5160@localhost \
--to=pablo@netfilter.org \
--cc=eric@regit.org \
--cc=netfilter-devel@vger.kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).