netfilter-devel.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
From: Eric Leblond <eric@regit.org>
To: netfilter-devel@vger.kernel.org
Cc: eric@regit.org
Subject: [Ulogd PATCH 0/4] Implement event loss prevention for db output
Date: Mon, 18 Mar 2013 01:01:20 +0100	[thread overview]
Message-ID: <1363564884-5957-1-git-send-email-eric@regit.org> (raw)


Hello,

The first three patches are cleaning that could be necessary to avoid
issue with the fourth one who does the real job.

The idea is to store the queries in memory if the database goes down and
to proceed to the insertion when it gets back up. To do so, a chained list
of queries is attached to the database instance and queries are added to
it till a memory cap is not reached. When the database came back, the
queries are played in order so it should not cause any issue for any
potential triggers.

This new feature is inactive by default and can be activated by setting
"backlog_memcap" to a non null value in the database instance configuration.

To avoid to cause overun on the event capture by inserting too many things
at once at recovery (and thus blocking capture for a certain time), a maximum
of "backlog_oneshot_requests" " are inserted at a time.

Patchset statistics:
 include/ulogd/db.h                    |   34 ++++++++++--
 output/mysql/ulogd_output_MYSQL.c     |    4 +-
 output/pgsql/ulogd_output_PGSQL.c     |    4 +-
 output/sqlite3/ulogd_output_SQLITE3.c |    2 +
 ulogd.conf.in                         |    9 +++
 util/db.c                             |  170 +++++++++++++++++++++++++++++++++++++++++++++++----------
 6 files changed, 188 insertions(+), 35 deletions(-)

BR,
--
Eric Leblond <eric@regit.org>

             reply	other threads:[~2013-03-18  0:01 UTC|newest]

Thread overview: 5+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2013-03-18  0:01 Eric Leblond [this message]
2013-03-18  0:01 ` [Ulogd PATCH 1/4] postgresql: add sanity checking Eric Leblond
2013-03-18  0:01 ` [Ulogd PATCH 2/4] mysql: " Eric Leblond
2013-03-18  0:01 ` [Ulogd PATCH 3/4] sqlite3: " Eric Leblond
2013-03-18  0:01 ` [Ulogd PATCH 4/4] db: store data in memory during database downtime Eric Leblond

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=1363564884-5957-1-git-send-email-eric@regit.org \
    --to=eric@regit.org \
    --cc=netfilter-devel@vger.kernel.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).