NFLOG Performance issues ...

All of lore.kernel.org
 help / color / mirror / Atom feed

* NFLOG Performance issues ...
@ 2017-08-30 17:39 Akshat Kakkar
  2017-08-31 18:39 ` Akshat Kakkar
  0 siblings, 1 reply; 6+ messages in thread
From: Akshat Kakkar @ 2017-08-30 17:39 UTC (permalink / raw)
  To: netfilter

 am using ulogd2 to log iptables activity.
However, when using pgsql as output plugin ... performance is very
very sluggish. (~130-150 entries per second)

To enhance performance I am trying

modprobe ipt_ULOG nlbufsiz=65535 flushtimeout=1000

but this gives error : ipt_ULOG module not found.

On the same lines, I tried

modprobe ipt_NFLOG nlbufsiz=65535 flushtimeout=1000
It didnt give any error !!!

But still there is no increase in performance.
Are these values effective?

I am also using  --nflog-threshold 50 in iptables rule.

I am setting buffer_size as
netlink_socket_buffer_size=104857600
netlink_socket_buffer_maxsize=1048576000

When running ulog, it gives message of setting buffer size as
21708600. Though it didnt give any message like
"ulogd_inppkt_NFLOG.c:443 We are losing events, increasing buffer size
to xxxxxx"
but again why it is only setting buffer size as 21708600 though I have
set in config as 104857600?

^ permalink raw reply	[flat|nested] 6+ messages in thread

* Re: NFLOG Performance issues ...
  2017-08-30 17:39 NFLOG Performance issues Akshat Kakkar
@ 2017-08-31 18:39 ` Akshat Kakkar
  2017-09-01  9:43   ` Pablo Neira Ayuso
  0 siblings, 1 reply; 6+ messages in thread
From: Akshat Kakkar @ 2017-08-31 18:39 UTC (permalink / raw)
  To: netfilter

Please help .... I hope I am not alone!!!

On 8/30/17, Akshat Kakkar <akshat.1984@gmail.com> wrote:
>  am using ulogd2 to log iptables activity.
> However, when using pgsql as output plugin ... performance is very
> very sluggish. (~130-150 entries per second)
>
> To enhance performance I am trying
>
> modprobe ipt_ULOG nlbufsiz=65535 flushtimeout=1000
>
> but this gives error : ipt_ULOG module not found.
>
>
> On the same lines, I tried
>
> modprobe ipt_NFLOG nlbufsiz=65535 flushtimeout=1000
> It didnt give any error !!!
>
> But still there is no increase in performance.
> Are these values effective?
>
> I am also using  --nflog-threshold 50 in iptables rule.
>
> I am setting buffer_size as
> netlink_socket_buffer_size=104857600
> netlink_socket_buffer_maxsize=1048576000
>
> When running ulog, it gives message of setting buffer size as
> 21708600. Though it didnt give any message like
> "ulogd_inppkt_NFLOG.c:443 We are losing events, increasing buffer size
> to xxxxxx"
> but again why it is only setting buffer size as 21708600 though I have
> set in config as 104857600?
>

^ permalink raw reply	[flat|nested] 6+ messages in thread

* Re: NFLOG Performance issues ...
  2017-08-31 18:39 ` Akshat Kakkar
@ 2017-09-01  9:43   ` Pablo Neira Ayuso
  2017-09-02 16:10     ` Akshat Kakkar
  0 siblings, 1 reply; 6+ messages in thread
From: Pablo Neira Ayuso @ 2017-09-01  9:43 UTC (permalink / raw)
  To: Akshat Kakkar; +Cc: netfilter

On Fri, Sep 01, 2017 at 12:09:45AM +0530, Akshat Kakkar wrote:
> Please help .... I hope I am not alone!!!

http://netfilter.org/mailinglists.html#list-rules

^ permalink raw reply	[flat|nested] 6+ messages in thread

* Re: NFLOG Performance issues ...
  2017-09-01  9:43   ` Pablo Neira Ayuso
@ 2017-09-02 16:10     ` Akshat Kakkar
  2017-09-05 12:50       ` Akshat Kakkar
  0 siblings, 1 reply; 6+ messages in thread
From: Akshat Kakkar @ 2017-09-02 16:10 UTC (permalink / raw)
  To: Pablo Neira Ayuso; +Cc: netfilter

Apologies for missing out on those. I thought those are only for
subject. Anyways, will keep a note in future.

But, has anybody faced or experienced similar issues with NFLOG performance ?
Besides, for last 2 years or more ... It has not seen any much direct
development also. So, is it like some other utility need to be used?

^ permalink raw reply	[flat|nested] 6+ messages in thread

* Re: NFLOG Performance issues ...
  2017-09-02 16:10     ` Akshat Kakkar
@ 2017-09-05 12:50       ` Akshat Kakkar
  2017-09-13  9:36         ` Arturo Borrero Gonzalez
  0 siblings, 1 reply; 6+ messages in thread
From: Akshat Kakkar @ 2017-09-05 12:50 UTC (permalink / raw)
  To: Pablo Neira Ayuso; +Cc: netfilter@vger.kernel.org

I am not able to log more than 150/sec.

Can anybody share their performance results when writing logs from
NFLOG/ULOG to postgres or mysql.

Please share your performance parameters too.

Thanks.

^ permalink raw reply	[flat|nested] 6+ messages in thread

* Re: NFLOG Performance issues ...
  2017-09-05 12:50       ` Akshat Kakkar
@ 2017-09-13  9:36         ` Arturo Borrero Gonzalez
  0 siblings, 0 replies; 6+ messages in thread
From: Arturo Borrero Gonzalez @ 2017-09-13  9:36 UTC (permalink / raw)
  To: Akshat Kakkar; +Cc: Pablo Neira Ayuso, netfilter@vger.kernel.org

On 5 September 2017 at 14:50, Akshat Kakkar <akshat.1984@gmail.com> wrote:
> I am not able to log more than 150/sec.
>
> Can anybody share their performance results when writing logs from
> NFLOG/ULOG to postgres or mysql.
>
> Please share your performance parameters too.
>

Hi,

I sent a patch other day regarding performance of ulogd2 [0].
Please, read the patch description, where I summarice different
approaches for increasing the performance of the daemon.

It would be great if could give a test to the patch and report back :-)

[0] http://patchwork.ozlabs.org/patch/810976/

^ permalink raw reply	[flat|nested] 6+ messages in thread

end of thread, other threads:[~2017-09-13  9:36 UTC | newest]

Thread overview: 6+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2017-08-30 17:39 NFLOG Performance issues Akshat Kakkar
2017-08-31 18:39 ` Akshat Kakkar
2017-09-01  9:43   ` Pablo Neira Ayuso
2017-09-02 16:10     ` Akshat Kakkar
2017-09-05 12:50       ` Akshat Kakkar
2017-09-13  9:36         ` Arturo Borrero Gonzalez

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.