Filtering pppoed frames

Filtering pppoed frames

[Marius Nicolae]

Hi,

Im not sure if this is the right list but here we go. In our small ISP
we've implemented a pppoe server using rp-pppoe  (
http://www.roaringpenguin.com/products/pppoe ) on a Ubuntu server
Lucid (last TLS) using a 2.6.32-34 linux kernel. For some time we're
also hit by the problem described here
http://lists.roaringpenguin.com/pipermail/rp-pppoe/2010q3/000162.html
. Put it short, during peak hours some buggy pppoe clients are
flooding the the server with PADT frames sent in the name of other
clients disturbing the rest of the clients by increasing cpu load and
traffic throughput (pings is getting worse as well). I could
constantly monitor the buggy clients and ask the users to
replace/upgrade their buggy routers but we're searching for a way of
getting our server immune to such floods. Since in this scenario we
cannot prevent the clients to send frames I'm thinking it might help
to drop such frames as soon as they enter on the stack network. The
involved ethernet protocol is 0x8863. It's described in the
/etc/ethertypes like this:
============= 8< ============
PPP_DISC    8863            # PPPoE discovery messages
============= 8< ============

After studying the problem a little bit I was thinking to implement a
userspace application which would limit the packets in a manner
similar witth hashlimit helper from iptables but using as hashes the
frames' source mac. After that, other goodies like sending daily
reports with "outlaws" macs would had been implemented but that's
another story.

Since pppoed frames aren't ip packets (ethernet type 0x800) they
cannot be matched with iptables. I didn't saw any way of matching such
frames but to implement a kernel module registering a(some) netfillter
hook(s) - I hope the terminology is correct. The filtering can happen
directly in the kernel or packets can be queued to a userspace
application. Before doing that:
1. Is there a better and easier way of matching pppoed frames and
limit them in the way just described?
2. If not, do you know an open source project on which I might involve
and contribute with such functionality?

Thanks!
--
Best regards,
Marius Nicolae

Re: Filtering pppoed frames

[Andy Furniss]

Marius Nicolae wrote:
> Hi,
>
> Im not sure if this is the right list but here we go. In our small ISP
> we've implemented a pppoe server using rp-pppoe  (
> http://www.roaringpenguin.com/products/pppoe ) on a Ubuntu server
> Lucid (last TLS) using a 2.6.32-34 linux kernel. For some time we're
> also hit by the problem described here
> http://lists.roaringpenguin.com/pipermail/rp-pppoe/2010q3/000162.html
> . Put it short, during peak hours some buggy pppoe clients are
> flooding the the server with PADT frames sent in the name of other
> clients disturbing the rest of the clients by increasing cpu load and
> traffic throughput (pings is getting worse as well). I could
> constantly monitor the buggy clients and ask the users to
> replace/upgrade their buggy routers but we're searching for a way of
> getting our server immune to such floods. Since in this scenario we
> cannot prevent the clients to send frames I'm thinking it might help
> to drop such frames as soon as they enter on the stack network. The
> involved ethernet protocol is 0x8863. It's described in the
> /etc/ethertypes like this:
> ============= 8<  ============
> PPP_DISC    8863            # PPPoE discovery messages
> ============= 8<  ============
>
> After studying the problem a little bit I was thinking to implement a
> userspace application which would limit the packets in a manner
> similar witth hashlimit helper from iptables but using as hashes the
> frames' source mac. After that, other goodies like sending daily
> reports with "outlaws" macs would had been implemented but that's
> another story.
>
> Since pppoed frames aren't ip packets (ethernet type 0x800) they
> cannot be matched with iptables. I didn't saw any way of matching such
> frames but to implement a kernel module registering a(some) netfillter
> hook(s) - I hope the terminology is correct. The filtering can happen
> directly in the kernel or packets can be queued to a userspace
> application. Before doing that:
> 1. Is there a better and easier way of matching pppoed frames and
> limit them in the way just described?
> 2. If not, do you know an open source project on which I might involve
> and contribute with such functionality?

I don't know pppoe so am not sure it will help in this case, but you can 
match/drop/limit non ip with tc and a policer - though it's not going to 
be as sophisticated as you describe doing with iptables.

If it's possible to identify a naughty frame just from the macs and or 
contents of the frame alone, then dropping them should be quite easy.

If you can't identify from the frame alone and need state from the pppoe 
server or some statistics then it's going to be trickier.

Fwd: Filtering pppoed frames

[Marius Nicolae]

Sorry, I was replying only to Andy.

---------- Forwarded message ----------
From: Marius Nicolae <marius.e.nicolae@gmail.com>
Date: Fri, Dec 16, 2011 at 10:16 AM
Subject: Re: Filtering pppoed frames
To: Andy Furniss <andyqos@ukfsn.org>


Hi Andy,

Thanks for your response.

On Thu, Dec 15, 2011 at 11:09 PM, Andy Furniss <andyqos@ukfsn.org> wrote:
> Marius Nicolae wrote:
>>
>> Hi,
>>
>> Im not sure if this is the right list but here we go. In our small ISP
...
>> Before doing that:
>> 1. Is there a better and easier way of matching pppoed frames and
>> limit them in the way just described?
>> 2. If not, do you know an open source project on which I might involve
>> and contribute with such functionality?
>
>
> I don't know pppoe so am not sure it will help in this case, but you can
> match/drop/limit non ip with tc and a policer - though it's not going to be
> as sophisticated as you describe doing with iptables.
While contemplating the ebtables packet flow picture from here
http://ebtables.sourceforge.net/br_fw_ia/PacketFlow.png I was first
considering using ebtables for matching the frames but since the
server has no bridge the packets they where not hitting any ebtables
table. I was assuming the frames where traversing network stack
without any interest from ebtables or then iptables. Probably ebtables
sees the frame is coming from a interface which is not part of any
bridge and iptables sees isn't an IP frame; I don't know. Also from
that picture, since it's hit first, I was thinking to match or mark
them within tc ingress phase but again I saw no way of filtering or
sending them to userland for further investigations but simply
policing such frames I think is a no go.

>
> If it's possible to identify a naughty frame just from the macs and or
> contents of the frame alone, then dropping them should be quite easy.
>
> If you can't identify from the frame alone and need state from the pppoe
> server or some statistics then it's going to be trickier.
Yes, is possible to identify the frames alone from macs and ethernet
protocol only, in a stateless manner, but it must be rejected only the
"noisy" macs. As a very simplistic description the pppoed protocol is
used to create and terminate pppoe sessions (frames with 0x8864
ethernet protocol) which encapsulates IP frames by signing and even
encrypting them. Thus is very important to let the good and legitimate
macs to send/receive such frames in order to create/terminate pppoe
sessions.

>
>

--
Best regards,
Marius Nicolae


-- 
Numai bine,
Marius Nicolae

Filtering pppoed frames

[Marius Nicolae]

Sorry I was replying only to Andy and forgot the rule about unchanged
subject :(.


---------- Forwarded message ----------
From: Marius Nicolae <marius.e.nicolae@gmail.com>
Date: Fri, Dec 16, 2011 at 10:16 AM
Subject: Re: Filtering pppoed frames
To: Andy Furniss <andyqos@ukfsn.org>


Hi Andy,

Thanks for your response.

On Thu, Dec 15, 2011 at 11:09 PM, Andy Furniss <andyqos@ukfsn.org> wrote:
> Marius Nicolae wrote:
>>
>> Hi,
>>
>> Im not sure if this is the right list but here we go. In our small ISP
...
>> Before doing that:
>> 1. Is there a better and easier way of matching pppoed frames and
>> limit them in the way just described?
>> 2. If not, do you know an open source project on which I might involve
>> and contribute with such functionality?
>
>
> I don't know pppoe so am not sure it will help in this case, but you can
> match/drop/limit non ip with tc and a policer - though it's not going to be
> as sophisticated as you describe doing with iptables.
While contemplating the ebtables packet flow picture from here
http://ebtables.sourceforge.net/br_fw_ia/PacketFlow.png I was first
considering using ebtables for matching the frames but since the
server has no bridge the packets they where not hitting any ebtables
table. I was assuming the frames where traversing network stack
without any interest from ebtables or then iptables. Probably ebtables
sees the frame is coming from a interface which is not part of any
bridge and iptables sees isn't an IP frame; I don't know. Also from
that picture, since it's hit first, I was thinking to match or mark
them within tc ingress phase but again I saw no way of filtering or
sending them to userland for further investigations but simply
policing such frames I think is a no go.

>
> If it's possible to identify a naughty frame just from the macs and or
> contents of the frame alone, then dropping them should be quite easy.
>
> If you can't identify from the frame alone and need state from the pppoe
> server or some statistics then it's going to be trickier.
Yes, is possible to identify the frames alone from macs and ethernet
protocol only, in a stateless manner, but it must be rejected only the
"noisy" macs. As a very simplistic description the pppoed protocol is
used to create and terminate pppoe sessions (frames with 0x8864
ethernet protocol) which encapsulates IP frames by signing and even
encrypting them. Thus is very important to let the good and legitimate
macs to send/receive such frames in order to create/terminate pppoe
sessions.

>
>

--
Best regards,
Marius Nicolae

Re: Filtering pppoed frames

[James Lay]

> Marius Nicolae wrote:
>> Hi,
>>
>> Im not sure if this is the right list but here we go. In our small ISP
>> we've implemented a pppoe server using rp-pppoe  (
>> http://www.roaringpenguin.com/products/pppoe ) on a Ubuntu server
>> Lucid (last TLS) using a 2.6.32-34 linux kernel. For some time we're
>> also hit by the problem described here
>> http://lists.roaringpenguin.com/pipermail/rp-pppoe/2010q3/000162.html
>> . Put it short, during peak hours some buggy pppoe clients are
>> flooding the the server with PADT frames sent in the name of other
>> clients disturbing the rest of the clients by increasing cpu load and
>> traffic throughput (pings is getting worse as well). I could
>> constantly monitor the buggy clients and ask the users to
>> replace/upgrade their buggy routers but we're searching for a way of
>> getting our server immune to such floods. Since in this scenario we
>> cannot prevent the clients to send frames I'm thinking it might help
>> to drop such frames as soon as they enter on the stack network. The
>> involved ethernet protocol is 0x8863. It's described in the
>> /etc/ethertypes like this:
>> ============= 8<  ============
>> PPP_DISC    8863            # PPPoE discovery messages
>> ============= 8<  ============
>>
>> After studying the problem a little bit I was thinking to implement a
>> userspace application which would limit the packets in a manner
>> similar witth hashlimit helper from iptables but using as hashes the
>> frames' source mac. After that, other goodies like sending daily
>> reports with "outlaws" macs would had been implemented but that's
>> another story.
>>
>> Since pppoed frames aren't ip packets (ethernet type 0x800) they
>> cannot be matched with iptables. I didn't saw any way of matching such
>> frames but to implement a kernel module registering a(some) netfillter
>> hook(s) - I hope the terminology is correct. The filtering can happen
>> directly in the kernel or packets can be queued to a userspace
>> application. Before doing that:
>> 1. Is there a better and easier way of matching pppoed frames and
>> limit them in the way just described?
>> 2. If not, do you know an open source project on which I might involve
>> and contribute with such functionality?

I'd install tshark on the ppp-oe server and try any one of the below:

sudo tshark -t ad -n -i eth* pppoed

it won't block anything, but may point you in the right direction on where
they're coming from.

James

Re: Filtering pppoed frames

[Andy Furniss]

Marius Nicolae wrote:

>> If you can't identify from the frame alone and need state from the pppoe
>> server or some statistics then it's going to be trickier.
> Yes, is possible to identify the frames alone from macs and ethernet
> protocol only, in a stateless manner, but it must be rejected only the
> "noisy" macs. As a very simplistic description the pppoed protocol is
> used to create and terminate pppoe sessions (frames with 0x8864
> ethernet protocol) which encapsulates IP frames by signing and even
> encrypting them. Thus is very important to let the good and legitimate
> macs to send/receive such frames in order to create/terminate pppoe
> sessions.

The only tc thing I can think of would be to keep a list of bad macs - 
maybe from a script parsing pppoe server logs or something and then 
periodically replace a tc filter that matches and drops those macs + 
protocol 0x8864.

Re: Filtering pppoed frames

[Marius Nicolae]

Hi Andy,

On Fri, Dec 16, 2011 at 9:46 PM, Andy Furniss <andyqos@ukfsn.org> wrote:
> Marius Nicolae wrote:
>
>>> If you can't identify from the frame alone and need state from the pppoe
>>> server or some statistics then it's going to be trickier.
>>
>> Yes, is possible to identify the frames alone from macs and ethernet
>> protocol only, in a stateless manner, but it must be rejected only the
>> "noisy" macs. As a very simplistic description the pppoed protocol is
>> used to create and terminate pppoe sessions (frames with 0x8864
>> ethernet protocol) which encapsulates IP frames by signing and even
>> encrypting them. Thus is very important to let the good and legitimate
>> macs to send/receive such frames in order to create/terminate pppoe
>> sessions.
>
>
> The only tc thing I can think of would be to keep a list of bad macs - maybe
> from a script parsing pppoe server logs or something and then periodically
> replace a tc filter that matches and drops those macs + protocol 0x8864.
Good idea. Another thing I can think of based on what you've said
would be to perform ingress policing and to assure a very small amount
of bandwidth for such frames. Everything which would exceed that
amount would be dropped. Simply put, the idea is to drop the frames
based on their generated traffic and not based on their number
(packets/second).

Currently I'm learning the Linux kernel network stack internals and
see if there is a more robust solution. I think netfilter is a no go.
It refers to IP packets (I guess its name mislead me) and it is placed
to high on the network stack. It appears it's needed a kernel module
right after device driver which register for a specific ethernet
protocol (with dev_add_pack() ) and receives the frames it registered
for. I've just found a good article as a starting point just for that
http://www.phrack.org/archives/55/p55_0x0c_Building Into The Linux
Network Layer_by_lifeline & kossak.txt . pppoe kernel module also
register for receiving such frames. Another idea would be to see if is
possible to investigate packets from there.
-- 
Best regards,
Marius Nicolae

Re: Filtering pppoed frames

[Marius Nicolae]

Hi James,

...
>>> 2. If not, do you know an open source project on which I might involve
>>> and contribute with such functionality?
>
>I'd install tshark on the ppp-oe server and try any one of the below:
>
>sudo tshark -t ad -n -i eth* pppoed
>
>it won't block anything, but may point you in the right direction on where
>they're coming from.
This is how we are currently doing but a more robust and automated way
we are searching for.

>
>James
-- 
Best regards,
Marius Nicolae