tracking usage by mac address

tracking usage by mac address

[Henry Baxter]

Hello,

I have been reading this list for several months, and I've really 
enjoyed learning all that I have, thank you everybody for the 
opportunity to listen:)

Ultimately I am hoping to track the bandwidth usage of about 50 client 
computers through my router based on their MAC address. I understand 
that by simply writing a rule that does nothing to the packet, such as 
'iptables -A FORWARD -m <mac address>' I can parse the netfilter log and 
find out what I need. This seems rather convoluted though - getting 
netfilter to create a basically human readable log file, and then 
parsing it.

All of the network traffic is passing through unmanaged switches until 
finally hitting the interface on the router.

I'm sure this must have been done by many others before, so could 
anybody give me some idea of what the most common way to handle this 
situation would be?

I appreciate any input.

Henry Baxter

Re: tracking usage by mac address

[Chris Brenton]

On Sun, 2004-08-29 at 22:42, Henry Baxter wrote:
>
> Ultimately I am hoping to track the bandwidth usage of about 50 client 
> computers through my router based on their MAC address. I understand 
> that by simply writing a rule that does nothing to the packet, such as 
> 'iptables -A FORWARD -m <mac address>' I can parse the netfilter log and 
> find out what I need.

How about:
iptables -A FORWARD -m mac --mac-source <mac address 1> -j LOG "
CLIENT_1 "
iptables -A FORWARD -m mac --mac-source <mac address 2> -j LOG "
CLIENT_2 "

Then when you need a report, just run:
iptables -L -nvx

Now the caveat is this will only show you outbound traffic, not inbound.
In other words, there is no "--mac-destination" option so you can't log
reply packets based on MAC address. Kind of a bummer in your case
because if these clients are mostly downloading data, that's the
direction that's going to see the most bandwidth usage.

Two options:
1) Log by IP instead of MAC (maybe hand the clients fixed IP's via DHCP)
2) Use ebtables instead of iptables (90% certain it will work but have
not tried it)

HTH,
C

Re: tracking usage by mac address

[Torsten Luettgert]

On Mon, 2004-08-30 at 04:42, Henry Baxter wrote:
> Ultimately I am hoping to track the bandwidth usage of about 50 client 
> computers through my router based on their MAC address. I understand 
> that by simply writing a rule that does nothing to the packet, such as 
> 'iptables -A FORWARD -m <mac address>' I can parse the netfilter log and 
> find out what I need. This seems rather convoluted though - getting 
> netfilter to create a basically human readable log file, and then 
> parsing it.

You could also use ULOG and the ulog-acctd from
http://alioth.debian.org/projects/pkg-ulog-acctd/

(if you want to use this on RedHat/Fedora, I could send you my RPM
I made from it)

This also generates a somewhat user-readable log file which you'd need
to parse, but it can aggregate several packets (thus reducing the size
of the log file) and generate a Cisco-compatible traffic log file.
Parsers for that should not be hard to find.

Greetings,
Torsten

Re: tracking usage by mac address

[George Alexandru Dragoi]

You can use ip_conntrack this way

iptables -t nat -A PREROUTING -i $LANIF -s $LANIPCLIENT1 -m mac
--mac-source $CLIENT1_MAC_ADDRESS -j ACCEPT
iptables -t nat -A PREROUTING -i $LANIF -s $LANIPCLIENT2 -m mac
--mac-source $CLIENT2_MAC_ADDRESS -j ACCEPT
......
iptables -t nat -A PREROUTING -i $LANIF -j DROP

Then

iptables -A FORWARD -s $LANIPCLIENT1 -i $LANIF -o $INETIF #upload
iptables -A FORWARD -d $LANIPCLIENT1 -d $LANIF -o $INETIF #download

and to see the traffic, use
iptables -L FORWARD -nv
and look for those 2 rules. You can add a -j LOG target, or whatever.
This way you will see the client's download based on his mac, because
you allow beginning streams only with those macs

On Mon, 30 Aug 2004 13:34:58 +0200, Torsten Luettgert
<t.luettgert@pressestimmen.de> wrote:
> On Mon, 2004-08-30 at 04:42, Henry Baxter wrote:
> > Ultimately I am hoping to track the bandwidth usage of about 50 client
> > computers through my router based on their MAC address. I understand
> > that by simply writing a rule that does nothing to the packet, such as
> > 'iptables -A FORWARD -m <mac address>' I can parse the netfilter log and
> > find out what I need. This seems rather convoluted though - getting
> > netfilter to create a basically human readable log file, and then
> > parsing it.
> 
> You could also use ULOG and the ulog-acctd from
> http://alioth.debian.org/projects/pkg-ulog-acctd/
> 
> (if you want to use this on RedHat/Fedora, I could send you my RPM
> I made from it)
> 
> This also generates a somewhat user-readable log file which you'd need
> to parse, but it can aggregate several packets (thus reducing the size
> of the log file) and generate a Cisco-compatible traffic log file.
> Parsers for that should not be hard to find.
> 
> Greetings,
> Torsten
> 
> 


-- 
Bla bla

Re: tracking usage by mac address

[Jose Maria Lopez]

El lun, 30 de 08 de 2004 a las 04:42, Henry Baxter escribió:
> Hello,
> 
> I have been reading this list for several months, and I've really 
> enjoyed learning all that I have, thank you everybody for the 
> opportunity to listen:)
> 
> Ultimately I am hoping to track the bandwidth usage of about 50 client 
> computers through my router based on their MAC address. I understand 
> that by simply writing a rule that does nothing to the packet, such as 
> 'iptables -A FORWARD -m <mac address>' I can parse the netfilter log and 
> find out what I need. This seems rather convoluted though - getting 
> netfilter to create a basically human readable log file, and then 
> parsing it.
> 
> All of the network traffic is passing through unmanaged switches until 
> finally hitting the interface on the router.
> 
> I'm sure this must have been done by many others before, so could 
> anybody give me some idea of what the most common way to handle this 
> situation would be?
> 
> I appreciate any input.
> 
> Henry Baxter

If you don't have a big number of users you can do something like this:

iptables -N MACSTATS
iptables -A INPUT -j MACSTATS
iptables -A OUTPUT -j MACSTATS
iptables -A FORWARD -j MACSTATS
iptables -A MACSTATS -m mac --mac-source $CLIENT1_MAC_ADDRESS -j RETURN
iptables -A MACSTATS -m mac --mac-source $CLIENT2_MAC_ADDRESS -j RETURN
...

So you can read the data transfered by each client with the command:
iptables -L MACSTATS -nv

More or less this is what we do in our bastion-firewall-stats module
from our bastion-firewall GPL firewall, but we extract the counters with
C code to put it in a rrdtool database and then create graphs with the
data. If need code you can look at the source code of this addon from
our firewall.

-- 
Jose Maria Lopez Hernandez
Director Tecnico de bgSEC
jkerouac@bgsec.com
bgSEC Seguridad y Consultoria de Sistemas Informaticos
http://www.bgsec.com
ESPAÑA

The only people for me are the mad ones -- the ones who are mad to live,
mad to talk, mad to be saved, desirous of everything at the same time,
the ones who never yawn or say a commonplace thing, but burn, burn, burn
like fabulous yellow Roman candles.
                -- Jack Kerouac, "On the Road"

Re: tracking usage by mac address

[George Alexandru Dragoi]

Well, i don't know if you want to log EVERYTHING.
Remember ip_conntrackworkson streams, so you can log only NEW packets.
I have like 90 rules with -m mac like those i said before + several
port forwarding, on a P2 450Mhz, 100mbit internet connections, used a
lot, almoust all the time at 11MB/s at upload (exactly where those
rules aremostly hitted), and top says the sys load is arround 40% at
most when i have full bandwith in use, but i think it is not because
of the netfilter, but the PCI usage. Traffic at 50% usually needs much
less CPU, like 5-10%. I also have many other rules for SYN scan
limiting, bandwith counting, and so on.

On 30 Aug 2004 20:54:36 +0200, Jose Maria Lopez <jkerouac@eresmas.com> wrote:
> El lun, 30 de 08 de 2004 a las 04:42, Henry Baxter escribió:
> 
> 
> > Hello,
> >
> > I have been reading this list for several months, and I've really
> > enjoyed learning all that I have, thank you everybody for the
> > opportunity to listen:)
> >
> > Ultimately I am hoping to track the bandwidth usage of about 50 client
> > computers through my router based on their MAC address. I understand
> > that by simply writing a rule that does nothing to the packet, such as
> > 'iptables -A FORWARD -m <mac address>' I can parse the netfilter log and
> > find out what I need. This seems rather convoluted though - getting
> > netfilter to create a basically human readable log file, and then
> > parsing it.
> >
> > All of the network traffic is passing through unmanaged switches until
> > finally hitting the interface on the router.
> >
> > I'm sure this must have been done by many others before, so could
> > anybody give me some idea of what the most common way to handle this
> > situation would be?
> >
> > I appreciate any input.
> >
> > Henry Baxter
> 
> If you don't have a big number of users you can do something like this:
> 
> iptables -N MACSTATS
> iptables -A INPUT -j MACSTATS
> iptables -A OUTPUT -j MACSTATS
> iptables -A FORWARD -j MACSTATS
> iptables -A MACSTATS -m mac --mac-source $CLIENT1_MAC_ADDRESS -j RETURN
> iptables -A MACSTATS -m mac --mac-source $CLIENT2_MAC_ADDRESS -j RETURN
> ...
> 
> So you can read the data transfered by each client with the command:
> iptables -L MACSTATS -nv
> 
> More or less this is what we do in our bastion-firewall-stats module
> from our bastion-firewall GPL firewall, but we extract the counters with
> C code to put it in a rrdtool database and then create graphs with the
> data. If need code you can look at the source code of this addon from
> our firewall.
> 
> --
> Jose Maria Lopez Hernandez
> Director Tecnico de bgSEC
> jkerouac@bgsec.com
> bgSEC Seguridad y Consultoria de Sistemas Informaticos
> http://www.bgsec.com
> ESPAÑA
> 
> The only people for me are the mad ones -- the ones who are mad to live,
> mad to talk, mad to be saved, desirous of everything at the same time,
> the ones who never yawn or say a commonplace thing, but burn, burn, burn
> like fabulous yellow Roman candles.
>                 -- Jack Kerouac, "On the Road"
> 
> 


-- 
Bla bla

Re: tracking usage by mac address

[Henry Baxter]

Thank you Jose, I'm going to go with parsing the log with C code which I 
wouldn't mind writing - but if you could point me to your source, that 
would be very helpful. From the sounds of your setup George it should 
work great for us here (a tenth of your bandwidth usage!).

This mailing list rocks

Henry Baxter

George Alexandru Dragoi wrote:

>Well, i don't know if you want to log EVERYTHING.
>Remember ip_conntrackworkson streams, so you can log only NEW packets.
>I have like 90 rules with -m mac like those i said before + several
>port forwarding, on a P2 450Mhz, 100mbit internet connections, used a
>lot, almoust all the time at 11MB/s at upload (exactly where those
>rules aremostly hitted), and top says the sys load is arround 40% at
>most when i have full bandwith in use, but i think it is not because
>of the netfilter, but the PCI usage. Traffic at 50% usually needs much
>less CPU, like 5-10%. I also have many other rules for SYN scan
>limiting, bandwith counting, and so on.
>
>On 30 Aug 2004 20:54:36 +0200, Jose Maria Lopez <jkerouac@eresmas.com> wrote:
>  
>
>>El lun, 30 de 08 de 2004 a las 04:42, Henry Baxter escribió:
>>
>>
>>    
>>
>>>Hello,
>>>
>>>I have been reading this list for several months, and I've really
>>>enjoyed learning all that I have, thank you everybody for the
>>>opportunity to listen:)
>>>
>>>Ultimately I am hoping to track the bandwidth usage of about 50 client
>>>computers through my router based on their MAC address. I understand
>>>that by simply writing a rule that does nothing to the packet, such as
>>>'iptables -A FORWARD -m <mac address>' I can parse the netfilter log and
>>>find out what I need. This seems rather convoluted though - getting
>>>netfilter to create a basically human readable log file, and then
>>>parsing it.
>>>
>>>All of the network traffic is passing through unmanaged switches until
>>>finally hitting the interface on the router.
>>>
>>>I'm sure this must have been done by many others before, so could
>>>anybody give me some idea of what the most common way to handle this
>>>situation would be?
>>>
>>>I appreciate any input.
>>>
>>>Henry Baxter
>>>      
>>>
>>If you don't have a big number of users you can do something like this:
>>
>>iptables -N MACSTATS
>>iptables -A INPUT -j MACSTATS
>>iptables -A OUTPUT -j MACSTATS
>>iptables -A FORWARD -j MACSTATS
>>iptables -A MACSTATS -m mac --mac-source $CLIENT1_MAC_ADDRESS -j RETURN
>>iptables -A MACSTATS -m mac --mac-source $CLIENT2_MAC_ADDRESS -j RETURN
>>...
>>
>>So you can read the data transfered by each client with the command:
>>iptables -L MACSTATS -nv
>>
>>More or less this is what we do in our bastion-firewall-stats module
>>from our bastion-firewall GPL firewall, but we extract the counters with
>>C code to put it in a rrdtool database and then create graphs with the
>>data. If need code you can look at the source code of this addon from
>>our firewall.
>>
>>--
>>Jose Maria Lopez Hernandez
>>Director Tecnico de bgSEC
>>jkerouac@bgsec.com
>>bgSEC Seguridad y Consultoria de Sistemas Informaticos
>>http://www.bgsec.com
>>ESPAÑA
>>
>>The only people for me are the mad ones -- the ones who are mad to live,
>>mad to talk, mad to be saved, desirous of everything at the same time,
>>the ones who never yawn or say a commonplace thing, but burn, burn, burn
>>like fabulous yellow Roman candles.
>>                -- Jack Kerouac, "On the Road"
>>
>>
>>    
>>
>
>
>  
>

Re: tracking usage by mac address

[Jose Maria Lopez]

El mar, 31 de 08 de 2004 a las 02:34, Henry Baxter escribió:
> Thank you Jose, I'm going to go with parsing the log with C code which I 
> wouldn't mind writing - but if you could point me to your source, that 
> would be very helpful. From the sounds of your setup George it should 
> work great for us here (a tenth of your bandwidth usage!).
> 
> This mailing list rocks

You can find the source code at our downloads web page:

http://www.bgsec.com/downloads.html

or at the sourceforge web site:

http://bastionfirewall.sourceforge.net

the module you could use it's named
bastion-firewall-stats-1.0.src.tar.bz2

But it's you can also look at google for the Querying
Libiptc HOWTO, that was the document we used to write
our code. Just have in mind that it has a big bug, because
it allocates memory when it open the chain to read the
counters but it doesn't free the memory. If you use the
code in the examples you must do this after you read the
counters:

iptc_free(&h);

If you don't do it your program starts to eat memory and
grows and grows and grows until it uses all the memory in
the system.

We have sent an email to the author of the Howto, but we have
not received any response yet. Hope he reads it's mail and
the new version of the HOWTO adds this code.


-- 
Jose Maria Lopez Hernandez
Director Tecnico de bgSEC
jkerouac@bgsec.com
bgSEC Seguridad y Consultoria de Sistemas Informaticos
http://www.bgsec.com
ESPAÑA

The only people for me are the mad ones -- the ones who are mad to live,
mad to talk, mad to be saved, desirous of everything at the same time,
the ones who never yawn or say a commonplace thing, but burn, burn, burn
like fabulous yellow Roman candles.
                -- Jack Kerouac, "On the Road"

Re: tracking usage by mac address

[Jose Maria Lopez]

El lun, 30 de 08 de 2004 a las 22:37, George Alexandru Dragoi escribió:
> Well, i don't know if you want to log EVERYTHING.
> Remember ip_conntrackworkson streams, so you can log only NEW packets.
> I have like 90 rules with -m mac like those i said before + several
> port forwarding, on a P2 450Mhz, 100mbit internet connections, used a
> lot, almoust all the time at 11MB/s at upload (exactly where those
> rules aremostly hitted), and top says the sys load is arround 40% at
> most when i have full bandwith in use, but i think it is not because
> of the netfilter, but the PCI usage. Traffic at 50% usually needs much
> less CPU, like 5-10%. I also have many other rules for SYN scan
> limiting, bandwith counting, and so on.
> 

Obviously our system it's useful for a not huge set of
rules, we use it for a per service basis, not per IP or MAC.
We have been using it with a big number of rules (services)
and it works like a charm, without slowing the system, but
if you have a lot of MACs our system can be surely a bad
idea.

-- 
Jose Maria Lopez Hernandez
Director Tecnico de bgSEC
jkerouac@bgsec.com
bgSEC Seguridad y Consultoria de Sistemas Informaticos
http://www.bgsec.com
ESPAÑA

The only people for me are the mad ones -- the ones who are mad to live,
mad to talk, mad to be saved, desirous of everything at the same time,
the ones who never yawn or say a commonplace thing, but burn, burn, burn
like fabulous yellow Roman candles.
                -- Jack Kerouac, "On the Road"