Query: Can Netfilter inspect xml soap traffic

Query: Can Netfilter inspect xml soap traffic

[william fitzgerald]

Dear Netfilter Experts,

Can Netfilter/iptables inspect xml/soap messages as xml based firewalls do?

Does the Layer-7 module have enough "smarts" to inspect web service 
messages.

I am asking in regard to the role of Network Access Control firewalls 
such as iptables within a dedicated enterprise web service SOA environment.

I have seen some posts that suggest that firewalls are now obsolete, 
particularly NACs, in regard to web services (everything is over http 
hence less effect restrictions).

However, my opinion is that its not as simple as opening ports 80 and 
443 to tunnel SOAP messages.

For example, I may want to restrict IP ranges, maybe I have some 
business partners and I only want them accessing the web service. Or 
maybe I need to control DoS attacks to web services.

I think if iptables has also the ability to deep packet inspect xml 
messages it then demonstrates that there is still an importance for NAC 
based firewalls.

All pointers to documentation and your comments are welcome.

I look forward to your support,
regards,
Will.

-- 
William M. Fitzgerald,
PhD Student,
Telecommunications Software & Systems Group,
ArcLabs Research and Innovation Centre,
Waterford Institute of Technology,
WIT West Campus,
Carriganore,
Waterford.
Office Ph: +353 51 302937
Mobile Ph: +353 87 9527083
Web: www.williamfitzgerald.org
      www.linkedin.com/in/williamfitzgerald
      www.ryze.com/go/wfitzgerald

Re: Query: Can Netfilter inspect xml soap traffic

[Grant Taylor]

On 03/25/08 10:01, william fitzgerald wrote:
> Can Netfilter/iptables inspect xml/soap messages as xml based 
> firewalls do?

Is NetFilter / IPTables capable of inspecting layer 7 traffic, yes. 
However it will probably be much more difficult to do than you might 
think.  I would expect it to be much like trying to write a regular 
expression in assembly verses trying to do it in Perl.  It can be done, 
but...

One thing to keep in mind is that the Layer-7 module only looks at some 
of the packet.  Here is a quote from the website:

"... l7-filter only looks at the first 10 packets or 2kB of each 
connection, whichever is smaller. ..."

> Does the Layer-7 module have enough "smarts" to inspect web service 
> messages.

I don't think the "smarts" you are referring to are (or should be) in 
the Layer-7 module.  Keep in mind that the Layer-7 module is a match 
extension as in does this packet have this data at layer 7.  The logic 
behind what to do with what match(es) and how to chain them together to 
make decisions there on (IMHO) should *NOT* be in the Layer-7 module, 
but rather in how you build your rules.

You will need to take in to account all the variances in the traffic 
that could happen.  When speaking SMTP, I can either HELO or EHLO with 
either a name or an IP.  Thus with in the first few packets you already 
have four different possibilities *IF* I play by the rules.  Your 
pattern will have to be very flexible.

> I am asking in regard to the role of Network Access Control firewalls 
> such as iptables within a dedicated enterprise web service SOA 
> environment.

You may have better luck taking the packet and passing it to user space 
and writing an application layer gateway (a.k.a. ALG) and having the ALG 
do the business logic of the filtering for you.

> I have seen some posts that suggest that firewalls are now obsolete, 
> particularly NACs, in regard to web services (everything is over http 
> hence less effect restrictions).

Firewalls are GREAT at filtering on layer 2 or layer 3 (depending what 
they are designed for.  If you want to filter on a higher layer, you 
need to use something that is designed to filter on that layer.

> However, my opinion is that its not as simple as opening ports 80 and 
> 443 to tunnel SOAP messages.

You don't "just open ports".  You "open ports and send them in to the 
next layer of security".  Layer 3 firewall in front of your ALG and then 
let the ALG deal with what it knows about with out worrying about other 
nasty things.

> For example, I may want to restrict IP ranges, maybe I have some 
> business partners and I only want them accessing the web service. Or 
> maybe I need to control DoS attacks to web services.

A layer 2 or layer 3 filter in front of the ALG will do this wonderfully 
with out the ALG having to know about allowed and / or banned IP 
address(es) and / or range(s).

> I think if iptables has also the ability to deep packet inspect xml 
> messages it then demonstrates that there is still an importance for 
> NAC based firewalls.

You can deep inspect to a point, but not as deep as you may be thinking.

> All pointers to documentation and your comments are welcome.

Reply (on or off list) if you'd like to continue the discussion of the 
business logic.




Grant. . . .

Re: Query: Can Netfilter inspect xml soap traffic

[william fitzgerald]

Thank you Grant for your speedy reply.

My argument is that it appears to me that (secure) Enterprise Web 
Service applications, particularly those involving access control, are 
typically focused at the application-domain only, rather than taking a 
more holistic approach to also include the underlying infrastructure 
(for example, firewalls). As a result, infrastructure configurations may 
unintentionally hinder and prohibit the normal operation of the Web Service.

Thus, the ideal firewall configuration is one that is aligned with the 
application supported by the system, that is, it permits valid 
application traffic, and, preferably, no more and no less.

What I presume is that Web Service developers assume the underlying 
infrastructure is magically available. Also there seems to be a tendency 
to tunnel (for example SOAP) over http or https.

 From this point of view, Semantic Web developers may form the opinion 
that firewalls are redundant as they typically have ports 80 and 443 
accessible (and forward traffic to specialized user-space programs for 
further packet processing). Maybe they are correct! Have you any comments?

In my opinion, deploying a network level firewall  (such as Linux 
Netfilter) provisioned for Enterprise Web Services is not simply about 
opening port 80 on the server for all traffic; one may wish to deny 
certain nodes (IP addresses, etc.), only accept HTTP traffic from some 
nodes, require other nodes to use HTTPS and also deal with HTTP traffic 
that is tunneled through proxies available on other ports.

While low level infrastructure such as firewalls may not solve all 
security issues ( as good as an application based XML firewall would) in 
regard to Web Service applications, I believe they have a role to play 
in applying the belt-and-braces approach to security best practices.

What I am really looking for is some concrete documents, publications, 
administrator experience that helps support my gut feelings of the role 
of Network Access Controls (NAC's) within an enterprise SOA environment.

If anyone feels I should resubmit this email under another heading for 
example:
"Is Netfilter obsolete in an Enterprise Web Service Environment" please 
let me know.

Looking forward to comments on this, albeit tomorrow before I can reply 
again.

regards,
Will.


Grant Taylor wrote:
> On 03/25/08 10:01, william fitzgerald wrote:
>> Can Netfilter/iptables inspect xml/soap messages as xml based 
>> firewalls do?
> 
> Is NetFilter / IPTables capable of inspecting layer 7 traffic, yes. 
> However it will probably be much more difficult to do than you might 
> think.  I would expect it to be much like trying to write a regular 
> expression in assembly verses trying to do it in Perl.  It can be done, 
> but...
> 
> One thing to keep in mind is that the Layer-7 module only looks at some 
> of the packet.  Here is a quote from the website:
> 
> "... l7-filter only looks at the first 10 packets or 2kB of each 
> connection, whichever is smaller. ..."
> 
>> Does the Layer-7 module have enough "smarts" to inspect web service 
>> messages.
> 
> I don't think the "smarts" you are referring to are (or should be) in 
> the Layer-7 module.  Keep in mind that the Layer-7 module is a match 
> extension as in does this packet have this data at layer 7.  The logic 
> behind what to do with what match(es) and how to chain them together to 
> make decisions there on (IMHO) should *NOT* be in the Layer-7 module, 
> but rather in how you build your rules.
> 
> You will need to take in to account all the variances in the traffic 
> that could happen.  When speaking SMTP, I can either HELO or EHLO with 
> either a name or an IP.  Thus with in the first few packets you already 
> have four different possibilities *IF* I play by the rules.  Your 
> pattern will have to be very flexible.
> 
>> I am asking in regard to the role of Network Access Control firewalls 
>> such as iptables within a dedicated enterprise web service SOA 
>> environment.
> 
> You may have better luck taking the packet and passing it to user space 
> and writing an application layer gateway (a.k.a. ALG) and having the ALG 
> do the business logic of the filtering for you.
> 
>> I have seen some posts that suggest that firewalls are now obsolete, 
>> particularly NACs, in regard to web services (everything is over http 
>> hence less effect restrictions).
> 
> Firewalls are GREAT at filtering on layer 2 or layer 3 (depending what 
> they are designed for.  If you want to filter on a higher layer, you 
> need to use something that is designed to filter on that layer.
> 
>> However, my opinion is that its not as simple as opening ports 80 and 
>> 443 to tunnel SOAP messages.
> 
> You don't "just open ports".  You "open ports and send them in to the 
> next layer of security".  Layer 3 firewall in front of your ALG and then 
> let the ALG deal with what it knows about with out worrying about other 
> nasty things.
> 
>> For example, I may want to restrict IP ranges, maybe I have some 
>> business partners and I only want them accessing the web service. Or 
>> maybe I need to control DoS attacks to web services.
> 
> A layer 2 or layer 3 filter in front of the ALG will do this wonderfully 
> with out the ALG having to know about allowed and / or banned IP 
> address(es) and / or range(s).
> 
>> I think if iptables has also the ability to deep packet inspect xml 
>> messages it then demonstrates that there is still an importance for 
>> NAC based firewalls.
> 
> You can deep inspect to a point, but not as deep as you may be thinking.
> 
>> All pointers to documentation and your comments are welcome.
> 
> Reply (on or off list) if you'd like to continue the discussion of the 
> business logic.
> 
> 
> 
> 
> Grant. . . .
> -- 
> To unsubscribe from this list: send the line "unsubscribe netfilter" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> 

-- 
William M. Fitzgerald,
PhD Student,
Telecommunications Software & Systems Group,
ArcLabs Research and Innovation Centre,
Waterford Institute of Technology,
WIT West Campus,
Carriganore,
Waterford.
Office Ph: +353 51 302937
Mobile Ph: +353 87 9527083
Web: www.williamfitzgerald.org
      www.linkedin.com/in/williamfitzgerald
      www.ryze.com/go/wfitzgerald

Re: Query: Can Netfilter inspect xml soap traffic

[Grant Taylor]

On 03/25/08 12:04, william fitzgerald wrote:
> Thank you Grant for your speedy reply.

*nod*

> My argument is that it appears to me that (secure) Enterprise Web 
> Service applications, particularly those involving access control, are 
> typically focused at the application-domain only, rather than taking a 
> more holistic approach to also include the underlying infrastructure 
> (for example, firewalls). As a result, infrastructure configurations may 
> unintentionally hinder and prohibit the normal operation of the Web 
> Service.

Possibly...

> Thus, the ideal firewall configuration is one that is aligned with the 
> application supported by the system, that is, it permits valid 
> application traffic, and, preferably, no more and no less.

*nod*

> What I presume is that Web Service developers assume the underlying 
> infrastructure is magically available. Also there seems to be a tendency 
> to tunnel (for example SOAP) over http or https.

Agreed...

>  From this point of view, Semantic Web developers may form the opinion 
> that firewalls are redundant as they typically have ports 80 and 443 
> accessible (and forward traffic to specialized user-space programs for 
> further packet processing). Maybe they are correct! Have you any comments?

I think people forget that firewalls are filtering 65,533 other ports 
for TCP as well as other protocols too.  The firewalls filter out other 
cruft (if you will) so that developers are free to concentrate on what 
they know, the web app, with out worrying about the rest of the cruft.

I also think a firewall should go so far as to filter out all traffic to 
the web server ports that is not valid, i.e. various types of attacks 
and things that are in an invalid date.  These are the types of things 
that firewalls are *VERY GOOD* at doing that are much harder to do on a 
higher layer.  I would much rather let an IPTables firewall or a Cisco 
PIX (or what ever you would like) filter out invalid TCP traffic 
*BEFORE* it gets to the web server.  This will (should) at least 
(significantly) reduce the exposure of the web server to malicious 
packets that may other wise hinder normal operation.

> In my opinion, deploying a network level firewall  (such as Linux 
> Netfilter) provisioned for Enterprise Web Services is not simply about 
> opening port 80 on the server for all traffic; one may wish to deny 
> certain nodes (IP addresses, etc.), only accept HTTP traffic from some 
> nodes, require other nodes to use HTTPS and also deal with HTTP traffic 
> that is tunneled through proxies available on other ports.

I think you need to look at things in layers (not related to OSI).
  - Have the first layer do your basic allow / deny based on source IP.
  - Have your next layer do basic protocol validation as well as SPI.
  - Have your next layer do some filtering of content and take action as 
necessary.
     - This may be a form of proxy or front end web server that will 
pass traffic on to the back end or redirect people to different URLs.
  - Have the back end do a more thorough analysis and subsequent 
processing of traffic.

> While low level infrastructure such as firewalls may not solve all 
> security issues ( as good as an application based XML firewall would) in 
> regard to Web Service applications, I believe they have a role to play 
> in applying the belt-and-braces approach to security best practices.

If you directly pass *ALL* TCP port 80 traffic in to an XML firewall, 
you are exposing the firewall to a lot more than it would other wise be 
exposed to.  Most ALGs also take more processing overhead to do their 
filtering.  Thus having the ALG do the filtering for all traffic will 
incur more processing over head than having the firewall filter some of 
the traffic thus reducing the amount of traffic that the ALG has to 
process thus reducing the processing resources consumed by the ALG.

> What I am really looking for is some concrete documents, publications, 
> administrator experience that helps support my gut feelings of the role 
> of Network Access Controls (NAC's) within an enterprise SOA environment.

Good luck finding that.

> If anyone feels I should resubmit this email under another heading for 
> example:
> "Is Netfilter obsolete in an Enterprise Web Service Environment" please 
> let me know.

I don't think you should re-word the subject as such.  Your question(s) 
are more along the lines of whether or not a simple layer 2 / layer 3 
firewall still has any place in the current B2B market, not whether or 
not NetFilter / IPTables is obsolete or not.  Focus on the real 
question(s) and problem(s).

> Looking forward to comments on this, albeit tomorrow before I can reply 
> again.

*nod*



Grant. . . .

Re: Query: Can Netfilter inspect xml soap traffic

[Grant Taylor]

On 03/25/08 12:04, william fitzgerald wrote:
> Thus, the ideal firewall configuration is one that is aligned with the 
> application supported by the system, that is, it permits valid 
> application traffic, and, preferably, no more and no less.

Not directly related to your question(s), but still appropriate.

I would like to see developers write their applications with 
documentation (be it auto generated or not) that indicates what type of 
traffic (and parameters there on) they expect to see and need to 
function correctly.  I'd like to then take said documentation and use it 
to build rules for a simple ALG that will pass any valid requests in to 
the back end application while correctly handling erroneous traffic.  I 
think said ALGs could easily function as a proxy with some simple rules 
as to what is and is not allowed to pass through the ALG.

The next step is to educate the ALG about traffic flow from resource to 
resource (read:  page to page) and define how to handle improper traffic 
flow.  If someone tries to jump further in, should we go to an error 
page, or should we send them back to the start page?

I think these types of ALGs would significantly reduce the security 
problems with these types of applications.  Or at least if there was an 
SQL injection vulnerability in a given back end, it could be filtered by 
an ALG by simply checking for valid characters in a particular object 
property.  The ALG could either scrub (remove) the object property or it 
could fall back to an errant condition and redirect elsewhere, or it 
could conditionally do both depending on the previous history of the 
client.  Say for example if you or I accidentally enter an inappropriate 
character in a string and get redirected back to the form to correct the 
error verses an SQL injection script trying different things against the 
page.

These are the types of things that ALGs can do that is very difficult to 
implement in the back end code.



Grant. . . .

Re: Query: Can Netfilter inspect xml soap traffic

[Grant Taylor]

On 03/25/08 12:33, Taylor, Grant wrote:
> I would like to see developers write their applications with 
> documentation (be it auto generated or not) that indicates what type of 
> traffic (and parameters there on) they expect to see and need to 
> function correctly.  I'd like to then take said documentation and use it 
> to build rules for a simple ALG that will pass any valid requests in to 
> the back end application while correctly handling erroneous traffic.  I 
> think said ALGs could easily function as a proxy with some simple rules 
> as to what is and is not allowed to pass through the ALG.

Note:  I don't think that the rules for the ALG should be auto generated 
on demand from the original code or class as this will be a performance 
hit for systems.  These rules need to be defined in a batch operation. 
Now that batch operation could load the back end class and call a method 
that will return what it is expecting to dynamically build the rules 
once a night or when ever things are updated.



Grant. . . .

Re: Query: Can Netfilter inspect xml soap traffic

[Benny Amorsen]

william fitzgerald <wfitzgerald@tssg.org> writes:

> "Is Netfilter obsolete in an Enterprise Web Service Environment"
> please let me know.

The discussion about whether packet filters really deserve the word
"firewall" applied to them goes back at least as far as mid 90's. You
can probably find flame wars with Gauntlet and Raptor Eagle/Symantec
Enterprise Firewall on one side and Firewall-1 and PIX on the other.

Anyway, with the Level-7 match or Deep Packet Inspection or whichever
buzz words you prefer, packet filters are closer in capabilities than
ever before. At the same time application level proxies are faster
than ever before. It's hard to pick a winner.


/Benny

Re: Query: Can Netfilter inspect xml soap traffic

[Grant Taylor]

On 03/25/08 14:56, Benny Amorsen wrote:
> Anyway, with the Level-7 match or Deep Packet Inspection or whichever 
> buzz words you prefer, packet filters are closer in capabilities than 
> ever before. At the same time application level proxies are faster 
> than ever before. It's hard to pick a winner.

Very good point.

I suppose one thing to think about is who is going to maintain what. 
Developers would probably be able to maintain (add / change / delete 
rules) an ALG better where as network administration staff would 
probably be able to maintain a hardware firewall better.  Of course, why 
not use some of each.  Use the hardware firewall for the lower end 
simpler aspects of it while using the ALG for the higher end more 
specific aspects.  Let the hardware ASICs do what they do best while 
letting the ALG do what it does best.



Grant. . . .

Re: Query: Can Netfilter inspect xml soap traffic

[william fitzgerald]

Thanks Benny and Grant,

I whole heartily agree that the term "firewall" is overloaded.

We also now have the term UTM's (Unified Threat Management appliances) 
used to drift away from the "firewall" term but is essentially the same 
thing (access control via firewall, ids, av etc).

A lot of focus in relation to web services is on xml firewalls (or other 
relevant ALG's). There seems to be no role for network-level firewalls 
like Netfilter (that can also filter to a degree at layer 7).

I still stand over the idea that:

"deploying a network level firewall provisioned for Enterprise Web 
Services is not simply about opening port 80 on the server for all 
traffic; one may wish to deny certain nodes (IP addresses, etc.), only 
accept HTTP traffic from some nodes, require other nodes to use HTTPS 
and also deal with HTTP traffic that is tunneled through proxies 
available on other ports."

But I have found no examples in literature. The above is something that 
an XML firewall cannot do.

Is there anyone out there that can give a typical enterprise web service 
scenario/architecture that justifies network level firewall protection?

What is actual practice? Surely is not just a simple firewall rule:

iptables -A FORWARD -p tcp --dport http -j ACCEPT

Surely its more complicated than what the application developers suggest 
is required.

For example: only permit a business partners subnet to and from a 
specific web service.

iptables -A FORWARD -i eth0 -s 193.1.1.0/0 -d  -p tcp --dport http -j ACCEPT
iptables -A FORWARD -o eth1 -s 192.168.1.1 -d 192.168.1.1 -p tcp --sport 
http -j ACCEPT

an other example might be to limit dos attacks at a network level to an 
enterprise web service:
iptables -A FORWARD -i eth0  -d 192.168.1.1-p tcp -dport http --syn -m 
limit --limit 5/s -j ACCEPT


any other examples of protecting web services at a network level are 
welcome?

I realize the firewalls are prone to "tunnel" problems over 80 and 443, 
in that how can we decipher what is genuine http traffic, soap, skype 
and so forth. DPI helps to an extent.

What happens if you are running both a standard web server and a web 
service that both use port 80? Would it be normal practice to assign a 
different port for example 8080 to the web service.

Surely Web Services don't have to operate over 80 and 443!

If anyone knows of any documentation that specifically includes the 
importance of firewalls (in particular, network firewalls) in an SOA 
please let me know.

In terms of compliance standards like HIPPA, PCI DSS and so forth a 
firewall is mandatory even though some people are calling for an end to 
firewalls using the term "de-perimeterization".

regards,
Will.


Grant Taylor wrote:
> On 03/25/08 14:56, Benny Amorsen wrote:
>> Anyway, with the Level-7 match or Deep Packet Inspection or whichever 
>> buzz words you prefer, packet filters are closer in capabilities than 
>> ever before. At the same time application level proxies are faster 
>> than ever before. It's hard to pick a winner.
> 
> Very good point.
> 
> I suppose one thing to think about is who is going to maintain what. 
> Developers would probably be able to maintain (add / change / delete 
> rules) an ALG better where as network administration staff would 
> probably be able to maintain a hardware firewall better.  Of course, why 
> not use some of each.  Use the hardware firewall for the lower end 
> simpler aspects of it while using the ALG for the higher end more 
> specific aspects.  Let the hardware ASICs do what they do best while 
> letting the ALG do what it does best.
> 
> 
> 
> Grant. . . .
> -- 
> To unsubscribe from this list: send the line "unsubscribe netfilter" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> 

-- 
William M. Fitzgerald,
PhD Student,
Telecommunications Software & Systems Group,
ArcLabs Research and Innovation Centre,
Waterford Institute of Technology,
WIT West Campus,
Carriganore,
Waterford.
Office Ph: +353 51 302937
Mobile Ph: +353 87 9527083
Web: www.williamfitzgerald.org
      www.linkedin.com/in/williamfitzgerald
      www.ryze.com/go/wfitzgerald