OpenVPN throttling problem

OpenVPN throttling problem

[J Webster]

I am having trouble with an OpenVPN service on my server throttling the 
connection for video streaming when connecting via udp and/or tcp.I do not 
have the same problem when clients connect to the same server box and use 
the proxy server to stream video so there is definitely an issue either with 
the OpenVPN settings or possibly he routing on the server.
I was advised by the OpenVPN group to change to udp and play about with the 
tun mtu settings as it might be an encryption problem but this is not 
helping and we are running out of ideas of things to try.
I'm really not sure what else to troubleshoot to find out why the connection 
is being throttled so much when connecting via VPN.

The actual server is in a data centre with 100Mbps so there's no restriction
on that end apart from network traffic.

So, for example the client accesses the proxy server and types in
www.googlevideos.com and plays the video with the proxy as an in between
server.
For the VPN, the client accesses the VPN server and types in
www.googlevideos.com and plays the video with the VPN as an in between
relay - it's not VPN in the strict sense of just gaining access to a private
network, it's more of a public server with security access restrictions for
geo IP location.

On going to speedtest.net I get this when connected to the VPN:
ping 289ms
Down 0.58Mbps
Up: 0.84 Mbps

On connecting directly to the proxy server on the same server box I get:
ping 414ms
Down 2.54Mbps
Up: 0.22 Mbps

That is a lot of throttling for an encryption though to lose a whole 2Mbps
andvideo can't be played very well at that speed.

This is my iptables script (the udp VPN server runs on xx.xx9):
# Generated by iptables-save v1.3.5 on Sat Aug  7 15:55:43 2010
*filter
:INPUT DROP [13:2248]
:FORWARD DROP [0:0]
:OUTPUT ACCEPT [5:260]
:RH-Firewall-1-INPUT - [0:0]
-A INPUT -i lo -j ACCEPT
-A INPUT -m state --state RELATED,ESTABLISHED -j ACCEPT
-A INPUT -i eth0 -p tcp -m tcp --dport 1057 -m state --state NEW -m
recent --set --name SSH --rsource
-A INPUT -i eth0 -p tcp -m tcp --dport 1057 -m state --state NEW -m
recent --update --seconds 60 --hitcount 2 --rttl --name SSH --rsource -j
DROP
-A INPUT -d 88.xxx.xxx.xx9 -p tcp -m tcp --dport 1057 -m state --state
NEW -j ACCEPT
-A INPUT -d 88.xxx.xxx.xx9 -p tcp -m tcp --dport 5555 -m state --state
NEW -j ACCEPT
-A INPUT -d 88.xxx.xxx.xx9 -p tcp -m tcp --dport 1194 -m state --state
NEW -j ACCEPT
-A INPUT -d 88.xxx.xxx.xx9 -p udp -m udp --dport 1194 -m state --state
NEW -j ACCEPT
-A INPUT -i tun+ -j ACCEPT
-A INPUT -i tap+ -j ACCEPT
-A INPUT -p udp -m udp --dport 53 -m state --state NEW -j ACCEPT
-A INPUT -p tcp -m tcp --dport 53 -m state --state NEW -j ACCEPT
-A INPUT -p udp -m udp --dport 123 -m state --state NEW -j ACCEPT
-A INPUT -p tcp -m tcp --dport 8002 -m state --state NEW -j ACCEPT
-A INPUT -p tcp -m tcp --dport 9001 -m state --state NEW -j ACCEPT
-A INPUT -p tcp -m tcp --dport 80 -m state --state NEW -j ACCEPT
-A INPUT -d 88.xxx.xxx.xx8 -p tcp -m state --state NEW -m tcp --dport
8080 -j ACCEPT
-A INPUT -d 88.xxx.xxx.xx8 -p tcp -m state --state NEW -m tcp --dport
1935 -j ACCEPT
-A INPUT -d 88.xxx.xxx.xx8 -p tcp -m state --state NEW -m tcp --dport 80 -j
ACCEPT
-A INPUT -d 88.xxx.xxx.xx8 -p tcp -m state --state NEW -m tcp --dport 443 -j
ACCEPT
-A INPUT -d 88.xxx.xxx.xx9 -p tcp -m state --state NEW -m tcp --dport 443 -j
ACCEPT
-A INPUT -p icmp -m limit --limit 1/sec --limit-burst 1 -j ACCEPT
-A INPUT -d 88.xxx.xxx.xx8 -p icmp -m icmp --icmp-type 8 -m state --state
NEW,RELATED,ESTABLISHED -j ACCEPT
-A FORWARD -i eth0 -o tun+ -m state --state RELATED,ESTABLISHED -j ACCEPT
-A FORWARD -i tun+ -j ACCEPT
-A FORWARD -i tap+ -j ACCEPT
-A OUTPUT -s 88.xxx.xxx.xx9 -p tcp -m tcp --dport 1194 -m state --state
NEW -j ACCEPT
-A OUTPUT -s 88.xxx.xxx.xx9 -p udp -m udp --dport 1194 -m state --state
NEW -j ACCEPT
-A OUTPUT -s 88.xxx.xxx.xx9 -p tcp -m tcp --dport 443 -m state --state
NEW -j ACCEPT
-A OUTPUT -s 88.xxx.xxx.xx8 -p icmp -m icmp --icmp-type 0 -m state --state
RELATED,ESTABLISHED -j ACCEPT
-A OUTPUT -o lo -j ACCEPT
-A OUTPUT -m state --state NEW,RELATED,ESTABLISHED -j ACCEPT
COMMIT
# Completed on Sat Aug  7 15:55:43 2010
# Generated by iptables-save v1.3.5 on Sat Aug  7 15:55:43 2010
*nat
:PREROUTING ACCEPT [13:7569]
:POSTROUTING ACCEPT [8:3135]
:OUTPUT ACCEPT [8:3135]
-A PREROUTING -d 88.xxx.xxx.xx9 -p tcp -m tcp --dport 443 -j
DNAT --to-destination 88.xxx.xxx.xx9:1194
-A POSTROUTING -s 172.16.0.0/255.255.255.0 -o eth0 -j MASQUERADE
-A POSTROUTING -s 10.8.0.0/255.255.255.0 -o eth0 -j SNAT --to-source
88.xxx.xxx.xx9
COMMIT
# Completed on Sat Aug  7 15:55:43 2010

These are my VPN server and client settings:
server.conf:
local 88.xxx.xxx.xxx
port 1194
proto udp
dev tun1
crl-verify /etc/openvpn/crl.pem
client-config-dir /etc/openvpn/ccd
ca ca.crt
cert server.crt
key server.key
dh dh1024.pem
server 10.8.0.0 255.255.255.0
push "redirect-gateway"
push "dhcp-option DNS 213.171.192.249"
push "dhcp-option DNS 213.171.192.245"
ifconfig-pool-persist ipp.txt
keepalive 10 120
comp-lzo
user nobody
group users
persist-key
persist-tun
status openvpn-status2.log
verb 0
log /var/log/openvpn2.log
tun-mtu 1500
;fragment 1300
;mssfix
;sndbuf 204800
;rcvbuf 204800

client:
client
dev tun1
proto udp
remote 88.xxx.xxx.xxx 1194
resolv-retry infinite
nobind
persist-key
persist-tun
ca ca.crt
cert adminuser.crt
key adminuser.key
ns-cert-type server
comp-lzo
verb 1
tun-mtu 1500
;fragment 1300
;mssfix
;sndbuf 204800
;rcvbuf 204800

 

Re: OpenVPN throttling problem

[Thomas Jacob]

On Mon, 2010-09-06 at 21:23 -0400, J Webster wrote:

> So, for example the client accesses the proxy server and types in
> www.googlevideos.com and plays the video with the proxy as an in between
> server.
> For the VPN, the client accesses the VPN server and types in
> www.googlevideos.com and plays the video with the VPN as an in between
> relay - it's not VPN in the strict sense of just gaining access to a private
> network, it's more of a public server with security access restrictions for
> geo IP location.
> 
> On going to speedtest.net I get this when connected to the VPN:
> ping 289ms
> Down 0.58Mbps
> Up: 0.84 Mbps
> 
> On connecting directly to the proxy server on the same server box I get:
> ping 414ms
> Down 2.54Mbps
> Up: 0.22 Mbps

Judging from the latter speed result you are probably connecting
through some sort of consumer connection, where there is a
pretty high likelihood that your path MTU will not be 1500, so
I'd say the OpenVPN guys are probably right when they think
it's an MTU problem. Your iptables rules certainly don't
look if they could be causing the throttling.

Your best option would probably be to fiddle around with
the MTU related settings of OpenVPN. Try searching
the OpenVPN-users mailing list, having MTU related problems
is apparently quite common, that's why there are
so many different options for that area. 
Also maybe look into the  --clamp-mss-to-pmtu thing.

Re: OpenVPN throttling problem

[J Webster]

If the path MTU were not 1500 then why would the proxy server work without 
video stuttering issues but the VPN have stuttering?
I would have thought most broadband connections were not limited in that 
way?
I did try some MTU setting before of 1400, 1460, 1300 and the difference was 
minimal.
Not sure what else to try or how to troubleshoot. I suppose I could follow 
the traffic but not sure if it would help resolve the throttling issue?

--------------------------------------------------
From: "Thomas Jacob" <jacob@internet24.de>
Sent: Tuesday, September 07, 2010 7:09 AM
To: "J Webster" <webster_jack@hotmail.com>
Cc: <netfilter@vger.kernel.org>
Subject: Re: OpenVPN throttling problem

> On Mon, 2010-09-06 at 21:23 -0400, J Webster wrote:
>
>> So, for example the client accesses the proxy server and types in
>> www.googlevideos.com and plays the video with the proxy as an in between
>> server.
>> For the VPN, the client accesses the VPN server and types in
>> www.googlevideos.com and plays the video with the VPN as an in between
>> relay - it's not VPN in the strict sense of just gaining access to a 
>> private
>> network, it's more of a public server with security access restrictions 
>> for
>> geo IP location.
>>
>> On going to speedtest.net I get this when connected to the VPN:
>> ping 289ms
>> Down 0.58Mbps
>> Up: 0.84 Mbps
>>
>> On connecting directly to the proxy server on the same server box I get:
>> ping 414ms
>> Down 2.54Mbps
>> Up: 0.22 Mbps
>
> Judging from the latter speed result you are probably connecting
> through some sort of consumer connection, where there is a
> pretty high likelihood that your path MTU will not be 1500, so
> I'd say the OpenVPN guys are probably right when they think
> it's an MTU problem. Your iptables rules certainly don't
> look if they could be causing the throttling.
>
> Your best option would probably be to fiddle around with
> the MTU related settings of OpenVPN. Try searching
> the OpenVPN-users mailing list, having MTU related problems
> is apparently quite common, that's why there are
> so many different options for that area.
> Also maybe look into the  --clamp-mss-to-pmtu thing.
>
> --
> To unsubscribe from this list: send the line "unsubscribe netfilter" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> 

Re: OpenVPN throttling problem

[Thomas Jacob]

On Tue, 2010-09-07 at 10:25 -0400, J Webster wrote:
> If the path MTU were not 1500 then why would the proxy server work without 
> video stuttering issues but the VPN have stuttering?

Because OpenVPN seems to prevent the normal path MTU algorithms
from working in some instances, so the dynamic MSS/MTU
calculations cannot happen. Anyway, a proxy server
doesn't forward TCP packets in the way OpenVPN does,
it opens a new TCP connection and just relays the Web data stream,
so it's really quite a different thing.

> I would have thought most broadband connections were not limited in that 
> way?

PPPoE DSL is, for instance.

> I did try some MTU setting before of 1400, 1460, 1300 and the difference was 
> minimal.

It's not enough to just configure that in OpenVPN, all the other
components (client NIC, gateway NICs, server NIC, intermediate router
NICs) also have their own MTU (hence the path MTU discovering
solution).

> Not sure what else to try or how to troubleshoot. I suppose I could follow 
> the traffic but not sure if it would help resolve the throttling issue?

Have you tried MSS clamping yet?

http://lartc.org/howto/lartc.cookbook.mtu-mss.html

Re: OpenVPN throttling problem

[J Webster]

Would the clamping only be tcp specific?
Could I add the same rule for the udp VPN service?
iptables -A FORWARD -p tcp --tcp-flags SYN,RST SYN -j 
CPMSS  --clamp-mss-to-pmtu

--------------------------------------------------
From: "Thomas Jacob" <jacob@internet24.de>
Sent: Tuesday, September 07, 2010 11:05 AM
To: "J Webster" <webster_jack@hotmail.com>
Cc: <netfilter@vger.kernel.org>
Subject: Re: OpenVPN throttling problem

> On Tue, 2010-09-07 at 10:25 -0400, J Webster wrote:
>> If the path MTU were not 1500 then why would the proxy server work 
>> without
>> video stuttering issues but the VPN have stuttering?
>
> Because OpenVPN seems to prevent the normal path MTU algorithms
> from working in some instances, so the dynamic MSS/MTU
> calculations cannot happen. Anyway, a proxy server
> doesn't forward TCP packets in the way OpenVPN does,
> it opens a new TCP connection and just relays the Web data stream,
> so it's really quite a different thing.
>
>> I would have thought most broadband connections were not limited in that
>> way?
>
> PPPoE DSL is, for instance.
>
>> I did try some MTU setting before of 1400, 1460, 1300 and the difference 
>> was
>> minimal.
>
> It's not enough to just configure that in OpenVPN, all the other
> components (client NIC, gateway NICs, server NIC, intermediate router
> NICs) also have their own MTU (hence the path MTU discovering
> solution).
>
>> Not sure what else to try or how to troubleshoot. I suppose I could 
>> follow
>> the traffic but not sure if it would help resolve the throttling issue?
>
> Have you tried MSS clamping yet?
>
> http://lartc.org/howto/lartc.cookbook.mtu-mss.html
>
> 

Re: OpenVPN throttling problem

[Thomas Jacob]

On Tue, 2010-09-07 at 11:12 -0400, J Webster wrote:
> Would the clamping only be tcp specific?

Correct, MSS (maximum segment size) is a TCP specific
feature.

> Could I add the same rule for the udp VPN service?
> iptables -A FORWARD -p tcp --tcp-flags SYN,RST SYN -j 
> CPMSS  --clamp-mss-to-pmtu

Nope, see above. But for UDP this is not often
a problem, as most standard protocols that use
UDP have smaller packet sizes,
unless of course your video streaming is done via UDP ;)

> --------------------------------------------------
> From: "Thomas Jacob" <jacob@internet24.de>
> Sent: Tuesday, September 07, 2010 11:05 AM
> To: "J Webster" <webster_jack@hotmail.com>
> Cc: <netfilter@vger.kernel.org>
> Subject: Re: OpenVPN throttling problem
> 
> > On Tue, 2010-09-07 at 10:25 -0400, J Webster wrote:
> >> If the path MTU were not 1500 then why would the proxy server work 
> >> without
> >> video stuttering issues but the VPN have stuttering?
> >
> > Because OpenVPN seems to prevent the normal path MTU algorithms
> > from working in some instances, so the dynamic MSS/MTU
> > calculations cannot happen. Anyway, a proxy server
> > doesn't forward TCP packets in the way OpenVPN does,
> > it opens a new TCP connection and just relays the Web data stream,
> > so it's really quite a different thing.
> >
> >> I would have thought most broadband connections were not limited in that
> >> way?
> >
> > PPPoE DSL is, for instance.
> >
> >> I did try some MTU setting before of 1400, 1460, 1300 and the difference 
> >> was
> >> minimal.
> >
> > It's not enough to just configure that in OpenVPN, all the other
> > components (client NIC, gateway NICs, server NIC, intermediate router
> > NICs) also have their own MTU (hence the path MTU discovering
> > solution).
> >
> >> Not sure what else to try or how to troubleshoot. I suppose I could 
> >> follow
> >> the traffic but not sure if it would help resolve the throttling issue?
> >
> > Have you tried MSS clamping yet?
> >
> > http://lartc.org/howto/lartc.cookbook.mtu-mss.html
> >
> > 

Re: OpenVPN throttling problem

[J Webster]

Well, I have openVPN running as a tcp and also as a udp service on the 
server but I am having problems with video stuttering on both connections.
The streaming is however probably done via tcp. For example, the client 
would access msn video or youtube from their browser, connection then goes 
through the VPN, then to the internet, then back to the VPN, then back to 
the client.
I can try the mss clamp for the tcp connection but it doesn;t solve the same 
bandwidth issue on the udp VPN connection I suppose.

--------------------------------------------------
From: "Thomas Jacob" <jacob@internet24.de>
Sent: Tuesday, September 07, 2010 11:20 AM
To: "J Webster" <webster_jack@hotmail.com>
Cc: <netfilter@vger.kernel.org>
Subject: Re: OpenVPN throttling problem

> On Tue, 2010-09-07 at 11:12 -0400, J Webster wrote:
>> Would the clamping only be tcp specific?
>
> Correct, MSS (maximum segment size) is a TCP specific
> feature.
>
>> Could I add the same rule for the udp VPN service?
>> iptables -A FORWARD -p tcp --tcp-flags SYN,RST SYN -j
>> CPMSS  --clamp-mss-to-pmtu
>
> Nope, see above. But for UDP this is not often
> a problem, as most standard protocols that use
> UDP have smaller packet sizes,
> unless of course your video streaming is done via UDP ;)
>
>> --------------------------------------------------
>> From: "Thomas Jacob" <jacob@internet24.de>
>> Sent: Tuesday, September 07, 2010 11:05 AM
>> To: "J Webster" <webster_jack@hotmail.com>
>> Cc: <netfilter@vger.kernel.org>
>> Subject: Re: OpenVPN throttling problem
>>
>> > On Tue, 2010-09-07 at 10:25 -0400, J Webster wrote:
>> >> If the path MTU were not 1500 then why would the proxy server work
>> >> without
>> >> video stuttering issues but the VPN have stuttering?
>> >
>> > Because OpenVPN seems to prevent the normal path MTU algorithms
>> > from working in some instances, so the dynamic MSS/MTU
>> > calculations cannot happen. Anyway, a proxy server
>> > doesn't forward TCP packets in the way OpenVPN does,
>> > it opens a new TCP connection and just relays the Web data stream,
>> > so it's really quite a different thing.
>> >
>> >> I would have thought most broadband connections were not limited in 
>> >> that
>> >> way?
>> >
>> > PPPoE DSL is, for instance.
>> >
>> >> I did try some MTU setting before of 1400, 1460, 1300 and the 
>> >> difference
>> >> was
>> >> minimal.
>> >
>> > It's not enough to just configure that in OpenVPN, all the other
>> > components (client NIC, gateway NICs, server NIC, intermediate router
>> > NICs) also have their own MTU (hence the path MTU discovering
>> > solution).
>> >
>> >> Not sure what else to try or how to troubleshoot. I suppose I could
>> >> follow
>> >> the traffic but not sure if it would help resolve the throttling 
>> >> issue?
>> >
>> > Have you tried MSS clamping yet?
>> >
>> > http://lartc.org/howto/lartc.cookbook.mtu-mss.html
>> >
>> >
>
>
> 

Re: OpenVPN throttling problem

[Thomas Jacob]

On Tue, 2010-09-07 at 11:25 -0400, J Webster wrote:
> Well, I have openVPN running as a tcp and also as a udp service on the 
> server but I am having problems with video stuttering on both connections.
> The streaming is however probably done via tcp. 

To clarify, the issue is TCP or UDP connections that
are sent thru the tunnel, not how the OpenVPN
tunnel connection itself. So unless you run UDP services
that transfer data (like NFS over UDP) via the tunnel,
you should have no significant problems with UDP.

BTW, using TCP for tunnelling is considered a
bad idea by some:

http://sites.inka.de/bigred/devel/tcp-tcp.html

> For example, the client 
> would access msn video or youtube from their browser, connection then goes 
> through the VPN, then to the internet, then back to the VPN, then back to 
> the client.

Well fooling geolocation this way only works until the companies in
question start blocking data centers as well as public proxy
services (=>
http://techcrunch.com/2009/05/06/control-freaks-hulu-now-blocks-anonymous-proxies-too/
) but that's not my problem :=)

> I can try the mss clamp for the tcp connection but it doesn;t solve the same 
> bandwidth issue on the udp VPN connection I suppose.

Re: OpenVPN throttling problem

[Payam Chychi]

Hey,

encryption is not easy, specially on a server. Dedicated 
routers/firewalls on a average show about 70-80% performance/throughput 
lost when IPsec/Encryption is used... so what makes you think that you 
should be getting the same speed result with/without encryption?

Also, when tunneling traffic, there is a lot of fragmentation happening 
(specially when you do not have the proper MTU value setup). You should 
note that on pkts that have the fragmented packet value set, the end 
router is the device responsible for placing all pkts back together 
which requires extreme cpu cycles (depending on the rate of traffic/pkts)

Put all of this together on a single server and performance issue is 
usually the least of your problems.

If you figure out your issue let us know, we can only speculate as to 
what the issue is at this point while you do the testing.

-Payam




J Webster wrote:
> I am having trouble with an OpenVPN service on my server throttling 
> the connection for video streaming when connecting via udp and/or 
> tcp.I do not have the same problem when clients connect to the same 
> server box and use the proxy server to stream video so there is 
> definitely an issue either with the OpenVPN settings or possibly he 
> routing on the server.
> I was advised by the OpenVPN group to change to udp and play about 
> with the tun mtu settings as it might be an encryption problem but 
> this is not helping and we are running out of ideas of things to try.
> I'm really not sure what else to troubleshoot to find out why the 
> connection is being throttled so much when connecting via VPN.
>
> The actual server is in a data centre with 100Mbps so there's no 
> restriction
> on that end apart from network traffic.
>
> So, for example the client accesses the proxy server and types in
> www.googlevideos.com and plays the video with the proxy as an in between
> server.
> For the VPN, the client accesses the VPN server and types in
> www.googlevideos.com and plays the video with the VPN as an in between
> relay - it's not VPN in the strict sense of just gaining access to a 
> private
> network, it's more of a public server with security access 
> restrictions for
> geo IP location.
>
> On going to speedtest.net I get this when connected to the VPN:
> ping 289ms
> Down 0.58Mbps
> Up: 0.84 Mbps
>
> On connecting directly to the proxy server on the same server box I get:
> ping 414ms
> Down 2.54Mbps
> Up: 0.22 Mbps
>
> That is a lot of throttling for an encryption though to lose a whole 
> 2Mbps
> andvideo can't be played very well at that speed.
>
> This is my iptables script (the udp VPN server runs on xx.xx9):
> # Generated by iptables-save v1.3.5 on Sat Aug  7 15:55:43 2010
> *filter
> :INPUT DROP [13:2248]
> :FORWARD DROP [0:0]
> :OUTPUT ACCEPT [5:260]
> :RH-Firewall-1-INPUT - [0:0]
> -A INPUT -i lo -j ACCEPT
> -A INPUT -m state --state RELATED,ESTABLISHED -j ACCEPT
> -A INPUT -i eth0 -p tcp -m tcp --dport 1057 -m state --state NEW -m
> recent --set --name SSH --rsource
> -A INPUT -i eth0 -p tcp -m tcp --dport 1057 -m state --state NEW -m
> recent --update --seconds 60 --hitcount 2 --rttl --name SSH --rsource -j
> DROP
> -A INPUT -d 88.xxx.xxx.xx9 -p tcp -m tcp --dport 1057 -m state --state
> NEW -j ACCEPT
> -A INPUT -d 88.xxx.xxx.xx9 -p tcp -m tcp --dport 5555 -m state --state
> NEW -j ACCEPT
> -A INPUT -d 88.xxx.xxx.xx9 -p tcp -m tcp --dport 1194 -m state --state
> NEW -j ACCEPT
> -A INPUT -d 88.xxx.xxx.xx9 -p udp -m udp --dport 1194 -m state --state
> NEW -j ACCEPT
> -A INPUT -i tun+ -j ACCEPT
> -A INPUT -i tap+ -j ACCEPT
> -A INPUT -p udp -m udp --dport 53 -m state --state NEW -j ACCEPT
> -A INPUT -p tcp -m tcp --dport 53 -m state --state NEW -j ACCEPT
> -A INPUT -p udp -m udp --dport 123 -m state --state NEW -j ACCEPT
> -A INPUT -p tcp -m tcp --dport 8002 -m state --state NEW -j ACCEPT
> -A INPUT -p tcp -m tcp --dport 9001 -m state --state NEW -j ACCEPT
> -A INPUT -p tcp -m tcp --dport 80 -m state --state NEW -j ACCEPT
> -A INPUT -d 88.xxx.xxx.xx8 -p tcp -m state --state NEW -m tcp --dport
> 8080 -j ACCEPT
> -A INPUT -d 88.xxx.xxx.xx8 -p tcp -m state --state NEW -m tcp --dport
> 1935 -j ACCEPT
> -A INPUT -d 88.xxx.xxx.xx8 -p tcp -m state --state NEW -m tcp --dport 
> 80 -j
> ACCEPT
> -A INPUT -d 88.xxx.xxx.xx8 -p tcp -m state --state NEW -m tcp --dport 
> 443 -j
> ACCEPT
> -A INPUT -d 88.xxx.xxx.xx9 -p tcp -m state --state NEW -m tcp --dport 
> 443 -j
> ACCEPT
> -A INPUT -p icmp -m limit --limit 1/sec --limit-burst 1 -j ACCEPT
> -A INPUT -d 88.xxx.xxx.xx8 -p icmp -m icmp --icmp-type 8 -m state --state
> NEW,RELATED,ESTABLISHED -j ACCEPT
> -A FORWARD -i eth0 -o tun+ -m state --state RELATED,ESTABLISHED -j ACCEPT
> -A FORWARD -i tun+ -j ACCEPT
> -A FORWARD -i tap+ -j ACCEPT
> -A OUTPUT -s 88.xxx.xxx.xx9 -p tcp -m tcp --dport 1194 -m state --state
> NEW -j ACCEPT
> -A OUTPUT -s 88.xxx.xxx.xx9 -p udp -m udp --dport 1194 -m state --state
> NEW -j ACCEPT
> -A OUTPUT -s 88.xxx.xxx.xx9 -p tcp -m tcp --dport 443 -m state --state
> NEW -j ACCEPT
> -A OUTPUT -s 88.xxx.xxx.xx8 -p icmp -m icmp --icmp-type 0 -m state 
> --state
> RELATED,ESTABLISHED -j ACCEPT
> -A OUTPUT -o lo -j ACCEPT
> -A OUTPUT -m state --state NEW,RELATED,ESTABLISHED -j ACCEPT
> COMMIT
> # Completed on Sat Aug  7 15:55:43 2010
> # Generated by iptables-save v1.3.5 on Sat Aug  7 15:55:43 2010
> *nat
> :PREROUTING ACCEPT [13:7569]
> :POSTROUTING ACCEPT [8:3135]
> :OUTPUT ACCEPT [8:3135]
> -A PREROUTING -d 88.xxx.xxx.xx9 -p tcp -m tcp --dport 443 -j
> DNAT --to-destination 88.xxx.xxx.xx9:1194
> -A POSTROUTING -s 172.16.0.0/255.255.255.0 -o eth0 -j MASQUERADE
> -A POSTROUTING -s 10.8.0.0/255.255.255.0 -o eth0 -j SNAT --to-source
> 88.xxx.xxx.xx9
> COMMIT
> # Completed on Sat Aug  7 15:55:43 2010
>
> These are my VPN server and client settings:
> server.conf:
> local 88.xxx.xxx.xxx
> port 1194
> proto udp
> dev tun1
> crl-verify /etc/openvpn/crl.pem
> client-config-dir /etc/openvpn/ccd
> ca ca.crt
> cert server.crt
> key server.key
> dh dh1024.pem
> server 10.8.0.0 255.255.255.0
> push "redirect-gateway"
> push "dhcp-option DNS 213.171.192.249"
> push "dhcp-option DNS 213.171.192.245"
> ifconfig-pool-persist ipp.txt
> keepalive 10 120
> comp-lzo
> user nobody
> group users
> persist-key
> persist-tun
> status openvpn-status2.log
> verb 0
> log /var/log/openvpn2.log
> tun-mtu 1500
> ;fragment 1300
> ;mssfix
> ;sndbuf 204800
> ;rcvbuf 204800
>
> client:
> client
> dev tun1
> proto udp
> remote 88.xxx.xxx.xxx 1194
> resolv-retry infinite
> nobind
> persist-key
> persist-tun
> ca ca.crt
> cert adminuser.crt
> key adminuser.key
> ns-cert-type server
> comp-lzo
> verb 1
> tun-mtu 1500
> ;fragment 1300
> ;mssfix
> ;sndbuf 204800
> ;rcvbuf 204800
>
>
>
> -- 
> To unsubscribe from this list: send the line "unsubscribe netfilter" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>

Re: OpenVPN throttling problem

[J Webster]

I have tried by adding these lines to the iptables script and restarted it:
-A POSTROUTING -o eth0 -p tcp -m tcp --tcp-flags SYN,RST SYN -j 
TCPMSS --set-mss 1460
-A POSTROUTING -o eth0 -p tcp -m tcp --tcp-flags SYN,RST SYN -j 
TCPMSS --clamp-mss-to-pmtu

However, the stuttering still occurs on the VPN server.
Some at the OpenVPN users list suggested that I could change the windows tcp 
stack?
"I was doing some testing with iperf, and found that changing the TCP
Window size can have a significant impact (~ 5-10x). Have you tried this?"

I think I have already done this but cannit see why that should make a 
massive difference. Not only that, but once you get into editing registry 
entried, it;s beyond the capabilities of most client users.

--------------------------------------------------
From: "Thomas Jacob" <jacob@internet24.de>
Sent: Tuesday, September 07, 2010 11:20 AM
To: "J Webster" <webster_jack@hotmail.com>
Cc: <netfilter@vger.kernel.org>
Subject: Re: OpenVPN throttling problem

> On Tue, 2010-09-07 at 11:12 -0400, J Webster wrote:
>> Would the clamping only be tcp specific?
>
> Correct, MSS (maximum segment size) is a TCP specific
> feature.
>
>> Could I add the same rule for the udp VPN service?
>> iptables -A FORWARD -p tcp --tcp-flags SYN,RST SYN -j
>> CPMSS  --clamp-mss-to-pmtu
>
> Nope, see above. But for UDP this is not often
> a problem, as most standard protocols that use
> UDP have smaller packet sizes,
> unless of course your video streaming is done via UDP ;)
>
>> --------------------------------------------------
>> From: "Thomas Jacob" <jacob@internet24.de>
>> Sent: Tuesday, September 07, 2010 11:05 AM
>> To: "J Webster" <webster_jack@hotmail.com>
>> Cc: <netfilter@vger.kernel.org>
>> Subject: Re: OpenVPN throttling problem
>>
>> > On Tue, 2010-09-07 at 10:25 -0400, J Webster wrote:
>> >> If the path MTU were not 1500 then why would the proxy server work
>> >> without
>> >> video stuttering issues but the VPN have stuttering?
>> >
>> > Because OpenVPN seems to prevent the normal path MTU algorithms
>> > from working in some instances, so the dynamic MSS/MTU
>> > calculations cannot happen. Anyway, a proxy server
>> > doesn't forward TCP packets in the way OpenVPN does,
>> > it opens a new TCP connection and just relays the Web data stream,
>> > so it's really quite a different thing.
>> >
>> >> I would have thought most broadband connections were not limited in 
>> >> that
>> >> way?
>> >
>> > PPPoE DSL is, for instance.
>> >
>> >> I did try some MTU setting before of 1400, 1460, 1300 and the 
>> >> difference
>> >> was
>> >> minimal.
>> >
>> > It's not enough to just configure that in OpenVPN, all the other
>> > components (client NIC, gateway NICs, server NIC, intermediate router
>> > NICs) also have their own MTU (hence the path MTU discovering
>> > solution).
>> >
>> >> Not sure what else to try or how to troubleshoot. I suppose I could
>> >> follow
>> >> the traffic but not sure if it would help resolve the throttling 
>> >> issue?
>> >
>> > Have you tried MSS clamping yet?
>> >
>> > http://lartc.org/howto/lartc.cookbook.mtu-mss.html
>> >
>> >
>
>
> 

limit badwidth not working

[J Webster]

I have a setup where uisers connect by VPN and are given IP addresses in the 
range 10.8.0.xxx
I would like to limit their bandwidth to 1.5Mbps per IP address. However, I 
don't want to limit the incoming connection.
For example, they access the VPN server by it's WAN IP 200.xx.xx.xx and are 
given a local IP of 10.0.8.x
User 1 goes to www.youtube.com and starts streaming videos, this should be 
limited to 1.5Mbps.
User 2 goes to www.youtube.com and starts downloading a video, this should 
also be limited to 1.5Mbps but the
server connection to youtube should have unlimited bandwidth to allow for 
multiple users, in this sinstance at least 3Mbps.

I tried the tc example below but am not sure whether I should apply the 
filter to the tun0 network 10.0.8x or to the entire iptables connections.

#!/bin/bash
#
#  tc uses the following units when passed as a parameter.
#  kbps: Kilobytes per second
#  mbps: Megabytes per second
#  kbit: Kilobits per second
#  mbit: Megabits per second
#  bps: Bytes per second
#       Amounts of data can be specified in:
#       kb or k: Kilobytes
#       mb or m: Megabytes
#       mbit: Megabits
#       kbit: Kilobits
#  To get the byte figure from bits, divide the number by 8 bit
#

#
# Name of the traffic control command.
TC=/sbin/tc

# The network interface we're planning on limiting bandwidth.
IF=eth0             # Interface

# Download limit (in mega bits)
DNLD=1mbit          # DOWNLOAD Limit

# Upload limit (in mega bits)
UPLD=1mbit          # UPLOAD Limit

# IP address of the machine we are controlling
IP=216.3.128.12     # Host IP

# Filter options for limiting the intended interface.
U32="$TC filter add dev $IF protocol ip parent 1:0 prio 1 u32"

start() {

# We'll use Hierarchical Token Bucket (HTB) to shape bandwidth.
# For detailed configuration options, please consult Linux man
# page.

    $TC qdisc add dev $IF root handle 1: htb default 30
    $TC class add dev $IF parent 1: classid 1:1 htb rate $DNLD
    $TC class add dev $IF parent 1: classid 1:2 htb rate $UPLD
    $U32 match ip dst $IP/32 flowid 1:1
    $U32 match ip src $IP/32 flowid 1:2

# The first line creates the root qdisc, and the next two lines
# create two child qdisc that are to be used to shape download
# and upload bandwidth.
#
# The 4th and 5th line creates the filter to match the interface.
# The 'dst' IP address is used to limit download speed, and the
# 'src' IP address is used to limit upload speed.

}

stop() {

# Stop the bandwidth shaping.
    $TC qdisc del dev $IF root

}

restart() {

# Self-explanatory.
    stop
    sleep 1
    start

}

show() {

# Display status of traffic control status.
    $TC -s qdisc ls dev $IF

}

case "$1" in

  start)

    echo -n "Starting bandwidth shaping: "
    start
    echo "done"
    ;;

  stop)

    echo -n "Stopping bandwidth shaping: "
    stop
    echo "done"
    ;;

  restart)

    echo -n "Restarting bandwidth shaping: "
    restart
    echo "done"
    ;;

  show)

    echo "Bandwidth shaping status for $IF:"
    show
    echo ""
    ;;

  *)

    pwd=$(pwd)
    echo "Usage: tc.bash {start|stop|restart|show}"
    ;;

esac

exit 0
 

Re: limit badwidth not working

[Andrew Beverley]

On Tue, 2010-12-28 at 12:12 +0100, J Webster wrote:
> I have a setup where uisers connect by VPN and are given IP addresses in the 
> range 10.8.0.xxx

I can't advise on the VPN aspects of this, but see below for some
general comments.

> I would like to limit their bandwidth to 1.5Mbps per IP address. However, I 
> don't want to limit the incoming connection.
> For example, they access the VPN server by it's WAN IP 200.xx.xx.xx and are 
> given a local IP of 10.0.8.x
> User 1 goes to www.youtube.com and starts streaming videos, this should be 
> limited to 1.5Mbps.
> User 2 goes to www.youtube.com and starts downloading a video, this should 
> also be limited to 1.5Mbps but the
> server connection to youtube should have unlimited bandwidth to allow for 
> multiple users, in this sinstance at least 3Mbps.

Please clarify - you state that you don't want to limit the incoming
connection, but then state that you want to limit the download limit per
IP address to 1.5Mbps. Do you mean that you don't want to limit the
overall inbound connection but want to limit per destination IP address?

> I tried the tc example below but am not sure whether I should apply the 
> filter to the tun0 network 10.0.8x or to the entire iptables connections.

Not sure I'm afraid.

<snip>

> 
> # The network interface we're planning on limiting bandwidth.
> IF=eth0             # Interface

Is eth0 your internet side interface or your local network side?

> # IP address of the machine we are controlling
> IP=216.3.128.12     # Host IP
> 
> # Filter options for limiting the intended interface.
> U32="$TC filter add dev $IF protocol ip parent 1:0 prio 1 u32"
> 

>     $TC qdisc add dev $IF root handle 1: htb default 30
>     $TC class add dev $IF parent 1: classid 1:1 htb rate $DNLD
>     $TC class add dev $IF parent 1: classid 1:2 htb rate $UPLD
>     $U32 match ip dst $IP/32 flowid 1:1
>     $U32 match ip src $IP/32 flowid 1:2

I *think* that you'll need a separate leaf class for each client on your
network. I think you'll also need an overall rate limit for the root
(which is kind of what you've already got above). If you want to avoid
rate limiting the overall interface, then don't set a default and only
filter by destination IP address.

I'm a bit confused about what you want to control (see comment above),
as you have references to upload and download limits. If you only want
to limit the download stream, then you can't do this by ingress on the
source interface. You'll have to either do it as egress on the outbound
interface, or use an IFB interface. That said, I don't know how a VPN
affects this and whether that makes a difference.

So, in summary:

- Use a root qdisc with an overall limit on the correct interface
- Add a leaf class for each client
- Filter into each leaf class based on IP address

Andy

Re: limit badwidth not working

[J Webster]

Do you know of any tutorials on this with examples? I've looked through the 
main tc tutorials and they are pretty hard to follow.
Re connections, my network is 100Mbps, I want to leave that as unlimited so 
their is no overall bucket level.
Users connect to the VPN and each of the IP addresses connected to the VPN 
should have a limit of 1.5Mbps.

--------------------------------------------------
From: "Andrew Beverley" <andy@andybev.com>
Sent: Saturday, January 01, 2011 5:08 PM
To: "J Webster" <webster_jack@hotmail.com>
Cc: <netfilter@vger.kernel.org>
Subject: Re: limit badwidth not working

> On Tue, 2010-12-28 at 12:12 +0100, J Webster wrote:
>> I have a setup where uisers connect by VPN and are given IP addresses in 
>> the
>> range 10.8.0.xxx
>
> I can't advise on the VPN aspects of this, but see below for some
> general comments.
>
>> I would like to limit their bandwidth to 1.5Mbps per IP address. However, 
>> I
>> don't want to limit the incoming connection.
>> For example, they access the VPN server by it's WAN IP 200.xx.xx.xx and 
>> are
>> given a local IP of 10.0.8.x
>> User 1 goes to www.youtube.com and starts streaming videos, this should 
>> be
>> limited to 1.5Mbps.
>> User 2 goes to www.youtube.com and starts downloading a video, this 
>> should
>> also be limited to 1.5Mbps but the
>> server connection to youtube should have unlimited bandwidth to allow for
>> multiple users, in this sinstance at least 3Mbps.
>
> Please clarify - you state that you don't want to limit the incoming
> connection, but then state that you want to limit the download limit per
> IP address to 1.5Mbps. Do you mean that you don't want to limit the
> overall inbound connection but want to limit per destination IP address?
>
>> I tried the tc example below but am not sure whether I should apply the
>> filter to the tun0 network 10.0.8x or to the entire iptables connections.
>
> Not sure I'm afraid.
>
> <snip>
>
>>
>> # The network interface we're planning on limiting bandwidth.
>> IF=eth0             # Interface
>
> Is eth0 your internet side interface or your local network side?
>
>> # IP address of the machine we are controlling
>> IP=216.3.128.12     # Host IP
>>
>> # Filter options for limiting the intended interface.
>> U32="$TC filter add dev $IF protocol ip parent 1:0 prio 1 u32"
>>
>
>>     $TC qdisc add dev $IF root handle 1: htb default 30
>>     $TC class add dev $IF parent 1: classid 1:1 htb rate $DNLD
>>     $TC class add dev $IF parent 1: classid 1:2 htb rate $UPLD
>>     $U32 match ip dst $IP/32 flowid 1:1
>>     $U32 match ip src $IP/32 flowid 1:2
>
> I *think* that you'll need a separate leaf class for each client on your
> network. I think you'll also need an overall rate limit for the root
> (which is kind of what you've already got above). If you want to avoid
> rate limiting the overall interface, then don't set a default and only
> filter by destination IP address.
>
> I'm a bit confused about what you want to control (see comment above),
> as you have references to upload and download limits. If you only want
> to limit the download stream, then you can't do this by ingress on the
> source interface. You'll have to either do it as egress on the outbound
> interface, or use an IFB interface. That said, I don't know how a VPN
> affects this and whether that makes a difference.
>
> So, in summary:
>
> - Use a root qdisc with an overall limit on the correct interface
> - Add a leaf class for each client
> - Filter into each leaf class based on IP address
>
> Andy
>
>
> 

Re: limit badwidth not working

[Andrew Beverley]

On Sun, 2011-01-02 at 16:18 +0100, J Webster wrote:
> > On Tue, 2010-12-28 at 12:12 +0100, J Webster wrote:
> >> I have a setup where uisers connect by VPN and are given IP addresses in 
> >> the
> >> range 10.8.0.xxx
> >
> > I can't advise on the VPN aspects of this, but see below for some
> > general comments.
> >
> >> I would like to limit their bandwidth to 1.5Mbps per IP address. 

<rant> Please don't top post. Even better, please also use an email
client that does proper quoting </rant>

> Do you know of any tutorials on this with examples? I've looked through the 
> main tc tutorials and they are pretty hard to follow.
> 

I found the following webpage to be really useful:

http://www.opalsoft.net/qos/DS-28.htm

> Re connections, my network is 100Mbps, I want to leave that as unlimited so 
> their is no overall bucket level.
> Users connect to the VPN and each of the IP addresses connected to the VPN 
> should have a limit of 1.5Mbps.
> 

The following is untested, but should give you an idea. $DEV should be
the *outbound* device, on the local network side, not the internet side.

# Add root qdisc
tc qdisc add dev $DEV root handle 1: htb

# Add parent class. The limit here should add up to all the leaf classes
tc class add dev $DEV parent 1: classid 1:1 htb rate 4.5mbit burst 15k

# Add leaf classes, each with 1.5mbit limit
tc class add dev $DEV parent 1:1 classid 1:10 htb rate 1.5mbit ceil 1.5mbit
tc class add dev $DEV parent 1:1 classid 1:20 htb rate 1.5mbit ceil 1.5mbit
tc class add dev $DEV parent 1:1 classid 1:30 htb rate 1.5mbit ceil 1.5mbit
...

# Add a filter to each leaf class to pipe in the traffic for each IP address
U32="tc filter add dev $DEV protocol ip parent 1:0 prio 1 u32"
$U32 match ip dst 10.0.8.1 flowid 1:10
$U32 match ip dst 10.0.8.2 flowid 1:20
$U32 match ip dst 10.0.8.3 flowid 1:30
...


Andy

Re: limit badwidth not working

[J Webster]

So 1:10, 1:20, 1:30 are the subclasses and I need 1 subcalss for every ip 
address?
I can set a max number connections on the openvpn server so say there are 20 
connections then I need to repeat the following 20 times?


$U32 match ip dst 10.0.8.1 flowid 1:10
all the way to
$U32 match ip dist 10.0.8.20 flowid 1:200 (not the 200 here for 20th 
subclass)?

Would subclass 1:10, 1:11, 1:12, 1:13 do the same?

It seems like the above will work when you know what the IP addresses are 
going to be on a VPN network. So, I don't need to know or care what the 
client's WAN IP address is to control the speed if they are connected via a 
VPN?