Adding SNAT support to LVS/NAT

Adding SNAT support to LVS/NAT

[Julius Volz]

Hi,

I'm wondering about how hard it would be to add SNAT support to IPVS
(that is, the director rewriting packets from the client to backends
to appear to be from the VIP and some randomly allocated port). This
would allow us to have remote real servers with LVS/NAT. I realize
that Jason Stubb's PRE/POSTROUTING patches would also make this
possible, but they seemed risky and we haven't heard of them in a long
time. Also, having this option directly integrated into the rest of
the IPVS NAT code might make it easier to use: just add another flag
on the ipvsadm command line when you want SNAT for an LVS/NAT backend.

Two problems I noticed immediately:

- Getting a free port:
We need to find an unused port for the SNAT for each connection. Is
there some subsystem function than can be used to easily find/allocate
a free TCP/UDP port (sockets and netfilter NAT would need this too)?
Also, this would only allow <64k connections to each backend. The only
way to possibly differentiate between more would be to look at
sequence numbers, but IPVS doesn't do that at all currently...

- Connection entry lookup:
Connection entries are currently hashed and looked up by [client IP,
client port]. In the new SNAT case, packets coming from the real
server to the director would have to be looked up by [VIP, xport],
where xport is a port that is allocated by IPVS for each connection. A
simple (hacky?) solution would be to just hash each connection entry
twice. Is there a better way?

Are there any other major problems with this? Is it the right way to
go in general? I'm mainly just doing some exploration into this now...

Thanks for any comments!

Julius

-- 
Julius Volz - Corporate Operations - SysOps

Google Switzerland GmbH - Identification No.: CH-020.4.028.116-1

Re: Adding SNAT support to LVS/NAT

[Joseph Mack NA3T]

On Sat, 13 Sep 2008, Julius Volz wrote:

> Hi,
>
> I'm wondering about how hard it would be to add SNAT 
> support to IPVS (that is, the director rewriting packets 
> from the client to backends to appear to be from the VIP 
> and some randomly allocated port).

when we've discussed it before, the packets will appear to 
come from the DIP, a private IP (or whatever IP the servers 
were using for their default route before they were put into 
an LVS).

> This would allow us to have remote real servers with 
> LVS/NAT.

more importantly, people who are bound by serious IT rules 
setup by managers, can duplicate their (real) servers only 
with a change in RIP on each server and can keep the default 
route.

> I realize that Jason Stubb's PRE/POSTROUTING patches would 
> also make this possible, but they seemed risky and we 
> haven't heard of them in a long time.

Jason just sent me his current patch this week. You could 
ask him for it directly. Apparently it's little/no different 
to the patch he posted at

http://archive.linuxvirtualserver.org/html/lvs-devel/2008-04/msg00041.html

(hope I'm paraphrasing Horms correctly here...)
Horms would be happy to have this in the main release if it 
could be invoked as an option.

> Also, having this option directly integrated into the rest 
> of the IPVS NAT code might make it easier to use: just add 
> another flag on the ipvsadm command line when you want 
> SNAT for an LVS/NAT backend.

sounds great to me.

>From talking to Horms, LVS-NAT (and possibly LVS-DR) sends 
the packets to the realservers via

ip_vs_post_routing()

which bypasses the OUTPUT chain. It would seem to me that if 
you instead injected the packets into the OUTPUT chain, then 
normal iptables rules could handle the SNAT to the 
realservers. It might be slower that rewriting the packets 
yourself, but for a first attempt, I think people will be so 
happy to have the functionality that they won't quibble 
about speed.

Joe

-- 
Joseph Mack NA3T EME(B,D), FM05lw North Carolina
jmack (at) wm7d (dot) net - azimuthal equidistant map
generator at http://www.wm7d.net/azproj.shtml
Homepage http://www.austintek.com/ It's GNU/Linux!

Re: Adding SNAT support to LVS/NAT

[Graeme Fowler]

Hi

On Sat, 2008-09-13 at 11:17 -0700, Joseph Mack NA3T wrote:
> when we've discussed it before, the packets will appear to 
> come from the DIP, a private IP (or whatever IP the servers 
> were using for their default route before they were put into 
> an LVS).

I'm a bit late to the party, but...

Ideally, the packets would appear to come from the "inside face" of the
director - usually (and in the most basic case) the NIC with an address
on the same layer 3 network as the real servers. This way the default
route would be ignored as there would normally be a more specific route
for that network, namely straight out of the device the packets arrived
on. Of course, if that coincides with the default gateway address, it'll
still work anyway.

> more importantly, people who are bound by serious IT rules 
> setup by managers, can duplicate their (real) servers only 
> with a change in RIP on each server and can keep the default 
> route.

This is the single most compelling thing about this setup.

It's simultaneously a problem in that for (say) an HTTP server, all the
requests will appear to be sourced from the director's internal address.
For many applications this won't be acceptable, not least in that for
the webserver case log processing will be an irrelevance!

Anyone know how the F5 hardware gets around this? I can't get my hands
on any to test...

Graeme

Re: Adding SNAT support to LVS/NAT

[Joseph Mack NA3T]

On Sat, 13 Sep 2008, Graeme Fowler wrote:

> It's simultaneously a problem in that for (say) an HTTP server, all the
> requests will appear to be sourced from the director's internal address.
> For many applications this won't be acceptable, not least in that for
> the webserver case log processing will be an irrelevance!

hmm, hadn't thought about this.

Joe

-- 
Joseph Mack NA3T EME(B,D), FM05lw North Carolina
jmack (at) wm7d (dot) net - azimuthal equidistant map
generator at http://www.wm7d.net/azproj.shtml
Homepage http://www.austintek.com/ It's GNU/Linux!

Re: Adding SNAT support to LVS/NAT

[Julius Volz]

On Sat, Sep 13, 2008 at 8:55 PM, Graeme Fowler <graeme@graemef.net> wrote:
> Hi
>
> On Sat, 2008-09-13 at 11:17 -0700, Joseph Mack NA3T wrote:
>> when we've discussed it before, the packets will appear to
>> come from the DIP, a private IP (or whatever IP the servers
>> were using for their default route before they were put into
>> an LVS).
>
> I'm a bit late to the party, but...
>
> Ideally, the packets would appear to come from the "inside face" of the
> director - usually (and in the most basic case) the NIC with an address
> on the same layer 3 network as the real servers. This way the default
> route would be ignored as there would normally be a more specific route
> for that network, namely straight out of the device the packets arrived
> on. Of course, if that coincides with the default gateway address, it'll
> still work anyway.

Yes. I think this is what I meant in my previous answer to Joe.

>> more importantly, people who are bound by serious IT rules
>> setup by managers, can duplicate their (real) servers only
>> with a change in RIP on each server and can keep the default
>> route.
>
> This is the single most compelling thing about this setup.
>
> It's simultaneously a problem in that for (say) an HTTP server, all the
> requests will appear to be sourced from the director's internal address.
> For many applications this won't be acceptable, not least in that for
> the webserver case log processing will be an irrelevance!
>
> Anyone know how the F5 hardware gets around this? I can't get my hands
> on any to test...

Yes, this is a known issue that is usually handled by injecting
something like an X-User-IP header into the HTTP request. Hard to do
in IPVS (could be done in an app helper though, I assume).

Julius

-- 
Julius Volz - Corporate Operations - SysOps

Google Switzerland GmbH - Identification No.: CH-020.4.028.116-1

Re: Adding SNAT support to LVS/NAT

[Julius Volz]

On Sat, Sep 13, 2008 at 8:17 PM, Joseph Mack NA3T <jmack@wm7d.net> wrote:
> On Sat, 13 Sep 2008, Julius Volz wrote:
>
>> Hi,
>>
>> I'm wondering about how hard it would be to add SNAT support to IPVS (that
>> is, the director rewriting packets from the client to backends to appear to
>> be from the VIP and some randomly allocated port).
>
> when we've discussed it before, the packets will appear to come from the
> DIP, a private IP (or whatever IP the servers were using for their default
> route before they were put into an LVS).

Good point, that would be better in some circumstances. We're already
looking up the route to the real server anyways and I think the route
entry also contains the source IP that a packet to that host would
have.

>> This would allow us to have remote real servers with LVS/NAT.
>
> more importantly, people who are bound by serious IT rules setup by
> managers, can duplicate their (real) servers only with a change in RIP on
> each server and can keep the default route.

Right!

>> I realize that Jason Stubb's PRE/POSTROUTING patches would also make this
>> possible, but they seemed risky and we haven't heard of them in a long time.
>
> Jason just sent me his current patch this week. You could ask him for it
> directly. Apparently it's little/no different to the patch he posted at
>
> http://archive.linuxvirtualserver.org/html/lvs-devel/2008-04/msg00041.html
>
> (hope I'm paraphrasing Horms correctly here...)
> Horms would be happy to have this in the main release if it could be invoked
> as an option.

Ok, just from the replies in that thread, it sounds like you have to
rethink many assumptions and cases which might be dangerous with that
patch. Or have these worries been resolved? Then it would be a nice
thing to have, of course.

>> Also, having this option directly integrated into the rest of the IPVS NAT
>> code might make it easier to use: just add another flag on the ipvsadm
>> command line when you want SNAT for an LVS/NAT backend.
>
> sounds great to me.
>
> From talking to Horms, LVS-NAT (and possibly LVS-DR) sends the packets to
> the realservers via
>
> ip_vs_post_routing()

Hm, that happens in ip_vs_in(), which then calls the connection's
transmit function. ip_vs_postrouting() just stops further netfilter
hook processing for packets handled by IPVS.

> which bypasses the OUTPUT chain. It would seem to me that if you instead
> injected the packets into the OUTPUT chain, then normal iptables rules could
> handle the SNAT to the realservers. It might be slower that rewriting the
> packets yourself, but for a first attempt, I think people will be so happy
> to have the functionality that they won't quibble about speed.

Looks like the packet is already injected into the OUTPUT chain in
IP_VS_XMIT(). To transmit the packet to the real server, it does:

NF_HOOK(PF_INET, NF_INET_LOCAL_OUT, (skb), NULL,
                (rt)->u.dst.dev, dst_output);

So maybe it would already work? ;)

Julius

-- 
Julius Volz - Corporate Operations - SysOps

Google Switzerland GmbH - Identification No.: CH-020.4.028.116-1

Re: Adding SNAT support to LVS/NAT

[Joseph Mack NA3T]

On Sun, 14 Sep 2008, Julius Volz wrote:

> So maybe it would already work? ;)

No. Some highly motivated people tried doing SNAT on OUTPUT 
in an attempt to do F5-SNAT and it didn't work. This lead to 
the write up in

http://www.austintek.com/LVS/LVS-HOWTO/HOWTO/LVS-HOWTO.non-modified_realservers.html#F5_snat

which brings us to where we are now.

Joe

-- 
Joseph Mack NA3T EME(B,D), FM05lw North Carolina
jmack (at) wm7d (dot) net - azimuthal equidistant map
generator at http://www.wm7d.net/azproj.shtml
Homepage http://www.austintek.com/ It's GNU/Linux!

Re: Adding SNAT support to LVS/NAT

[Julius Volz]

On Sun, Sep 14, 2008 at 3:37 AM, Joseph Mack NA3T <jmack@wm7d.net> wrote:
> On Sun, 14 Sep 2008, Julius Volz wrote:
>
>> So maybe it would already work? ;)
>
> No. Some highly motivated people tried doing SNAT on OUTPUT in an attempt to
> do F5-SNAT and it didn't work. This lead to the write up in
>
> http://www.austintek.com/LVS/LVS-HOWTO/HOWTO/LVS-HOWTO.non-modified_realservers.html#F5_snat
>
> which brings us to where we are now.

Thanks for the info! Right, I even said myself in the previous reply
that ip_vs_postrouting() stops further processing in the POSTROUTING
chain, so it never reaches netfilter NAT code.

I still think an in-IPVS SNAT would be nice anyways... don't know if
I'll find the time to hack on it soon, though.

Julius

-- 
Julius Volz - Corporate Operations - SysOps

Google Switzerland GmbH - Identification No.: CH-020.4.028.116-1

Re: Adding SNAT support to LVS/NAT

[Julius Volz]

On Sun, Sep 14, 2008 at 12:39 PM, Julius Volz <juliusv@google.com> wrote:
> On Sun, Sep 14, 2008 at 3:37 AM, Joseph Mack NA3T <jmack@wm7d.net> wrote:
>> On Sun, 14 Sep 2008, Julius Volz wrote:
>>
>>> So maybe it would already work? ;)
>>
>> No. Some highly motivated people tried doing SNAT on OUTPUT in an attempt to
>> do F5-SNAT and it didn't work. This lead to the write up in
>>
>> http://www.austintek.com/LVS/LVS-HOWTO/HOWTO/LVS-HOWTO.non-modified_realservers.html#F5_snat
>>
>> which brings us to where we are now.
>
> Thanks for the info! Right, I even said myself in the previous reply
> that ip_vs_postrouting() stops further processing in the POSTROUTING
> chain, so it never reaches netfilter NAT code.

Actually, what if we modify or remove that function to allow further
processing in POSTROUTING? Could SNAT work with IPVS then?

The comment above it says that the function specifically wants to
avoid further NAT by netfilter. But is this always a problem?

Julius

-- 
Julius Volz - Corporate Operations - SysOps

Google Switzerland GmbH - Identification No.: CH-020.4.028.116-1

Re: Adding SNAT support to LVS/NAT

[Joseph Mack NA3T]

On Sun, 14 Sep 2008, Julius Volz wrote:

> Actually, what if we modify or remove that function to 
> allow further processing in POSTROUTING? Could SNAT work 
> with IPVS then?

sounds OK by me. You should get an opinion from people who 
know the code better than I do.

> The comment above it says that the function specifically 
> wants to avoid further NAT by netfilter. But is this 
> always a problem?

I assume the function is there

o to speed up LVS (useful but not required for LVS-NAT)

o to push the layer 2 packet for LVS-DR out on the wire. 
This packet has the MAC address of the RIP, but 
dst_addr=VIP, so can't be sent through routing.

Joe

-- 
Joseph Mack NA3T EME(B,D), FM05lw North Carolina
jmack (at) wm7d (dot) net - azimuthal equidistant map
generator at http://www.wm7d.net/azproj.shtml
Homepage http://www.austintek.com/ It's GNU/Linux!

Re: Adding SNAT support to LVS/NAT

[Simon Horman]

On Sun, Sep 14, 2008 at 04:47:51PM +0200, Julius Volz wrote:
> On Sun, Sep 14, 2008 at 12:39 PM, Julius Volz <juliusv@google.com> wrote:
> > On Sun, Sep 14, 2008 at 3:37 AM, Joseph Mack NA3T <jmack@wm7d.net> wrote:
> >> On Sun, 14 Sep 2008, Julius Volz wrote:
> >>
> >>> So maybe it would already work? ;)
> >>
> >> No. Some highly motivated people tried doing SNAT on OUTPUT in an attempt to
> >> do F5-SNAT and it didn't work. This lead to the write up in
> >>
> >> http://www.austintek.com/LVS/LVS-HOWTO/HOWTO/LVS-HOWTO.non-modified_realservers.html#F5_snat
> >>
> >> which brings us to where we are now.
> >
> > Thanks for the info! Right, I even said myself in the previous reply
> > that ip_vs_postrouting() stops further processing in the POSTROUTING
> > chain, so it never reaches netfilter NAT code.
> 
> Actually, what if we modify or remove that function to allow further
> processing in POSTROUTING? Could SNAT work with IPVS then?
> 
> The comment above it says that the function specifically wants to
> avoid further NAT by netfilter. But is this always a problem?

Well, it would be a problem if it gets DNATed a second time.
But perhaps we can take a slightly different approach such that
we protect against DNAT while allowing SNAT.


Perhaps it might just be easier to allow iptables to explictly match
packets that have been mangled by LVS-NAT. Perhaps by poviding
a match rule for skb->ipvs_property? Or by using Siim Põder's match
against connections in the LVS connection table.

http://lists.graemef.net/pipermail/lvs-users/2008-July/021081.html

-- 
Simon Horman
  VA Linux Systems Japan K.K., Sydney, Australia Satellite Office
  H: www.vergenet.net/~horms/             W: www.valinux.co.jp/en

Re: Adding SNAT support to LVS/NAT

[Joseph Mack NA3T]

On Mon, 15 Sep 2008, Simon Horman wrote:

> Well, it would be a problem if it gets DNATed a second time.

Are you just being really safe? Are you trying to prevent 
someone from adding DNAT rules to OUTPUT?

Would it be better (as much as possible) for LVS to appear 
to be just another netfilter module, in which case if 
someone wants to DNAT in OUTPUT, this should be allowed 
(whether it's sensible or not). Currently LVS-NAT doesn't 
allow SNAT on OUTPUT, which no-one thought about when 
LVS-NAT was first written and it turns out to be useful.

Joe

-- 
Joseph Mack NA3T EME(B,D), FM05lw North Carolina
jmack (at) wm7d (dot) net - azimuthal equidistant map
generator at http://www.wm7d.net/azproj.shtml
Homepage http://www.austintek.com/ It's GNU/Linux!

Re: Adding SNAT support to LVS/NAT

[Jason Stubbs]

On Tuesday 16 September 2008 00:24:38 Joseph Mack NA3T wrote:
> On Mon, 15 Sep 2008, Simon Horman wrote:
> 
> > Well, it would be a problem if it gets DNATed a second time.
> 
> Are you just being really safe? Are you trying to prevent 
> someone from adding DNAT rules to OUTPUT?
> 
> Would it be better (as much as possible) for LVS to appear 
> to be just another netfilter module, in which case if 
> someone wants to DNAT in OUTPUT, this should be allowed 
> (whether it's sensible or not). Currently LVS-NAT doesn't 
> allow SNAT on OUTPUT, which no-one thought about when 
> LVS-NAT was first written and it turns out to be useful.

For what it's worth, I'm currently using DNAT alongside LVS-NAT
for certain connections. It only serves a secondary purpose and
there are other ways (although not as simple) to achieve the
purpose, but it has proven useful.

-- 
Jason Stubbs <j.stubbs@linkthink.co.jp>
LINKTHINK INC.
東京都渋谷区桜ヶ丘町22-14 N.E.S S棟 3F
TEL 03-5728-4772  FAX 03-5728-4773

Re: Adding SNAT support to LVS/NAT

[Joseph Mack NA3T]

On Tue, 16 Sep 2008, Jason Stubbs wrote:

> For what it's worth, I'm currently using DNAT alongside LVS-NAT
> for certain connections. It only serves a secondary purpose and
> there are other ways (although not as simple) to achieve the
> purpose, but it has proven useful.

I'm having a bit of trouble fitting DNAT into the OUTPUT of 
a director (at least on the OUTPUT of the inside nic). What 
are you doing this for?

Joe

-- 
Joseph Mack NA3T EME(B,D), FM05lw North Carolina
jmack (at) wm7d (dot) net - azimuthal equidistant map
generator at http://www.wm7d.net/azproj.shtml
Homepage http://www.austintek.com/ It's GNU/Linux!

Re: Adding SNAT support to LVS/NAT

[Jason Stubbs]

On Tuesday 16 September 2008 10:54:04 Joseph Mack NA3T wrote:
> On Tue, 16 Sep 2008, Jason Stubbs wrote:
> 
> > For what it's worth, I'm currently using DNAT alongside LVS-NAT
> > for certain connections. It only serves a secondary purpose and
> > there are other ways (although not as simple) to achieve the
> > purpose, but it has proven useful.
> 
> I'm having a bit of trouble fitting DNAT into the OUTPUT of 
> a director (at least on the OUTPUT of the inside nic). What 
> are you doing this for?

Hmm.. with LVS-NAT happening in OUTPUT, it may not be possible - or at
least not useful. I'm using DNAT before LVS-NAT to redirect the VIP to
a different internal IP for certain source addresses. If LVS-NAT
happens before DNAT, DNAT could only be used with RIPs which I can't
imagine to be useful at all...

-- 
Jason Stubbs <j.stubbs@linkthink.co.jp>
LINKTHINK INC.
東京都渋谷区桜ヶ丘町22-14 N.E.S S棟 3F
TEL 03-5728-4772  FAX 03-5728-4773

Re: Adding SNAT support to LVS/NAT

[Julius Volz]

On Mon, Sep 15, 2008 at 3:43 AM, Simon Horman <horms@verge.net.au> wrote:
> On Sun, Sep 14, 2008 at 04:47:51PM +0200, Julius Volz wrote:
>> On Sun, Sep 14, 2008 at 12:39 PM, Julius Volz <juliusv@google.com> wrote:
>> > On Sun, Sep 14, 2008 at 3:37 AM, Joseph Mack NA3T <jmack@wm7d.net> wrote:
>> >> On Sun, 14 Sep 2008, Julius Volz wrote:
>> >>
>> >>> So maybe it would already work? ;)
>> >>
>> >> No. Some highly motivated people tried doing SNAT on OUTPUT in an attempt to
>> >> do F5-SNAT and it didn't work. This lead to the write up in
>> >>
>> >> http://www.austintek.com/LVS/LVS-HOWTO/HOWTO/LVS-HOWTO.non-modified_realservers.html#F5_snat
>> >>
>> >> which brings us to where we are now.
>> >
>> > Thanks for the info! Right, I even said myself in the previous reply
>> > that ip_vs_postrouting() stops further processing in the POSTROUTING
>> > chain, so it never reaches netfilter NAT code.
>>
>> Actually, what if we modify or remove that function to allow further
>> processing in POSTROUTING? Could SNAT work with IPVS then?
>>
>> The comment above it says that the function specifically wants to
>> avoid further NAT by netfilter. But is this always a problem?
>
> Well, it would be a problem if it gets DNATed a second time.
> But perhaps we can take a slightly different approach such that
> we protect against DNAT while allowing SNAT.

So I did some experiments and removed the ip_vs_post_routing()
function that stops further POSTROUTING processing. I added an SNAT
rule to POSTROUTING like this:

$ iptables –t nat –A POSTROUTING –o eth1 –j SNAT –-to $DIP

Amazingly, the first SYN and the SYN/ACK of a TCP connection to the
VIP:vport do not traverse the NAT chain in POSTROUTING at all
(verified by LOG target), so their source is not rewritten (like in
normal LVS/NAT). However, the client's ACK response to the SYN/ACK
finally appears in this table and only this third packet is SNATed
correctly (the real server then answers with an RST, since it doesn't
know a connection from that source). The packet trace on the director
looks something like this:

Packet 1:
CIP => VIP SYN (eth0, before IPVS)
CIP => RIP SYN (eth1, after IPVS)

Packet 2:
RIP => CIP SYN/ACK (eth1, before IPVS)
VIP => CIP SYN/ACK (eth0, after IPVS)

Packet 3:
CIP => VIP ACK (eth0, before IPVS)
DIP => RIP ACK (eth1, after IPVS) - this is the first successfully
SNATed packet!

Packet 4:
RIP => DIP RST - real server doesn't know this connection from the DIP

Any ideas why the first packets never appear in the POSTROUTING chain?
I do not understand this, since IPVS injects all handled packets into
the OUTPUT chain, whereafter they should traverse POSTROUTING...

Without any IPVS, SNAT works without any problems and as expected,
only the first (SYN) packet of a connection appears on the POSTROUTING
chain.

> Perhaps it might just be easier to allow iptables to explictly match
> packets that have been mangled by LVS-NAT. Perhaps by poviding
> a match rule for skb->ipvs_property? Or by using Siim Põder's match
> against connections in the LVS connection table.
>
> http://lists.graemef.net/pipermail/lvs-users/2008-July/021081.html

Thanks, interesting! Though for processing an already IPVS-ed packet
in POSTROUTING, it should be enough to match on skb->ipvs_property, I
think.

Julius

-- 
Julius Volz - Corporate Operations - SysOps

Google Switzerland GmbH - Identification No.: CH-020.4.028.116-1

Re: Adding SNAT support to LVS/NAT

[Joseph Mack NA3T]

On Tue, 16 Sep 2008, Julius Volz wrote:

> Amazingly, the first SYN and the SYN/ACK of a TCP connection to the
> VIP:vport do not traverse the NAT chain in POSTROUTING at all

:-(

> (verified by LOG target),

you didn't see the packets in the logs?

Joe

-- 
Joseph Mack NA3T EME(B,D), FM05lw North Carolina
jmack (at) wm7d (dot) net - azimuthal equidistant map
generator at http://www.wm7d.net/azproj.shtml
Homepage http://www.austintek.com/ It's GNU/Linux!

Re: Adding SNAT support to LVS/NAT

[Julius Volz]

On Thu, Sep 18, 2008 at 12:53 AM, Joseph Mack NA3T <jmack@wm7d.net> wrote:
> On Tue, 16 Sep 2008, Julius Volz wrote:
>
>> Amazingly, the first SYN and the SYN/ACK of a TCP connection to the
>> VIP:vport do not traverse the NAT chain in POSTROUTING at all
>
> :-(
>
>> (verified by LOG target),
>
> you didn't see the packets in the logs?

Exactly. No matter what I do, only the ACK in response to the SYN/ACK
appears in the logs. With SNAT without IPVS, the SYN packet correctly
enters the chain/table.

I haven't found anything in the IPVS or Netfilter code yet that could
cause this problem...

Julius

-- 
Julius Volz - Corporate Operations - SysOps

Google Switzerland GmbH - Identification No.: CH-020.4.028.116-1

Re: Adding SNAT support to LVS/NAT

[Julian Anastasov]

	Hello,

On Sun, 14 Sep 2008, Julius Volz wrote:

> > Thanks for the info! Right, I even said myself in the previous reply
> > that ip_vs_postrouting() stops further processing in the POSTROUTING
> > chain, so it never reaches netfilter NAT code.
> 
> Actually, what if we modify or remove that function to allow further
> processing in POSTROUTING? Could SNAT work with IPVS then?
> 
> The comment above it says that the function specifically wants to
> avoid further NAT by netfilter. But is this always a problem?

	This check (now flag ipvs_property) was implemented to avoid
netfilter to modify packet which was already changed by IPVS.
What happened was that FTP commands (TCP header and payload) were
modified first by ip_vs_ftp and then by netfilter. The result:
packet with wrong SEQ number. Later, after some Netfilter 
changes (2.6.11), TCP payload was modified always in POST_ROUTING
while address can be  modified in PRE_ROUTING. Not sure what happens
now, Netfilter code was reorganized and new code review and tests
are needed, may be such double manipulation (if ipvs_property is
not set) still can cause problems.

Regards

--
Julian Anastasov <ja@ssi.bg>