* Re: oops in tcp_xmit_retransmit_queue() w/ v2.6.32.15
From: Eric Dumazet @ 2010-07-11 17:46 UTC (permalink / raw)
To: Ilpo Järvinen
Cc: Tejun Heo, David S. Miller, lkml, netdev@vger.kernel.org,
Fehrmann, Henning, Carsten Aulbert
In-Reply-To: <1278867977.2538.167.camel@edumazet-laptop>
Le dimanche 11 juillet 2010 à 19:06 +0200, Eric Dumazet a écrit :
> Le dimanche 11 juillet 2010 à 19:09 +0300, Ilpo Järvinen a écrit :
> > On Thu, 8 Jul 2010, Tejun Heo wrote:
> >
> > > We've been seeing oops in tcp_xmit_retransmit_queue() w/ 2.6.32.15.
> > > Please see the attached photoshoot. This is happening on a HPC
> > > cluster and very interestingly caused by one particular job. How long
> > > it takes isn't clear yet (at least more than a day) but when it
> > > happens it happens on a lot of machines in relatively short time.
> > >
> > > With a bit of disassemblying, I've found that the oops is happening
> > > during tcp_for_write_queue_from() because the skb->next points to
> > > NULL.
> > >
> > > void tcp_xmit_retransmit_queue(struct sock *sk)
> > > {
> > > ...
> > > if (tp->retransmit_skb_hint) {
> > > skb = tp->retransmit_skb_hint;
> > > last_lost = TCP_SKB_CB(skb)->end_seq;
> > > if (after(last_lost, tp->retransmit_high))
> > > last_lost = tp->retransmit_high;
> > > } else {
> > > skb = tcp_write_queue_head(sk);
> > > last_lost = tp->snd_una;
> > > }
> > >
> > > => tcp_for_write_queue_from(skb, sk) {
> > > __u8 sacked = TCP_SKB_CB(skb)->sacked;
> > >
> > > if (skb == tcp_send_head(sk))
> > > break;
> > > /* we could do better than to assign each time */
> > > if (hole == NULL)
> > >
> > > This can happen for one of the following reasons,
> > >
> > > 1. tp->retransmit_skb_hint is NULL and tcp_write_queue_head() is NULL
> > > too. ie. tcp_xmit_retransmit_queue() is called on an empty write
> > > queue for some reason.
> > >
> > > 2. tp->retransmit_skb_hint is pointing to a skb which is not on the
> > > write_queue. ie. somebody forgot to update hint while removing the
> > > skb from the write queue.
> >
> > Once again I've read the unlinkers through, and only thing that could
> > cause this is tcp_send_synack (others do deal with the hints) but I think
> > Eric already proposed a patch to that but we never got anywhere due to
> > some counterargument why it wouldn't take place (too far away for me to
> > remember, see archives about the discussions). ...But if you want be dead
> > sure some WARN_ON there might not hurt. Also the purging of the whole
> > queue was a similar suspect I then came across (but that would only
> > materialize with sk reuse happening e.g., with nfs which the other guys
> > weren't using).
> >
>
> Hmm.
>
> This sounds familiar to me, but I cannot remember the discussion you
> mention or the patch.
>
> Or maybe it was the TCP transaction thing ? (including data in SYN or
> SYN-ACK packet)
Hmm, I cannot find where we reset restransmit_skb_hint in
tcp_mtu_probe(), if we call tcp_unlink_write_queue().
if (skb->len <= copy) {
/* We've eaten all the data from this skb.
* Throw it away. */
TCP_SKB_CB(nskb)->flags |= TCP_SKB_CB(skb)->flags;
<<>> tcp_unlink_write_queue(skb, sk);
sk_wmem_free_skb(sk, skb);
} else {
Sorry if this was already discussed. We might add a comment here in anycase ;)
^ permalink raw reply
* RE: [PATCH] bnx2x: add support for receive hashing
From: Eric Dumazet @ 2010-07-11 17:41 UTC (permalink / raw)
To: Vladislav Zolotarov
Cc: David Miller, therbert@google.com, netdev@vger.kernel.org
In-Reply-To: <8628FE4E7912BF47A96AE7DD7BAC0AADDDE6470521@SJEXCHCCR02.corp.ad.broadcom.com>
Le dimanche 11 juillet 2010 à 10:22 -0700, Vladislav Zolotarov a écrit :
>
> > -----Original Message-----
> > From: netdev-owner@vger.kernel.org [mailto:netdev-owner@vger.kernel.org] On
> > Behalf Of Eric Dumazet
> > Sent: Sunday, July 11, 2010 7:45 PM
> > To: Vladislav Zolotarov
> > Cc: David Miller; therbert@google.com; netdev@vger.kernel.org
> > Subject: RE: [PATCH] bnx2x: add support for receive hashing
> >
> > Le dimanche 11 juillet 2010 à 06:16 -0700, Vladislav Zolotarov a écrit :
> > > Dave, it's obvious that there a demand for a new HW/FW configuration
> > > from our side - "rx hash enable" which is currently tightly coupled
> > > with the RSS capability. As long as RSS flow in our FW includes a few
> > > more things apart from just creating an RSS hash and as long as there
> > > are flows (even hypothetical) that demand the RSS hash regardless the
> > > RSS itself we started to work on separation of these two features from
> > > FW perspective. This of course will demand a new FW version but once
> > > we have it we'll be able to be more specific in HW configuration and
> > > have a cleaner code.
> > >
> > > Technically, our FW may provide the Rx HASH always and in a current
> > > driver configuration this is what it does.
> > > I wonder if the driver always provides the HW RX HASH in the
> > > skb->rxhash regardless the value of NETIF_F_RXHASH bit in a
> > > netdev->features will it cause any harm? If not we can get rid of two
> > > extra conditionals in the bnx2x_rx_int().
> >
> > Hi
> >
> > Please dont top-post on netdev, thanks.
>
> This discussion is directly related to Tom's patch that's why I'm posting on this thread.
>
Thats not the question. I am saying "dont top-post", not "dont change
the subject"
> >
> > NETIF_RX_HASH bit is needed so that we can disable skb->rxhash from a
> > particular NIC if we know the hardware provided rxhash is not fulfilling
> > our needs (We prefer spend some cpu cycles to recompute a software
> > rxhash).
> >
> > Software one for example deliver same rxhash values for both ways of a
> > TCP flow, it can help conntracking for example.
>
> I understand that, in that case the proper implementation would be to check the netdev->features when u decide to calculate the SW rxhash, isn't it?
>
Nope, please check get_rps_cpus()
We only do :
if (skb->rxhash)
goto got_hash; /* Skip hash computation on packet header */
That means if skb->rxhash is set, we dont force a recompute.
Your suggestion would move a test from device driver into
get_rps_cpus() ?
Given RPS is more targeted to old devices (not able to provide rxhash),
I am not sure it brings anything.
> >
> > The conditional in driver rx is cheap, since the condition is the same
> > for all packets and CPU can predicts the branch.
>
> Not exactly. Our FW/HW won't provide the rxhash for none-IP packets
> setting the hash CQE field to zero and clearing the
> ETH_FAST_PATH_RX_CQE_RSS_HASH_FLG in the CQE statsu_flags (for ARPs
> for instance). Branch prediction is nice but considering my previous
> remark why do we need this branch at all?
Please limit your lines to 70 chars
We want drivers to set skb->rxhash only if allowed to.
If you feel this is bad for your firmware/hardware/driver, dont set
skb->rxhash.
^ permalink raw reply
* FW: NET_NS: unregister_netdevice: waiting for lo to become free (after using openvpn) (was Re: sysfs bug when using tun with network namespaces)
From: Michael Leun @ 2010-07-11 17:29 UTC (permalink / raw)
To: netdev; +Cc: davem, linux-kernel
In-Reply-To: <20100710235323.5336f627@xenia.leun.net>
Hi,
On Sat, 10 Jul 2010 16:52:08 +0200
Michael Leun <lkml20100708@newton.leun.net> wrote:
> # tunctl -u ml -t tap1
> works as expected, but
> # unshare -n /bin/bash
> # tunctl -u ml -t tap1
[bug (no sysfs support for net namespaces at all) solved in 2.6.35-rcX
- I used 2.6.34.1]
Now that we have solved that last one I've another glitch (this time using 2.6.35-rc4):
In an network namespace I can use an tun/tap tunnel through ssh and
when closing that namespace then eveything is fine.
But when using openvpn (also tunnel trough tun/tap) in an network
namespace and then closing that namespace I get:
unregister_netdevice: waiting for lo to become free
[repeated]
Please see the following two examples showing that difference:
# > unshare -n /bin/bash
# > # how to setup veth device pair to get connectivity into namespace not shown here
# > tunctl -u ml -t tap1
# > ssh -o Tunnel=Ethernet -w 1:1 somewhere
[ running some traffic over tap1 not shown here ]
^d # logging out from somewhere
# > tunctl -d tap1
# > exit # logging out from shell in network namespace
Now the veth device pair used automagically vanishes and nothing
from that different network namespace remains - very well.
but
# > unshare -n /bin/bash
# > # how to setup veth device pair to get connectivity into namespace not shown here
# > openvpn --config some.config
[ running some traffic over vpn device not shown here ]
^c # stopping openvpn
# > lsof -i
# > netstat -an
Active Internet connections (servers and established)
Proto Recv-Q Send-Q Local Address Foreign Address State
Active UNIX domain sockets (servers and established)
Proto RefCnt Flags Type State I-Node Path
# > ps ax|grep openvpn|grep -v grep
# > # cannot find anything that suggests there is anything left from that openvpn session
# > exit # logging out from shell in network namespace
Now I get
Jul 10 20:02:36 doris kernel: unregister_netdevice: waiting for lo to become free. Usage count = 3
[repeated]
Now one might say it is fault of openvpn (used OpenVPN 2.1_rc20
i586-suse-linux - the one in openSuSE 11.2 package), openvpn didn't
close some ressource and ssh does fine.
But: should'nt kernel clean up after process when it exits?
And/or: Should'nt kernel clean up if last process in network namespace
exits - there is nothing left which might use that interface?!
Greg KH <greg@kroah.com> wrote:
> Yes, you are correct. Care to resend all of this to the
> network-namespace developer(s) and the netdev mailing list so that the
> correct people are notified so they can fix it all?
[X] done - hopefully, cannot find a particular network namespace
developer in MAINTAINERS or source files. If such a one exists, please
forward.
Thanks.
--
MfG,
Michael Leun
^ permalink raw reply
* RE: [PATCH] bnx2x: add support for receive hashing
From: Vladislav Zolotarov @ 2010-07-11 17:22 UTC (permalink / raw)
To: Eric Dumazet; +Cc: David Miller, therbert@google.com, netdev@vger.kernel.org
In-Reply-To: <1278866713.2538.148.camel@edumazet-laptop>
> -----Original Message-----
> From: netdev-owner@vger.kernel.org [mailto:netdev-owner@vger.kernel.org] On
> Behalf Of Eric Dumazet
> Sent: Sunday, July 11, 2010 7:45 PM
> To: Vladislav Zolotarov
> Cc: David Miller; therbert@google.com; netdev@vger.kernel.org
> Subject: RE: [PATCH] bnx2x: add support for receive hashing
>
> Le dimanche 11 juillet 2010 à 06:16 -0700, Vladislav Zolotarov a écrit :
> > Dave, it's obvious that there a demand for a new HW/FW configuration
> > from our side - "rx hash enable" which is currently tightly coupled
> > with the RSS capability. As long as RSS flow in our FW includes a few
> > more things apart from just creating an RSS hash and as long as there
> > are flows (even hypothetical) that demand the RSS hash regardless the
> > RSS itself we started to work on separation of these two features from
> > FW perspective. This of course will demand a new FW version but once
> > we have it we'll be able to be more specific in HW configuration and
> > have a cleaner code.
> >
> > Technically, our FW may provide the Rx HASH always and in a current
> > driver configuration this is what it does.
> > I wonder if the driver always provides the HW RX HASH in the
> > skb->rxhash regardless the value of NETIF_F_RXHASH bit in a
> > netdev->features will it cause any harm? If not we can get rid of two
> > extra conditionals in the bnx2x_rx_int().
>
> Hi
>
> Please dont top-post on netdev, thanks.
This discussion is directly related to Tom's patch that's why I'm posting on this thread.
>
> NETIF_RX_HASH bit is needed so that we can disable skb->rxhash from a
> particular NIC if we know the hardware provided rxhash is not fulfilling
> our needs (We prefer spend some cpu cycles to recompute a software
> rxhash).
>
> Software one for example deliver same rxhash values for both ways of a
> TCP flow, it can help conntracking for example.
I understand that, in that case the proper implementation would be to check the netdev->features when u decide to calculate the SW rxhash, isn't it?
>
> The conditional in driver rx is cheap, since the condition is the same
> for all packets and CPU can predicts the branch.
Not exactly. Our FW/HW won't provide the rxhash for none-IP packets setting the hash CQE field to zero and clearing the ETH_FAST_PATH_RX_CQE_RSS_HASH_FLG in the CQE statsu_flags (for ARPs for instance). Branch prediction is nice but considering my previous remark why do we need this branch at all?
>
>
>
> --
> To unsubscribe from this list: send the line "unsubscribe netdev" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply
* Re: [REGRESSION,BISECTED] Panic on ifup
From: George Spelvin @ 2010-07-11 17:09 UTC (permalink / raw)
To: linux, timo.teras; +Cc: davem, netdev
In-Reply-To: <4C39D834.8080206@iki.fi>
> On 07/11/2010 03:38 PM, George Spelvin wrote:
>> No, although /etc/ipec-tools.conf runs setkey. As I said,
>> I was mostly running in single-user mode; a ps axf
>> output is appended.
>>
>> Ah! A discovery! There are rules for the gateway!
>>
>> add <my_ip> <gw_ip> esp 0x200 -E aes-cbc
>> <key>;
>> add <gw_ip> <my_ip> esp 0x300 -E aes-cbc
>> <key>;
>> spdadd <gw_ip> <my_ip> any -P in ipsec
>> esp/transport//use;
>> spdadd <my_ip> <gw_ip> any -P out ipsec
>> esp/transport//use;
> This means optional encryption. Probably something goes wrong
> constructing the bundle which results in encryption not being applied.
> And it would look like xfrm_resolve_and_create_bundle() does not take
> this into account properly. I need to fix it with optional policies.
>
> In the mean while, could confirm if everything works if you change the
> last line to:
> esp/transport//require;
Will do.
This might lead to no traffic if there's something else broken, however
it should not crash. This change is needed only for the 'out' policy, as
the bundles are used only on xmit code paths.
> yup, xfrm_resolve_and_create_bundle looks to be the culprit. I'll try to
> figure out a patch for testing.
> Ok, this is basically what setkey did for you. Looks like it was ran
> twice and you are missing flush and spdflush from setkey, so you get
> duplicates here. Otherwise it's ok.
Um, I am *not* missing flush and spdflush. The entire file, with comments
and blank lines stripped, and some details censored, is:
#!/usr/sbin/setkey -f
flush;
spdflush;
add $LOCALNET.1 $LOCALNET.62 esp 0x200 -E aes-cbc <key redacted>;
add $LOCALNET.62 $LOCALNET.1 esp 0x300 -E aes-cbc <key redacted>;
add $LOCALNET.3 $LOCALNET.62 esp 0x400 -E aes-cbc <key redacted>;
add $LOCALNET.62 $LOCALNET.3 esp 0x500 -E aes-cbc <key redacted>;
spdadd $LOCALNET.1 $LOCALNET.62 any -P in ipsec esp/transport//use;
spdadd $LOCALNET.62 $LOCALNET.1 any -P out ipsec esp/transport//use;
spdadd $LOCALNET.3 $LOCALNET.62 any -P in ipsec esp/transport//use;
spdadd $LOCALNET.62 $LOCALNET.3 any -P out ipsec esp/transport//use;
Anyway, thank you very much!
^ permalink raw reply
* Re: oops in tcp_xmit_retransmit_queue() w/ v2.6.32.15
From: Eric Dumazet @ 2010-07-11 17:06 UTC (permalink / raw)
To: Ilpo Järvinen
Cc: Tejun Heo, David S. Miller, lkml, netdev@vger.kernel.org,
Fehrmann, Henning, Carsten Aulbert
In-Reply-To: <alpine.DEB.2.00.1007111825510.15736@melkinpaasi.cs.helsinki.fi>
Le dimanche 11 juillet 2010 à 19:09 +0300, Ilpo Järvinen a écrit :
> On Thu, 8 Jul 2010, Tejun Heo wrote:
>
> > We've been seeing oops in tcp_xmit_retransmit_queue() w/ 2.6.32.15.
> > Please see the attached photoshoot. This is happening on a HPC
> > cluster and very interestingly caused by one particular job. How long
> > it takes isn't clear yet (at least more than a day) but when it
> > happens it happens on a lot of machines in relatively short time.
> >
> > With a bit of disassemblying, I've found that the oops is happening
> > during tcp_for_write_queue_from() because the skb->next points to
> > NULL.
> >
> > void tcp_xmit_retransmit_queue(struct sock *sk)
> > {
> > ...
> > if (tp->retransmit_skb_hint) {
> > skb = tp->retransmit_skb_hint;
> > last_lost = TCP_SKB_CB(skb)->end_seq;
> > if (after(last_lost, tp->retransmit_high))
> > last_lost = tp->retransmit_high;
> > } else {
> > skb = tcp_write_queue_head(sk);
> > last_lost = tp->snd_una;
> > }
> >
> > => tcp_for_write_queue_from(skb, sk) {
> > __u8 sacked = TCP_SKB_CB(skb)->sacked;
> >
> > if (skb == tcp_send_head(sk))
> > break;
> > /* we could do better than to assign each time */
> > if (hole == NULL)
> >
> > This can happen for one of the following reasons,
> >
> > 1. tp->retransmit_skb_hint is NULL and tcp_write_queue_head() is NULL
> > too. ie. tcp_xmit_retransmit_queue() is called on an empty write
> > queue for some reason.
> >
> > 2. tp->retransmit_skb_hint is pointing to a skb which is not on the
> > write_queue. ie. somebody forgot to update hint while removing the
> > skb from the write queue.
>
> Once again I've read the unlinkers through, and only thing that could
> cause this is tcp_send_synack (others do deal with the hints) but I think
> Eric already proposed a patch to that but we never got anywhere due to
> some counterargument why it wouldn't take place (too far away for me to
> remember, see archives about the discussions). ...But if you want be dead
> sure some WARN_ON there might not hurt. Also the purging of the whole
> queue was a similar suspect I then came across (but that would only
> materialize with sk reuse happening e.g., with nfs which the other guys
> weren't using).
>
Hmm.
This sounds familiar to me, but I cannot remember the discussion you
mention or the patch.
Or maybe it was the TCP transaction thing ? (including data in SYN or
SYN-ACK packet)
> > 3. The hint is pointing to a skb on the list but the list itself is
> > corrupt.
> >
> > I added some debug code and the crash is happening when
> > tp->retransmit_skb_hint is not NULL but tp->retransmit_skb_hint->next
> > is NULL. So, #1 is out; unfortunately, I didn't have debug code in
> > place to discern between #2 and #3.
> >
> > Does anything ring a bell? This is a production system and debugging
> > affects quite a number of people. I can put debug code in to discern
> > between #2 and #3 but I'm basically shooting in the dark and it would
> > be great if someone has a better idea.
>
> Thanks for taking this up. I've been kind of waiting somebody to show up
> who actually has some way of reproducing it. Once I had one guy in the
> hook but his ability to reproduce was for some reason lost when he tried
> with a debug patch [1].
>
> I now realize that the debug patch should probably also print the write
> queue too when the problem is caught in order to discern the cases you
> mention.
>
> Something along these lines:
>
> tcp_for_write_queue(skb, sk) {
> printk("skb %p (%u-%u) next %p prev %p sacked %u\n", ...);
> }
>
> Anyway, my debugging patch should be such that in a lucky case it avoids
> crashing the system too, though price to pay might then be a stuck
> connection. In case #3 I'd expect the box to die elsewhere in TCP code
> pretty soon anyway so it depends whether avoiding oops is really so
> useful, but if you're lucky other mechanism in TCP will recover
> the lost one for you (basically RTO driven retransmission).
>
^ permalink raw reply
* RE: [PATCH] bnx2x: add support for receive hashing
From: Eric Dumazet @ 2010-07-11 16:45 UTC (permalink / raw)
To: Vladislav Zolotarov
Cc: David Miller, therbert@google.com, netdev@vger.kernel.org
In-Reply-To: <8628FE4E7912BF47A96AE7DD7BAC0AADDDE64704FA@SJEXCHCCR02.corp.ad.broadcom.com>
Le dimanche 11 juillet 2010 à 06:16 -0700, Vladislav Zolotarov a écrit :
> Dave, it's obvious that there a demand for a new HW/FW configuration
> from our side - "rx hash enable" which is currently tightly coupled
> with the RSS capability. As long as RSS flow in our FW includes a few
> more things apart from just creating an RSS hash and as long as there
> are flows (even hypothetical) that demand the RSS hash regardless the
> RSS itself we started to work on separation of these two features from
> FW perspective. This of course will demand a new FW version but once
> we have it we'll be able to be more specific in HW configuration and
> have a cleaner code.
>
> Technically, our FW may provide the Rx HASH always and in a current
> driver configuration this is what it does.
> I wonder if the driver always provides the HW RX HASH in the
> skb->rxhash regardless the value of NETIF_F_RXHASH bit in a
> netdev->features will it cause any harm? If not we can get rid of two
> extra conditionals in the bnx2x_rx_int().
Hi
Please dont top-post on netdev, thanks.
NETIF_RX_HASH bit is needed so that we can disable skb->rxhash from a
particular NIC if we know the hardware provided rxhash is not fulfilling
our needs (We prefer spend some cpu cycles to recompute a software
rxhash).
Software one for example deliver same rxhash values for both ways of a
TCP flow, it can help conntracking for example.
The conditional in driver rx is cheap, since the condition is the same
for all packets and CPU can predicts the branch.
^ permalink raw reply
* Re: oops in tcp_xmit_retransmit_queue() w/ v2.6.32.15
From: Ilpo Järvinen @ 2010-07-11 16:09 UTC (permalink / raw)
To: Tejun Heo
Cc: David S. Miller, lkml, netdev@vger.kernel.org, Fehrmann, Henning,
Carsten Aulbert, Eric Dumazet
In-Reply-To: <4C358AAA.9080400@kernel.org>
On Thu, 8 Jul 2010, Tejun Heo wrote:
> We've been seeing oops in tcp_xmit_retransmit_queue() w/ 2.6.32.15.
> Please see the attached photoshoot. This is happening on a HPC
> cluster and very interestingly caused by one particular job. How long
> it takes isn't clear yet (at least more than a day) but when it
> happens it happens on a lot of machines in relatively short time.
>
> With a bit of disassemblying, I've found that the oops is happening
> during tcp_for_write_queue_from() because the skb->next points to
> NULL.
>
> void tcp_xmit_retransmit_queue(struct sock *sk)
> {
> ...
> if (tp->retransmit_skb_hint) {
> skb = tp->retransmit_skb_hint;
> last_lost = TCP_SKB_CB(skb)->end_seq;
> if (after(last_lost, tp->retransmit_high))
> last_lost = tp->retransmit_high;
> } else {
> skb = tcp_write_queue_head(sk);
> last_lost = tp->snd_una;
> }
>
> => tcp_for_write_queue_from(skb, sk) {
> __u8 sacked = TCP_SKB_CB(skb)->sacked;
>
> if (skb == tcp_send_head(sk))
> break;
> /* we could do better than to assign each time */
> if (hole == NULL)
>
> This can happen for one of the following reasons,
>
> 1. tp->retransmit_skb_hint is NULL and tcp_write_queue_head() is NULL
> too. ie. tcp_xmit_retransmit_queue() is called on an empty write
> queue for some reason.
>
> 2. tp->retransmit_skb_hint is pointing to a skb which is not on the
> write_queue. ie. somebody forgot to update hint while removing the
> skb from the write queue.
Once again I've read the unlinkers through, and only thing that could
cause this is tcp_send_synack (others do deal with the hints) but I think
Eric already proposed a patch to that but we never got anywhere due to
some counterargument why it wouldn't take place (too far away for me to
remember, see archives about the discussions). ...But if you want be dead
sure some WARN_ON there might not hurt. Also the purging of the whole
queue was a similar suspect I then came across (but that would only
materialize with sk reuse happening e.g., with nfs which the other guys
weren't using).
> 3. The hint is pointing to a skb on the list but the list itself is
> corrupt.
>
> I added some debug code and the crash is happening when
> tp->retransmit_skb_hint is not NULL but tp->retransmit_skb_hint->next
> is NULL. So, #1 is out; unfortunately, I didn't have debug code in
> place to discern between #2 and #3.
>
> Does anything ring a bell? This is a production system and debugging
> affects quite a number of people. I can put debug code in to discern
> between #2 and #3 but I'm basically shooting in the dark and it would
> be great if someone has a better idea.
Thanks for taking this up. I've been kind of waiting somebody to show up
who actually has some way of reproducing it. Once I had one guy in the
hook but his ability to reproduce was for some reason lost when he tried
with a debug patch [1].
I now realize that the debug patch should probably also print the write
queue too when the problem is caught in order to discern the cases you
mention.
Something along these lines:
tcp_for_write_queue(skb, sk) {
printk("skb %p (%u-%u) next %p prev %p sacked %u\n", ...);
}
Anyway, my debugging patch should be such that in a lucky case it avoids
crashing the system too, though price to pay might then be a stuck
connection. In case #3 I'd expect the box to die elsewhere in TCP code
pretty soon anyway so it depends whether avoiding oops is really so
useful, but if you're lucky other mechanism in TCP will recover
the lost one for you (basically RTO driven retransmission).
--
i.
[1] http://marc.info/?l=linux-kernel&m=126624014117610&w=2
^ permalink raw reply
* [PATCHv3] extensions: libxt_CHECKSUM extension
From: Michael S. Tsirkin @ 2010-07-11 15:14 UTC (permalink / raw)
To: Patrick McHardy, David S. Miller, Jan Engelhardt, Randy Dunlap,
netfilter-devel
In-Reply-To: <20100711150623.GA7420@redhat.com>
This adds a `CHECKSUM' target, which can be used in the iptables mangle
table.
You can use this target to compute and fill in the checksum in
a packet that lacks a checksum. This is particularly useful,
if you need to work around old applications such as dhcp clients,
that do not work well with checksum offloads, but don't want to disable
checksum offload in your device.
The problem happens in the field with virtualized applications.
For reference, see Red Hat bz 605555, as well as
http://www.spinics.net/lists/kvm/msg37660.html
Typical expected use (helps old dhclient binary running in a VM):
iptables -A POSTROUTING -t mangle -p udp --dport bootpc \
-j CHECKSUM --checksum-fill
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
---
Correction in the documentation. Sorry about the noise.
Changes from v2:
updated man file
Changes from v1:
switched from ipt to xt
extensions/libxt_CHECKSUM.c | 99 +++++++++++++++++++++++++++++++++++++++++
extensions/libxt_CHECKSUM.man | 8 +++
2 files changed, 107 insertions(+), 0 deletions(-)
create mode 100644 extensions/libxt_CHECKSUM.c
create mode 100644 extensions/libxt_CHECKSUM.man
diff --git a/extensions/libxt_CHECKSUM.c b/extensions/libxt_CHECKSUM.c
new file mode 100644
index 0000000..00fbd8f
--- /dev/null
+++ b/extensions/libxt_CHECKSUM.c
@@ -0,0 +1,99 @@
+/* Shared library add-on to xtables for CHECKSUM
+ *
+ * (C) 2002 by Harald Welte <laforge@gnumonks.org>
+ * (C) 2010 by Red Hat, Inc
+ * Author: Michael S. Tsirkin <mst@redhat.com>
+ *
+ * This program is distributed under the terms of GNU GPL v2, 1991
+ *
+ * libxt_CHECKSUM.c borrowed some bits from libipt_ECN.c
+ *
+ * $Id$
+ */
+#include <stdio.h>
+#include <string.h>
+#include <stdlib.h>
+#include <getopt.h>
+
+#include <xtables.h>
+#include <linux/netfilter/xt_CHECKSUM.h>
+
+static void CHECKSUM_help(void)
+{
+ printf(
+"CHECKSUM target options\n"
+" --checksum-fill Fill in packet checksum.\n");
+}
+
+static const struct option CHECKSUM_opts[] = {
+ { "checksum-fill", 0, NULL, 'F' },
+ { .name = NULL }
+};
+
+static int CHECKSUM_parse(int c, char **argv, int invert, unsigned int *flags,
+ const void *entry, struct xt_entry_target **target)
+{
+ struct xt_CHECKSUM_info *einfo
+ = (struct xt_CHECKSUM_info *)(*target)->data;
+
+ switch (c) {
+ case 'F':
+ if (*flags)
+ xtables_error(PARAMETER_PROBLEM,
+ "CHECKSUM target: Only use --checksum-fill ONCE!");
+ einfo->operation = XT_CHECKSUM_OP_FILL;
+ *flags |= XT_CHECKSUM_OP_FILL;
+ break;
+ default:
+ return 0;
+ }
+
+ return 1;
+}
+
+static void CHECKSUM_check(unsigned int flags)
+{
+ if (!flags)
+ xtables_error(PARAMETER_PROBLEM,
+ "CHECKSUM target: Parameter --checksum-fill is required");
+}
+
+static void CHECKSUM_print(const void *ip, const struct xt_entry_target *target,
+ int numeric)
+{
+ const struct xt_CHECKSUM_info *einfo =
+ (const struct xt_CHECKSUM_info *)target->data;
+
+ printf("CHECKSUM ");
+
+ if (einfo->operation & XT_CHECKSUM_OP_FILL)
+ printf("fill ");
+}
+
+static void CHECKSUM_save(const void *ip, const struct xt_entry_target *target)
+{
+ const struct xt_CHECKSUM_info *einfo =
+ (const struct xt_CHECKSUM_info *)target->data;
+
+ if (einfo->operation & XT_CHECKSUM_OP_FILL)
+ printf("--checksum-fill ");
+}
+
+static struct xtables_target checksum_tg_reg = {
+ .name = "CHECKSUM",
+ .version = XTABLES_VERSION,
+ .family = NFPROTO_UNSPEC,
+ .size = XT_ALIGN(sizeof(struct xt_CHECKSUM_info)),
+ .userspacesize = XT_ALIGN(sizeof(struct xt_CHECKSUM_info)),
+ .help = CHECKSUM_help,
+ .parse = CHECKSUM_parse,
+ .final_check = CHECKSUM_check,
+ .print = CHECKSUM_print,
+ .save = CHECKSUM_save,
+ .extra_opts = CHECKSUM_opts,
+};
+
+void _init(void)
+{
+ xtables_register_target(&checksum_tg_reg);
+}
diff --git a/extensions/libxt_CHECKSUM.man b/extensions/libxt_CHECKSUM.man
new file mode 100644
index 0000000..92ae700
--- /dev/null
+++ b/extensions/libxt_CHECKSUM.man
@@ -0,0 +1,8 @@
+This target allows to selectively work around broken/old applications.
+It can only be used in the mangle table.
+.TP
+\fB\-\-checksum\-fill\fP
+Compute and fill in the checksum in a packet that lacks a checksum.
+This is particularly useful, if you need to work around old applications
+such as dhcp clients, that do not work well with checksum offloads,
+but don't want to disable checksum offload in your device.
--
1.7.2.rc0.14.g41c1c
^ permalink raw reply related
* [PATCHv2] extensions: libxt_CHECKSUM extension
From: Michael S. Tsirkin @ 2010-07-11 15:08 UTC (permalink / raw)
To: Patrick McHardy, David S. Miller, Jan Engelhardt, Randy Dunlap,
netfilter-devel
In-Reply-To: <20100711150623.GA7420@redhat.com>
This adds a `CHECKSUM' target, which can be used in the iptables mangle
table.
You can use this target to compute and fill in the checksum in
a packet that lacks a checksum. This is particularly useful,
if you need to work around old applications such as dhcp clients,
that do not work well with checksum offloads, but don't want to disable
checksum offload in your device.
The problem happens in the field with virtualized applications.
For reference, see Red Hat bz 605555, as well as
http://www.spinics.net/lists/kvm/msg37660.html
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
---
Changes from v1:
switched from ipt to xt target
extensions/libipt_CHECKSUM.man | 8 +++
extensions/libxt_CHECKSUM.c | 99 ++++++++++++++++++++++++++++++++++++++++
2 files changed, 107 insertions(+), 0 deletions(-)
create mode 100644 extensions/libipt_CHECKSUM.man
create mode 100644 extensions/libxt_CHECKSUM.c
diff --git a/extensions/libipt_CHECKSUM.man b/extensions/libipt_CHECKSUM.man
new file mode 100644
index 0000000..4f83335
--- /dev/null
+++ b/extensions/libipt_CHECKSUM.man
@@ -0,0 +1,8 @@
+This target allows to selectively work around broken/old applications.
+It can only be used in the mangle table.
+.TP
+\fB\-\-checksum\-fill\fP
+Compute and fill in the checksum in a packet that lacks a checksum.
+This is particularly useful, if you need to work around old applications
+such as dhcp clients, that do not work well with checksum offloads,
+but don't want to disable checksum offload in your device.
diff --git a/extensions/libxt_CHECKSUM.c b/extensions/libxt_CHECKSUM.c
new file mode 100644
index 0000000..00fbd8f
--- /dev/null
+++ b/extensions/libxt_CHECKSUM.c
@@ -0,0 +1,99 @@
+/* Shared library add-on to xtables for CHECKSUM
+ *
+ * (C) 2002 by Harald Welte <laforge@gnumonks.org>
+ * (C) 2010 by Red Hat, Inc
+ * Author: Michael S. Tsirkin <mst@redhat.com>
+ *
+ * This program is distributed under the terms of GNU GPL v2, 1991
+ *
+ * libxt_CHECKSUM.c borrowed some bits from libipt_ECN.c
+ *
+ * $Id$
+ */
+#include <stdio.h>
+#include <string.h>
+#include <stdlib.h>
+#include <getopt.h>
+
+#include <xtables.h>
+#include <linux/netfilter/xt_CHECKSUM.h>
+
+static void CHECKSUM_help(void)
+{
+ printf(
+"CHECKSUM target options\n"
+" --checksum-fill Fill in packet checksum.\n");
+}
+
+static const struct option CHECKSUM_opts[] = {
+ { "checksum-fill", 0, NULL, 'F' },
+ { .name = NULL }
+};
+
+static int CHECKSUM_parse(int c, char **argv, int invert, unsigned int *flags,
+ const void *entry, struct xt_entry_target **target)
+{
+ struct xt_CHECKSUM_info *einfo
+ = (struct xt_CHECKSUM_info *)(*target)->data;
+
+ switch (c) {
+ case 'F':
+ if (*flags)
+ xtables_error(PARAMETER_PROBLEM,
+ "CHECKSUM target: Only use --checksum-fill ONCE!");
+ einfo->operation = XT_CHECKSUM_OP_FILL;
+ *flags |= XT_CHECKSUM_OP_FILL;
+ break;
+ default:
+ return 0;
+ }
+
+ return 1;
+}
+
+static void CHECKSUM_check(unsigned int flags)
+{
+ if (!flags)
+ xtables_error(PARAMETER_PROBLEM,
+ "CHECKSUM target: Parameter --checksum-fill is required");
+}
+
+static void CHECKSUM_print(const void *ip, const struct xt_entry_target *target,
+ int numeric)
+{
+ const struct xt_CHECKSUM_info *einfo =
+ (const struct xt_CHECKSUM_info *)target->data;
+
+ printf("CHECKSUM ");
+
+ if (einfo->operation & XT_CHECKSUM_OP_FILL)
+ printf("fill ");
+}
+
+static void CHECKSUM_save(const void *ip, const struct xt_entry_target *target)
+{
+ const struct xt_CHECKSUM_info *einfo =
+ (const struct xt_CHECKSUM_info *)target->data;
+
+ if (einfo->operation & XT_CHECKSUM_OP_FILL)
+ printf("--checksum-fill ");
+}
+
+static struct xtables_target checksum_tg_reg = {
+ .name = "CHECKSUM",
+ .version = XTABLES_VERSION,
+ .family = NFPROTO_UNSPEC,
+ .size = XT_ALIGN(sizeof(struct xt_CHECKSUM_info)),
+ .userspacesize = XT_ALIGN(sizeof(struct xt_CHECKSUM_info)),
+ .help = CHECKSUM_help,
+ .parse = CHECKSUM_parse,
+ .final_check = CHECKSUM_check,
+ .print = CHECKSUM_print,
+ .save = CHECKSUM_save,
+ .extra_opts = CHECKSUM_opts,
+};
+
+void _init(void)
+{
+ xtables_register_target(&checksum_tg_reg);
+}
^ permalink raw reply related
* [PATCHv2] netfilter: add CHECKSUM target
From: Michael S. Tsirkin @ 2010-07-11 15:06 UTC (permalink / raw)
To: Patrick McHardy, Michael S. Tsirkin, David S. Miller,
Jan Engelhardt, Randy Dunlap
This adds a `CHECKSUM' target, which can be used in the iptables mangle
table.
You can use this target to compute and fill in the checksum in
a packet that lacks a checksum. This is particularly useful,
if you need to work around old applications such as dhcp clients,
that do not work well with checksum offloads, but don't want to
disable checksum offload in your device.
The problem happens in the field with virtualized applications.
For reference, see Red Hat bz 605555, as well as
http://www.spinics.net/lists/kvm/msg37660.html
Typical expected use (helps old dhclient binary running in a VM):
iptables -A POSTROUTING -t mangle -p udp --dport bootpc \
-j CHECKSUM --checksum-fill
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
---
Changes from v1:
moved from ipt to xt
get rid of any ipv4 dependencies
coding style tweaks
include/linux/netfilter/xt_CHECKSUM.h | 18 ++++++++
net/netfilter/Kconfig | 17 +++++++-
net/netfilter/Makefile | 1 +
net/netfilter/xt_CHECKSUM.c | 72 +++++++++++++++++++++++++++++++++
4 files changed, 107 insertions(+), 1 deletions(-)
create mode 100644 include/linux/netfilter/xt_CHECKSUM.h
create mode 100644 net/netfilter/xt_CHECKSUM.c
diff --git a/include/linux/netfilter/xt_CHECKSUM.h b/include/linux/netfilter/xt_CHECKSUM.h
new file mode 100644
index 0000000..56afe57
--- /dev/null
+++ b/include/linux/netfilter/xt_CHECKSUM.h
@@ -0,0 +1,18 @@
+/* Header file for iptables ipt_CHECKSUM target
+ *
+ * (C) 2002 by Harald Welte <laforge@gnumonks.org>
+ * (C) 2010 Red Hat Inc
+ * Author: Michael S. Tsirkin <mst@redhat.com>
+ *
+ * This software is distributed under GNU GPL v2, 1991
+*/
+#ifndef _IPT_CHECKSUM_TARGET_H
+#define _IPT_CHECKSUM_TARGET_H
+
+#define XT_CHECKSUM_OP_FILL 0x01 /* fill in checksum in IP header */
+
+struct xt_CHECKSUM_info {
+ u_int8_t operation; /* bitset of operations */
+};
+
+#endif /* _IPT_CHECKSUM_TARGET_H */
diff --git a/net/netfilter/Kconfig b/net/netfilter/Kconfig
index 8593a77..1cf4852 100644
--- a/net/netfilter/Kconfig
+++ b/net/netfilter/Kconfig
@@ -294,7 +294,7 @@ endif # NF_CONNTRACK
config NETFILTER_TPROXY
tristate "Transparent proxying support (EXPERIMENTAL)"
depends on EXPERIMENTAL
- depends on IP_NF_MANGLE
+ depends on IP_NF_MANGLE || IP6_NF_MANGLE
depends on NETFILTER_ADVANCED
help
This option enables transparent proxying support, that is,
@@ -347,6 +347,21 @@ config NETFILTER_XT_CONNMARK
comment "Xtables targets"
+config NETFILTER_XT_TARGET_CHECKSUM
+ tristate "CHECKSUM target support"
+ depends on NETFILTER_ADVANCED
+ ---help---
+ This option adds a `CHECKSUM' target, which can be used in the iptables mangle
+ table.
+
+ You can use this target to compute and fill in the checksum in
+ a packet that lacks a checksum. This is particularly useful,
+ if you need to work around old applications such as dhcp clients,
+ that do not work well with checksum offloads, but don't want to disable
+ checksum offload in your device.
+
+ To compile it as a module, choose M here. If unsure, say N.
+
config NETFILTER_XT_TARGET_CLASSIFY
tristate '"CLASSIFY" target support'
depends on NETFILTER_ADVANCED
diff --git a/net/netfilter/Makefile b/net/netfilter/Makefile
index 14e3a8f..8eb541d 100644
--- a/net/netfilter/Makefile
+++ b/net/netfilter/Makefile
@@ -45,6 +45,7 @@ obj-$(CONFIG_NETFILTER_XT_MARK) += xt_mark.o
obj-$(CONFIG_NETFILTER_XT_CONNMARK) += xt_connmark.o
# targets
+obj-$(CONFIG_NETFILTER_XT_TARGET_CHECKSUM) += xt_CHECKSUM.o
obj-$(CONFIG_NETFILTER_XT_TARGET_CLASSIFY) += xt_CLASSIFY.o
obj-$(CONFIG_NETFILTER_XT_TARGET_CONNSECMARK) += xt_CONNSECMARK.o
obj-$(CONFIG_NETFILTER_XT_TARGET_CT) += xt_CT.o
diff --git a/net/netfilter/xt_CHECKSUM.c b/net/netfilter/xt_CHECKSUM.c
new file mode 100644
index 0000000..0fee1a7
--- /dev/null
+++ b/net/netfilter/xt_CHECKSUM.c
@@ -0,0 +1,72 @@
+/* iptables module for the packet checksum mangling
+ *
+ * (C) 2002 by Harald Welte <laforge@netfilter.org>
+ * (C) 2010 Red Hat, Inc.
+ *
+ * Author: Michael S. Tsirkin <mst@redhat.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+*/
+#define pr_fmt(fmt) KBUILD_MODNAME ": " fmt
+#include <linux/in.h>
+#include <linux/module.h>
+#include <linux/skbuff.h>
+#include <linux/netdevice.h>
+
+#include <linux/netfilter/x_tables.h>
+#include <linux/netfilter/xt_CHECKSUM.h>
+
+MODULE_LICENSE("GPL");
+MODULE_AUTHOR("Michael S. Tsirkin <mst@redhat.com>");
+MODULE_DESCRIPTION("Xtables: checksum modification");
+MODULE_ALIAS("ipt_CHECKSUM");
+MODULE_ALIAS("ip6t_CHECKSUM");
+
+static unsigned int
+checksum_tg(struct sk_buff *skb, const struct xt_action_param *par)
+{
+ if (skb->ip_summed == CHECKSUM_PARTIAL)
+ skb_checksum_help(skb);
+
+ return XT_CONTINUE;
+}
+
+static int checksum_tg_check(const struct xt_tgchk_param *par)
+{
+ const struct xt_CHECKSUM_info *einfo = par->targinfo;
+
+ if (einfo->operation & ~XT_CHECKSUM_OP_FILL) {
+ pr_info("unsupported CHECKSUM operation %x\n", einfo->operation);
+ return -EINVAL;
+ }
+ if (!einfo->operation) {
+ pr_info("no CHECKSUM operation enabled\n");
+ return -EINVAL;
+ }
+ return 0;
+}
+
+static struct xt_target checksum_tg_reg __read_mostly = {
+ .name = "CHECKSUM",
+ .family = NFPROTO_UNSPEC,
+ .target = checksum_tg,
+ .targetsize = sizeof(struct xt_CHECKSUM_info),
+ .table = "mangle",
+ .checkentry = checksum_tg_check,
+ .me = THIS_MODULE,
+};
+
+static int __init checksum_tg_init(void)
+{
+ return xt_register_target(&checksum_tg_reg);
+}
+
+static void __exit checksum_tg_exit(void)
+{
+ xt_unregister_target(&checksum_tg_reg);
+}
+
+module_init(checksum_tg_init);
+module_exit(checksum_tg_exit);
--
1.7.2.rc0.14.g41c1c
^ permalink raw reply related
* Re: [PATCH] netfilter: add CHECKSUM target
From: Michael S. Tsirkin @ 2010-07-11 13:19 UTC (permalink / raw)
To: Changli Gao
Cc: Jan Engelhardt, Patrick McHardy, David S. Miller,
Alexey Kuznetsov, Pekka Savola (ipv6), James Morris,
Hideaki YOSHIFUJI, linux-kernel, netfilter-devel, netfilter,
coreteam, netdev, herbert.xu, kvm
In-Reply-To: <AANLkTikamhUUuoQ8EjwPn9gB4zHZkeDuzpiqd4XcN3lx@mail.gmail.com>
On Sun, Jul 11, 2010 at 08:45:01PM +0800, Changli Gao wrote:
> On Sun, Jul 11, 2010 at 8:27 PM, Michael S. Tsirkin <mst@redhat.com> wrote:
> > On Fri, Jul 09, 2010 at 11:56:26AM +0200, Jan Engelhardt wrote:
> >>
> >> On Friday 2010-07-09 00:29, Michael S. Tsirkin wrote:
> >> >
> >> > include/linux/netfilter_ipv4/ipt_CHECKSUM.h | 18 +++++++
> >> > net/ipv4/netfilter/Kconfig | 16 ++++++
> >> > net/ipv4/netfilter/Makefile | 1 +
> >> > net/ipv4/netfilter/ipt_CHECKSUM.c | 72 +++++++++++++++++++++++++++
> >>
> >> New modules should use xt.
> >
> > I tried moving the module to xt_CHECKSUM but now
> > iptables -A POSTROUTING -t mangle -p udp --dport 68 -j CHECKSUM --checksum-fill
> > does not to load the module.
> > It seems that xt_request_find_target in x_tables uses the prefix 'ip' for a module.
> > What am I missing?
> >
>
> You should add MODULE_ALIAS clause, i.e.
>
> MODULE_ALIAS("ipt_CHECKSUM");
This worked, thanks!
> --
> Regards,
> Changli Gao(xiaosuo@gmail.com)
^ permalink raw reply
* Re: [PATCH] netfilter: add CHECKSUM target
From: Michael S. Tsirkin @ 2010-07-11 13:19 UTC (permalink / raw)
To: Jan Engelhardt
Cc: Patrick McHardy, David S. Miller, Alexey Kuznetsov,
Pekka Savola (ipv6), James Morris, Hideaki YOSHIFUJI,
linux-kernel, netfilter-devel, netfilter, coreteam, netdev,
herbert.xu, kvm
In-Reply-To: <alpine.LSU.2.01.1007111445320.6015@obet.zrqbmnf.qr>
On Sun, Jul 11, 2010 at 02:45:49PM +0200, Jan Engelhardt wrote:
>
> On Sunday 2010-07-11 14:27, Michael S. Tsirkin wrote:
> >> >
> >> > include/linux/netfilter_ipv4/ipt_CHECKSUM.h | 18 +++++++
> >> > net/ipv4/netfilter/Kconfig | 16 ++++++
> >> > net/ipv4/netfilter/Makefile | 1 +
> >> > net/ipv4/netfilter/ipt_CHECKSUM.c | 72 +++++++++++++++++++++++++++
> >>
> >> New modules should use xt.
> >
> >I tried moving the module to xt_CHECKSUM but now
> > iptables -A POSTROUTING -t mangle -p udp --dport 68 -j CHECKSUM --checksum-fill
> >does not to load the module.
> >It seems that xt_request_find_target in x_tables uses the prefix 'ip' for a module.
> >What am I missing?
>
> MODULE_ALIAS like the others have.
Worked for me, thanks!
^ permalink raw reply
* RE: [PATCH] bnx2x: add support for receive hashing
From: Vladislav Zolotarov @ 2010-07-11 13:16 UTC (permalink / raw)
To: Vladislav Zolotarov, David Miller, therbert@google.com
Cc: netdev@vger.kernel.org
In-Reply-To: <8628FE4E7912BF47A96AE7DD7BAC0AADDDE64704DC@SJEXCHCCR02.corp.ad.broadcom.com>
Dave, it's obvious that there a demand for a new HW/FW configuration from our side - "rx hash enable" which is currently tightly coupled with the RSS capability. As long as RSS flow in our FW includes a few more things apart from just creating an RSS hash and as long as there are flows (even hypothetical) that demand the RSS hash regardless the RSS itself we started to work on separation of these two features from FW perspective. This of course will demand a new FW version but once we have it we'll be able to be more specific in HW configuration and have a cleaner code.
Technically, our FW may provide the Rx HASH always and in a current driver configuration this is what it does.
I wonder if the driver always provides the HW RX HASH in the skb->rxhash regardless the value of NETIF_F_RXHASH bit in a netdev->features will it cause any harm? If not we can get rid of two extra conditionals in the bnx2x_rx_int().
Thanks,
vlad
> -----Original Message-----
> From: netdev-owner@vger.kernel.org [mailto:netdev-owner@vger.kernel.org] On
> Behalf Of Vladislav Zolotarov
> Sent: Sunday, July 11, 2010 1:02 PM
> To: David Miller; therbert@google.com
> Cc: netdev@vger.kernel.org
> Subject: RE: [PATCH] bnx2x: add support for receive hashing
>
>
>
> > -----Original Message-----
> > From: David Miller [mailto:davem@davemloft.net]
> > Sent: Sunday, July 11, 2010 5:13 AM
> > To: therbert@google.com
> > Cc: Vladislav Zolotarov; netdev@vger.kernel.org
> > Subject: Re: [PATCH] bnx2x: add support for receive hashing
> >
> > From: Tom Herbert <therbert@google.com>
> > Date: Wed, 7 Jul 2010 12:17:31 -0700
> >
> > > It is to enable the device to provide the RSS hash in RX descriptor.
> > > The hash severs two purposes now, it's used internally in the device
> > > to perform RSS table lookup and also value in RX descriptor. The
> > > latter does not require multi-queue. Strictly speaking, on a single
> > > processor system without multqueue, it would be true that enabling the
> > > RX hash on bnx2x is currently superfluous (notwithstanding some other
> > > use of the hash might be implemented).
> >
> > We intend to use the card provided RSS hash to optimize GSO
> > flow comparisons at some point.
>
> Could u explain what did u mean here, pls.? GSO is a Tx flow while RSS hash
> is generated for the ingress traffic.
>
> >
> > There are other possible uses as well.
>
> Could u elaborate, pls?
>
> >
> > Therefore even in a single RX queue configuration, the driver
> > should provide the hash if it can.
>
> Providing an RSS hash adds an additional work to our FW and HW. We would like
> to understand why do we need to pay this penalty. And surely we don't want to
> pay it for a possible use. We'd prefer to pay it when it's really needed.
>
> Thanks,
> vlad
>
>
> --
> To unsubscribe from this list: send the line "unsubscribe netdev" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply
* Re: Splice status
From: Changli Gao @ 2010-07-11 13:08 UTC (permalink / raw)
To: Eric Dumazet; +Cc: Jens Axboe, Ofer Heifetz, netdev@vger.kernel.org
In-Reply-To: <1278388580.2466.305.camel@edumazet-laptop>
On Tue, Jul 6, 2010 at 11:56 AM, Eric Dumazet <eric.dumazet@gmail.com> wrote:
> Le mardi 06 juillet 2010 à 10:01 +0800, Changli Gao a écrit :
>>
>> If we don't drain the pipe before calling splice(2), the data spliced
>> from pipe maybe not be what we expect. Then data corruption occurs.
>>
>
> This is not true. A pipe is a pipe is a buffer. You dont need it to be
> empty when using it. Nowhere in documentation its stated.
Do you mean splice(2) empties the pipe buffer before using it as an
output buffer? If not, the pipe draining is needed to avoid data
corruption.
>
> However, a single skb can fill a pipe, even if "its empty"
>
Yea. Because tcp_splice_read() doesn't know if the __tcp_splice_read
returns due to pipe fulling.
>
>> >
>> > splice(sock, pipe) can block if caller dont use appropriate "non
>> > blocking pipe' splice() mode, even if pipe is empty before a splice()
>> > call.
>>
>> I don't think it is expected. The code of sys_recvfile is much like
>> the sendfile(2) implementation in kernel. If sys_recvfile may block
>> without non_block flag, sendfile(2) may block too.
>
> Then it would be a bug. You might fix it easily.
It seems reasonable. I'll fix it.
>
> Using splice() correctly (ie, not blocking on sock->pipe) should work
> too.
>
> Again, you can block on splice(sock, pipe), iff you have a second thread
> doing the opposite (pipe->file) in parallel to unblock you. But samba
> recvfile algo is using a single thread.
>
--
Regards,
Changli Gao(xiaosuo@gmail.com)
^ permalink raw reply
* Re: [PATCH] netfilter: add CHECKSUM target
From: Jan Engelhardt @ 2010-07-11 12:45 UTC (permalink / raw)
To: Michael S. Tsirkin
Cc: Patrick McHardy, David S. Miller, Alexey Kuznetsov,
Pekka Savola (ipv6), James Morris, Hideaki YOSHIFUJI,
linux-kernel, netfilter-devel, netfilter, coreteam, netdev,
herbert.xu, kvm
In-Reply-To: <20100711122708.GA3573@redhat.com>
On Sunday 2010-07-11 14:27, Michael S. Tsirkin wrote:
>> >
>> > include/linux/netfilter_ipv4/ipt_CHECKSUM.h | 18 +++++++
>> > net/ipv4/netfilter/Kconfig | 16 ++++++
>> > net/ipv4/netfilter/Makefile | 1 +
>> > net/ipv4/netfilter/ipt_CHECKSUM.c | 72 +++++++++++++++++++++++++++
>>
>> New modules should use xt.
>
>I tried moving the module to xt_CHECKSUM but now
> iptables -A POSTROUTING -t mangle -p udp --dport 68 -j CHECKSUM --checksum-fill
>does not to load the module.
>It seems that xt_request_find_target in x_tables uses the prefix 'ip' for a module.
>What am I missing?
MODULE_ALIAS like the others have.
^ permalink raw reply
* Re: [PATCH] netfilter: add CHECKSUM target
From: Changli Gao @ 2010-07-11 12:45 UTC (permalink / raw)
To: Michael S. Tsirkin
Cc: Jan Engelhardt, Patrick McHardy, David S. Miller,
Alexey Kuznetsov, Pekka Savola (ipv6), James Morris,
Hideaki YOSHIFUJI, linux-kernel, netfilter-devel, netfilter,
coreteam, netdev, herbert.xu, kvm
In-Reply-To: <20100711122708.GA3573@redhat.com>
On Sun, Jul 11, 2010 at 8:27 PM, Michael S. Tsirkin <mst@redhat.com> wrote:
> On Fri, Jul 09, 2010 at 11:56:26AM +0200, Jan Engelhardt wrote:
>>
>> On Friday 2010-07-09 00:29, Michael S. Tsirkin wrote:
>> >
>> > include/linux/netfilter_ipv4/ipt_CHECKSUM.h | 18 +++++++
>> > net/ipv4/netfilter/Kconfig | 16 ++++++
>> > net/ipv4/netfilter/Makefile | 1 +
>> > net/ipv4/netfilter/ipt_CHECKSUM.c | 72 +++++++++++++++++++++++++++
>>
>> New modules should use xt.
>
> I tried moving the module to xt_CHECKSUM but now
> iptables -A POSTROUTING -t mangle -p udp --dport 68 -j CHECKSUM --checksum-fill
> does not to load the module.
> It seems that xt_request_find_target in x_tables uses the prefix 'ip' for a module.
> What am I missing?
>
You should add MODULE_ALIAS clause, i.e.
MODULE_ALIAS("ipt_CHECKSUM");
--
Regards,
Changli Gao(xiaosuo@gmail.com)
^ permalink raw reply
* Re: [PATCH] netfilter: add CHECKSUM target
From: Michael S. Tsirkin @ 2010-07-11 12:27 UTC (permalink / raw)
To: Jan Engelhardt
Cc: Patrick McHardy, David S. Miller, Alexey Kuznetsov,
Pekka Savola (ipv6), James Morris, Hideaki YOSHIFUJI,
linux-kernel, netfilter-devel, netfilter, coreteam, netdev,
herbert.xu, kvm
In-Reply-To: <alpine.LSU.2.01.1007091154120.7235@obet.zrqbmnf.qr>
On Fri, Jul 09, 2010 at 11:56:26AM +0200, Jan Engelhardt wrote:
>
> On Friday 2010-07-09 00:29, Michael S. Tsirkin wrote:
> >
> > include/linux/netfilter_ipv4/ipt_CHECKSUM.h | 18 +++++++
> > net/ipv4/netfilter/Kconfig | 16 ++++++
> > net/ipv4/netfilter/Makefile | 1 +
> > net/ipv4/netfilter/ipt_CHECKSUM.c | 72 +++++++++++++++++++++++++++
>
> New modules should use xt.
I tried moving the module to xt_CHECKSUM but now
iptables -A POSTROUTING -t mangle -p udp --dport 68 -j CHECKSUM --checksum-fill
does not to load the module.
It seems that xt_request_find_target in x_tables uses the prefix 'ip' for a module.
What am I missing?
> >+static unsigned int
> >+checksum_tg(struct sk_buff *skb, const struct xt_action_param *par)
> >+{
> >+ if (skb->ip_summed == CHECKSUM_PARTIAL) {
> >+ skb_checksum_help(skb);
> >+ }
>
> - {}
^ permalink raw reply
* Re: [PATCH] netfilter: add CHECKSUM target
From: Michael S. Tsirkin @ 2010-07-11 10:47 UTC (permalink / raw)
To: Patrick McHardy
Cc: David S. Miller, Alexey Kuznetsov, Pekka Savola (ipv6),
James Morris, Hideaki YOSHIFUJI, linux-kernel, netfilter-devel,
netfilter, coreteam, netdev, herbert.xu, kvm
In-Reply-To: <4C373D90.8070000@trash.net>
On Fri, Jul 09, 2010 at 05:17:36PM +0200, Patrick McHardy wrote:
> Am 09.07.2010 00:29, schrieb Michael S. Tsirkin:
> > This adds a `CHECKSUM' target, which can be used in the iptables mangle
> > table.
> >
> > You can use this target to compute and fill in the checksum in
> > an IP packet that lacks a checksum. This is particularly useful,
> > if you need to work around old applications such as dhcp clients,
> > that do not work well with checksum offloads, but don't want to
> > disable checksum offload in your device.
> >
> > The problem happens in the field with virtualized applications.
> > For reference, see Red Hat bz 605555, as well as
> > http://www.spinics.net/lists/kvm/msg37660.html
> >
> > Typical expected use (helps old dhclient binary running in a VM):
> > iptables -A POSTROUTING -t mangle -p udp --dport 68 -j CHECKSUM
> > --checksum-fill
>
> I'm not sure this is something we want to merge upstream and
> support indefinitely. Dave suggested this as a temporary
> out-of-tree workaround until the majority of guest dhcp clients
> are fixed. Has anything changed that makes this course of
> action impractical?
If I understand what Dave said correctly, it's up to you ...
The arguments for putting this upstream are:
Given the track record, I wouldn't hope for quick fix in the majority of
guest dhcp clients, unfortunately :(. We are talking years here.
Even after that, one of the uses of virtualization is
to keep old guests running. So yes, I think we'll
keep using work-arounds for this for a very long time.
Further, since we have to add the module and we have to teach management
to program it, it will be much less painful for everyone
involved if we can put the code upstream, rather than forking
management code.
--
MST
^ permalink raw reply
* Re: [PATCH] netfilter: add CHECKSUM target
From: Herbert Xu @ 2010-07-11 10:24 UTC (permalink / raw)
To: Michael S. Tsirkin
Cc: Patrick McHardy, David S. Miller, Alexey Kuznetsov,
Pekka Savola (ipv6), James Morris, Hideaki YOSHIFUJI,
linux-kernel, netfilter-devel, netfilter, coreteam, netdev, kvm
In-Reply-To: <20100711092117.GA16502@redhat.com>
On Sun, Jul 11, 2010 at 12:21:17PM +0300, Michael S. Tsirkin wrote:
>
> > I'd think that this target would be protocol-agnostic, no?
>
> Meaning it should go into net/netfilter/? Will do.
Right, and you should get rid of any IPv4 dependencies.
Thanks,
--
Email: Herbert Xu <herbert@gondor.apana.org.au>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt
^ permalink raw reply
* RE: [PATCH] bnx2x: add support for receive hashing
From: Vladislav Zolotarov @ 2010-07-11 10:02 UTC (permalink / raw)
To: David Miller, therbert@google.com; +Cc: netdev@vger.kernel.org
In-Reply-To: <20100710.191242.70196353.davem@davemloft.net>
> -----Original Message-----
> From: David Miller [mailto:davem@davemloft.net]
> Sent: Sunday, July 11, 2010 5:13 AM
> To: therbert@google.com
> Cc: Vladislav Zolotarov; netdev@vger.kernel.org
> Subject: Re: [PATCH] bnx2x: add support for receive hashing
>
> From: Tom Herbert <therbert@google.com>
> Date: Wed, 7 Jul 2010 12:17:31 -0700
>
> > It is to enable the device to provide the RSS hash in RX descriptor.
> > The hash severs two purposes now, it's used internally in the device
> > to perform RSS table lookup and also value in RX descriptor. The
> > latter does not require multi-queue. Strictly speaking, on a single
> > processor system without multqueue, it would be true that enabling the
> > RX hash on bnx2x is currently superfluous (notwithstanding some other
> > use of the hash might be implemented).
>
> We intend to use the card provided RSS hash to optimize GSO
> flow comparisons at some point.
Could u explain what did u mean here, pls.? GSO is a Tx flow while RSS hash is generated for the ingress traffic.
>
> There are other possible uses as well.
Could u elaborate, pls?
>
> Therefore even in a single RX queue configuration, the driver
> should provide the hash if it can.
Providing an RSS hash adds an additional work to our FW and HW. We would like to understand why do we need to pay this penalty. And surely we don't want to pay it for a possible use. We'd prefer to pay it when it's really needed.
Thanks,
vlad
^ permalink raw reply
* Re: [PATCH] netfilter: add CHECKSUM target
From: Michael S. Tsirkin @ 2010-07-11 9:21 UTC (permalink / raw)
To: Herbert Xu
Cc: Patrick McHardy, David S. Miller, Alexey Kuznetsov,
Pekka Savola (ipv6), James Morris, Hideaki YOSHIFUJI,
linux-kernel, netfilter-devel, netfilter, coreteam, netdev, kvm
In-Reply-To: <20100709071814.GA27964@gondor.apana.org.au>
On Fri, Jul 09, 2010 at 03:18:14PM +0800, Herbert Xu wrote:
> On Fri, Jul 09, 2010 at 01:29:13AM +0300, Michael S. Tsirkin wrote:
> > This adds a `CHECKSUM' target, which can be used in the iptables mangle
> > table.
> >
> > You can use this target to compute and fill in the checksum in
> > an IP packet that lacks a checksum. This is particularly useful,
> > if you need to work around old applications such as dhcp clients,
> > that do not work well with checksum offloads, but don't want to
> > disable checksum offload in your device.
> >
> > The problem happens in the field with virtualized applications.
> > For reference, see Red Hat bz 605555, as well as
> > http://www.spinics.net/lists/kvm/msg37660.html
> >
> > Typical expected use (helps old dhclient binary running in a VM):
> > iptables -A POSTROUTING -t mangle -p udp --dport 68 -j CHECKSUM
> > --checksum-fill
> >
> > Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
>
> I'd think that this target would be protocol-agnostic, no?
Meaning it should go into net/netfilter/? Will do.
> Cheers,
> --
> Email: Herbert Xu <herbert@gondor.apana.org.au>
> Home Page: http://gondor.apana.org.au/~herbert/
> PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt
^ permalink raw reply
* [patch v2.6 4/4] libxt_ipvs: user-space lib for netfilter matcher xt_ipvs
From: horms, Simon Horman @ 2010-07-11 9:03 UTC (permalink / raw)
To: lvs-devel, netdev, linux-kernel, netfilter, netfilter-devel
Cc: Malcolm Turnbull, Wensong Zhang, Julius Volz, Patrick McHardy,
David S. Miller, Hannes Eder
In-Reply-To: <20100711090342.035149543@vergenet.net>
[-- Attachment #1: libxt_ipvs-user-space-lib-for-netfilter-matcher-xt_ipvs.patch --]
[-- Type: text/plain, Size: 13930 bytes --]
From: Hannes Eder <heder@google.com>
The user-space library for the netfilter matcher xt_ipvs.
[ trivial up-port by Simon Horman <horms@verge.net.au> ]
Signed-off-by: Hannes Eder <heder@google.com>
Acked-by: Simon Horman <horms@verge.net.au>
configure.ac | 10 -
extensions/libxt_ipvs.c | 365 +++++++++++++++++++++++++++++++++++++
extensions/libxt_ipvs.man | 24 ++
include/linux/netfilter/xt_ipvs.h | 25 +++
4 files changed, 422 insertions(+), 2 deletions(-)
create mode 100644 extensions/libxt_ipvs.c
create mode 100644 extensions/libxt_ipvs.man
create mode 100644 include/linux/netfilter/xt_ipvs.h
v2.1, v2.3
Trivial up-port
v2.2
No change
Index: iptables/configure.ac
===================================================================
--- iptables.orig/configure.ac 2010-07-04 20:21:07.000000000 +0900
+++ iptables/configure.ac 2010-07-04 20:23:30.000000000 +0900
@@ -52,12 +52,18 @@ AC_ARG_WITH([pkgconfigdir], AS_HELP_STRI
[Path to the pkgconfig directory [[LIBDIR/pkgconfig]]]),
[pkgconfigdir="$withval"], [pkgconfigdir='${libdir}/pkgconfig'])
-AC_CHECK_HEADER([linux/dccp.h])
-
blacklist_modules="";
+
+AC_CHECK_HEADER([linux/dccp.h])
if test "$ac_cv_header_linux_dccp_h" != "yes"; then
blacklist_modules="$blacklist_modules dccp";
fi;
+
+AC_CHECK_HEADER([linux/ip_vs.h])
+if test "$ac_cv_header_linux_ip_vs_h" != "yes"; then
+ blacklist_modules="$blacklist_modules ipvs";
+fi;
+
AC_SUBST([blacklist_modules])
AM_CONDITIONAL([ENABLE_STATIC], [test "$enable_static" = "yes"])
Index: iptables/extensions/libxt_ipvs.c
===================================================================
--- /dev/null 1970-01-01 00:00:00.000000000 +0000
+++ iptables/extensions/libxt_ipvs.c 2010-07-04 20:24:01.000000000 +0900
@@ -0,0 +1,365 @@
+/*
+ * Shared library add-on to iptables to add IPVS matching.
+ *
+ * Detailed doc is in the kernel module source net/netfilter/xt_ipvs.c
+ *
+ * Author: Hannes Eder <heder@google.com>
+ */
+#include <sys/types.h>
+#include <assert.h>
+#include <ctype.h>
+#include <errno.h>
+#include <getopt.h>
+#include <netdb.h>
+#include <stdlib.h>
+#include <stdio.h>
+#include <string.h>
+#include <xtables.h>
+#include <linux/ip_vs.h>
+#include <linux/netfilter/xt_ipvs.h>
+
+static const struct option ipvs_mt_opts[] = {
+ { .name = "ipvs", .has_arg = false, .val = '0' },
+ { .name = "vproto", .has_arg = true, .val = '1' },
+ { .name = "vaddr", .has_arg = true, .val = '2' },
+ { .name = "vport", .has_arg = true, .val = '3' },
+ { .name = "vdir", .has_arg = true, .val = '4' },
+ { .name = "vmethod", .has_arg = true, .val = '5' },
+ { .name = "vportctl", .has_arg = true, .val = '6' },
+ { .name = NULL }
+};
+
+static void ipvs_mt_help(void)
+{
+ printf(
+"IPVS match options:\n"
+"[!] --ipvs packet belongs to an IPVS connection\n"
+"\n"
+"Any of the following options implies --ipvs (even negated)\n"
+"[!] --vproto protocol VIP protocol to match; by number or name,\n"
+" e.g. \"tcp\"\n"
+"[!] --vaddr address[/mask] VIP address to match\n"
+"[!] --vport port VIP port to match; by number or name,\n"
+" e.g. \"http\"\n"
+" --vdir {ORIGINAL|REPLY} flow direction of packet\n"
+"[!] --vmethod {GATE|IPIP|MASQ} IPVS forwarding method used\n"
+"[!] --vportctl port VIP port of the controlling connection to\n"
+" match, e.g. 21 for FTP\n"
+ );
+}
+
+static void ipvs_mt_parse_addr_and_mask(const char *arg,
+ union nf_inet_addr *address,
+ union nf_inet_addr *mask,
+ unsigned int family)
+{
+ struct in_addr *addr = NULL;
+ struct in6_addr *addr6 = NULL;
+ unsigned int naddrs = 0;
+
+ if (family == NFPROTO_IPV4) {
+ xtables_ipparse_any(arg, &addr, &mask->in, &naddrs);
+ if (naddrs > 1)
+ xtables_error(PARAMETER_PROBLEM,
+ "multiple IP addresses not allowed");
+ if (naddrs == 1)
+ memcpy(&address->in, addr, sizeof(*addr));
+ } else if (family == NFPROTO_IPV6) {
+ xtables_ip6parse_any(arg, &addr6, &mask->in6, &naddrs);
+ if (naddrs > 1)
+ xtables_error(PARAMETER_PROBLEM,
+ "multiple IP addresses not allowed");
+ if (naddrs == 1)
+ memcpy(&address->in6, addr6, sizeof(*addr6));
+ } else {
+ /* Hu? */
+ assert(false);
+ }
+}
+
+/* Function which parses command options; returns true if it ate an option */
+static int ipvs_mt_parse(int c, char **argv, int invert, unsigned int *flags,
+ const void *entry, struct xt_entry_match **match,
+ unsigned int family)
+{
+ struct xt_ipvs_mtinfo *data = (void *)(*match)->data;
+ char *p = NULL;
+ u_int8_t op = 0;
+
+ if ('0' <= c && c <= '6') {
+ static const int ops[] = {
+ XT_IPVS_IPVS_PROPERTY,
+ XT_IPVS_PROTO,
+ XT_IPVS_VADDR,
+ XT_IPVS_VPORT,
+ XT_IPVS_DIR,
+ XT_IPVS_METHOD,
+ XT_IPVS_VPORTCTL
+ };
+ op = ops[c - '0'];
+ } else
+ return 0;
+
+ if (*flags & op & XT_IPVS_ONCE_MASK)
+ goto multiple_use;
+
+ switch (c) {
+ case '0': /* --ipvs */
+ /* Nothing to do here. */
+ break;
+
+ case '1': /* --vproto */
+ /* Canonicalize into lower case */
+ for (p = optarg; *p != '\0'; ++p)
+ *p = tolower(*p);
+
+ data->l4proto = xtables_parse_protocol(optarg);
+ break;
+
+ case '2': /* --vaddr */
+ ipvs_mt_parse_addr_and_mask(optarg, &data->vaddr,
+ &data->vmask, family);
+ break;
+
+ case '3': /* --vport */
+ data->vport = htons(xtables_parse_port(optarg, "tcp"));
+ break;
+
+ case '4': /* --vdir */
+ xtables_param_act(XTF_NO_INVERT, "ipvs", "--vdir", invert);
+ if (strcasecmp(optarg, "ORIGINAL") == 0) {
+ data->bitmask |= XT_IPVS_DIR;
+ data->invert &= ~XT_IPVS_DIR;
+ } else if (strcasecmp(optarg, "REPLY") == 0) {
+ data->bitmask |= XT_IPVS_DIR;
+ data->invert |= XT_IPVS_DIR;
+ } else {
+ xtables_param_act(XTF_BAD_VALUE,
+ "ipvs", "--vdir", optarg);
+ }
+ break;
+
+ case '5': /* --vmethod */
+ if (strcasecmp(optarg, "GATE") == 0)
+ data->fwd_method = IP_VS_CONN_F_DROUTE;
+ else if (strcasecmp(optarg, "IPIP") == 0)
+ data->fwd_method = IP_VS_CONN_F_TUNNEL;
+ else if (strcasecmp(optarg, "MASQ") == 0)
+ data->fwd_method = IP_VS_CONN_F_MASQ;
+ else
+ xtables_param_act(XTF_BAD_VALUE,
+ "ipvs", "--vmethod", optarg);
+ break;
+
+ case '6': /* --vportctl */
+ data->vportctl = htons(xtables_parse_port(optarg, "tcp"));
+ break;
+
+ default:
+ /* Hu? How did we come here? */
+ assert(false);
+ return 0;
+ }
+
+ if (op & XT_IPVS_ONCE_MASK) {
+ if (data->invert & XT_IPVS_IPVS_PROPERTY)
+ xtables_error(PARAMETER_PROBLEM,
+ "! --ipvs cannot be together with"
+ " other options");
+ data->bitmask |= XT_IPVS_IPVS_PROPERTY;
+ }
+
+ data->bitmask |= op;
+ if (invert)
+ data->invert |= op;
+ *flags |= op;
+ return 1;
+
+multiple_use:
+ xtables_error(PARAMETER_PROBLEM,
+ "multiple use of the same IPVS option is not allowed");
+}
+
+static int ipvs_mt4_parse(int c, char **argv, int invert, unsigned int *flags,
+ const void *entry, struct xt_entry_match **match)
+{
+ return ipvs_mt_parse(c, argv, invert, flags, entry, match,
+ NFPROTO_IPV4);
+}
+
+static int ipvs_mt6_parse(int c, char **argv, int invert, unsigned int *flags,
+ const void *entry, struct xt_entry_match **match)
+{
+ return ipvs_mt_parse(c, argv, invert, flags, entry, match,
+ NFPROTO_IPV6);
+}
+
+static void ipvs_mt_check(unsigned int flags)
+{
+ if (flags == 0)
+ xtables_error(PARAMETER_PROBLEM,
+ "IPVS: At least one option is required");
+}
+
+/* Shamelessly copied from libxt_conntrack.c */
+static void ipvs_mt_dump_addr(const union nf_inet_addr *addr,
+ const union nf_inet_addr *mask,
+ unsigned int family, bool numeric)
+{
+ char buf[BUFSIZ];
+
+ if (family == NFPROTO_IPV4) {
+ if (!numeric && addr->ip == 0) {
+ printf("anywhere ");
+ return;
+ }
+ if (numeric)
+ strcpy(buf, xtables_ipaddr_to_numeric(&addr->in));
+ else
+ strcpy(buf, xtables_ipaddr_to_anyname(&addr->in));
+ strcat(buf, xtables_ipmask_to_numeric(&mask->in));
+ printf("%s ", buf);
+ } else if (family == NFPROTO_IPV6) {
+ if (!numeric && addr->ip6[0] == 0 && addr->ip6[1] == 0 &&
+ addr->ip6[2] == 0 && addr->ip6[3] == 0) {
+ printf("anywhere ");
+ return;
+ }
+ if (numeric)
+ strcpy(buf, xtables_ip6addr_to_numeric(&addr->in6));
+ else
+ strcpy(buf, xtables_ip6addr_to_anyname(&addr->in6));
+ strcat(buf, xtables_ip6mask_to_numeric(&mask->in6));
+ printf("%s ", buf);
+ }
+}
+
+static void ipvs_mt_dump(const void *ip, const struct xt_ipvs_mtinfo *data,
+ unsigned int family, bool numeric, const char *prefix)
+{
+ if (data->bitmask == XT_IPVS_IPVS_PROPERTY) {
+ if (data->invert & XT_IPVS_IPVS_PROPERTY)
+ printf("! ");
+ printf("%sipvs ", prefix);
+ }
+
+ if (data->bitmask & XT_IPVS_PROTO) {
+ if (data->invert & XT_IPVS_PROTO)
+ printf("! ");
+ printf("%sproto %u ", prefix, data->l4proto);
+ }
+
+ if (data->bitmask & XT_IPVS_VADDR) {
+ if (data->invert & XT_IPVS_VADDR)
+ printf("! ");
+
+ printf("%svaddr ", prefix);
+ ipvs_mt_dump_addr(&data->vaddr, &data->vmask, family, numeric);
+ }
+
+ if (data->bitmask & XT_IPVS_VPORT) {
+ if (data->invert & XT_IPVS_VPORT)
+ printf("! ");
+
+ printf("%svport %u ", prefix, ntohs(data->vport));
+ }
+
+ if (data->bitmask & XT_IPVS_DIR) {
+ if (data->invert & XT_IPVS_DIR)
+ printf("%svdir REPLY ", prefix);
+ else
+ printf("%svdir ORIGINAL ", prefix);
+ }
+
+ if (data->bitmask & XT_IPVS_METHOD) {
+ if (data->invert & XT_IPVS_METHOD)
+ printf("! ");
+
+ printf("%svmethod ", prefix);
+ switch (data->fwd_method) {
+ case IP_VS_CONN_F_DROUTE:
+ printf("GATE ");
+ break;
+ case IP_VS_CONN_F_TUNNEL:
+ printf("IPIP ");
+ break;
+ case IP_VS_CONN_F_MASQ:
+ printf("MASQ ");
+ break;
+ default:
+ /* Hu? */
+ printf("UNKNOWN ");
+ break;
+ }
+ }
+
+ if (data->bitmask & XT_IPVS_VPORTCTL) {
+ if (data->invert & XT_IPVS_VPORTCTL)
+ printf("! ");
+
+ printf("%svportctl %u ", prefix, ntohs(data->vportctl));
+ }
+}
+
+static void ipvs_mt4_print(const void *ip, const struct xt_entry_match *match,
+ int numeric)
+{
+ const struct xt_ipvs_mtinfo *data = (const void *)match->data;
+ ipvs_mt_dump(ip, data, NFPROTO_IPV4, numeric, "");
+}
+
+static void ipvs_mt6_print(const void *ip, const struct xt_entry_match *match,
+ int numeric)
+{
+ const struct xt_ipvs_mtinfo *data = (const void *)match->data;
+ ipvs_mt_dump(ip, data, NFPROTO_IPV6, numeric, "");
+}
+
+static void ipvs_mt4_save(const void *ip, const struct xt_entry_match *match)
+{
+ const struct xt_ipvs_mtinfo *data = (const void *)match->data;
+ ipvs_mt_dump(ip, data, NFPROTO_IPV4, true, "--");
+}
+
+static void ipvs_mt6_save(const void *ip, const struct xt_entry_match *match)
+{
+ const struct xt_ipvs_mtinfo *data = (const void *)match->data;
+ ipvs_mt_dump(ip, data, NFPROTO_IPV6, true, "--");
+}
+
+static struct xtables_match ipvs_matches_reg[] = {
+ {
+ .version = XTABLES_VERSION,
+ .name = "ipvs",
+ .revision = 0,
+ .family = NFPROTO_IPV4,
+ .size = XT_ALIGN(sizeof(struct xt_ipvs_mtinfo)),
+ .userspacesize = XT_ALIGN(sizeof(struct xt_ipvs_mtinfo)),
+ .help = ipvs_mt_help,
+ .parse = ipvs_mt4_parse,
+ .final_check = ipvs_mt_check,
+ .print = ipvs_mt4_print,
+ .save = ipvs_mt4_save,
+ .extra_opts = ipvs_mt_opts,
+ },
+ {
+ .version = XTABLES_VERSION,
+ .name = "ipvs",
+ .revision = 0,
+ .family = NFPROTO_IPV6,
+ .size = XT_ALIGN(sizeof(struct xt_ipvs_mtinfo)),
+ .userspacesize = XT_ALIGN(sizeof(struct xt_ipvs_mtinfo)),
+ .help = ipvs_mt_help,
+ .parse = ipvs_mt6_parse,
+ .final_check = ipvs_mt_check,
+ .print = ipvs_mt6_print,
+ .save = ipvs_mt6_save,
+ .extra_opts = ipvs_mt_opts,
+ },
+};
+
+void _init(void)
+{
+ xtables_register_matches(ipvs_matches_reg,
+ ARRAY_SIZE(ipvs_matches_reg));
+}
Index: iptables/extensions/libxt_ipvs.man
===================================================================
--- /dev/null 1970-01-01 00:00:00.000000000 +0000
+++ iptables/extensions/libxt_ipvs.man 2010-07-04 20:23:30.000000000 +0900
@@ -0,0 +1,24 @@
+Match IPVS connection properties.
+.TP
+[\fB!\fR] \fB\-\-ipvs\fP
+packet belongs to an IPVS connection
+.TP
+Any of the following options implies \-\-ipvs (even negated)
+.TP
+[\fB!\fR] \fB\-\-vproto\fP \fIprotocol\fP
+VIP protocol to match; by number or name, e.g. "tcp"
+.TP
+[\fB!\fR] \fB\-\-vaddr\fP \fIaddress\fP[\fB/\fP\fImask\fP]
+VIP address to match
+.TP
+[\fB!\fR] \fB\-\-vport\fP \fIport\fP
+VIP port to match; by number or name, e.g. "http"
+.TP
+\fB\-\-vdir\fP {\fBORIGINAL\fP|\fBREPLY\fP}
+flow direction of packet
+.TP
+[\fB!\fR] \fB\-\-vmethod\fP {\fBGATE\fP|\fBIPIP\fP|\fBMASQ\fP}
+IPVS forwarding method used
+.TP
+[\fB!\fR] \fB\-\-vportctl\fP \fIport\fP
+VIP port of the controlling connection to match, e.g. 21 for FTP
Index: iptables/include/linux/netfilter/xt_ipvs.h
===================================================================
--- /dev/null 1970-01-01 00:00:00.000000000 +0000
+++ iptables/include/linux/netfilter/xt_ipvs.h 2010-07-04 20:23:30.000000000 +0900
@@ -0,0 +1,25 @@
+#ifndef _XT_IPVS_H
+#define _XT_IPVS_H 1
+
+#define XT_IPVS_IPVS_PROPERTY (1 << 0) /* all other options imply this one */
+#define XT_IPVS_PROTO (1 << 1)
+#define XT_IPVS_VADDR (1 << 2)
+#define XT_IPVS_VPORT (1 << 3)
+#define XT_IPVS_DIR (1 << 4)
+#define XT_IPVS_METHOD (1 << 5)
+#define XT_IPVS_VPORTCTL (1 << 6)
+#define XT_IPVS_MASK ((1 << 7) - 1)
+#define XT_IPVS_ONCE_MASK (XT_IPVS_MASK & ~XT_IPVS_IPVS_PROPERTY)
+
+struct xt_ipvs_mtinfo {
+ union nf_inet_addr vaddr, vmask;
+ __be16 vport;
+ __u16 l4proto;
+ __u16 fwd_method;
+ __be16 vportctl;
+
+ __u8 invert;
+ __u8 bitmask;
+};
+
+#endif /* _XT_IPVS_H */
^ permalink raw reply
* [patch v2.6 3/4] IPVS: make FTP work with full NAT support
From: horms, Simon Horman @ 2010-07-11 9:03 UTC (permalink / raw)
To: lvs-devel, netdev, linux-kernel, netfilter, netfilter-devel
Cc: Malcolm Turnbull, Wensong Zhang, Julius Volz, Patrick McHardy,
David S. Miller, Hannes Eder
In-Reply-To: <20100711090342.035149543@vergenet.net>
[-- Attachment #1: IPVS-make-FTP-work-with-full-NAT-support.patch --]
[-- Type: text/plain, Size: 12645 bytes --]
From: Hannes Eder <heder@google.com>
Use nf_conntrack/nf_nat code to do the packet mangling and the TCP
sequence adjusting. The function 'ip_vs_skb_replace' is now dead
code, so it is removed.
To SNAT FTP, use something like:
% iptables -t nat -A POSTROUTING -m ipvs --vaddr 192.168.100.30/32 \
> --vport 21 -j SNAT --to-source 192.168.10.10
and for the data connections in passive mode:
% iptables -t nat -A POSTROUTING -m ipvs --vaddr 192.168.100.30/32 \
> --vportctl 21 -j SNAT --to-source 192.168.10.10
using '-m state --state RELATED' would also works.
Make sure the kernel modules ip_vs_ftp, nf_conntrack_ftp, and
nf_nat_ftp are loaded.
[ up-port and minor fixes by Simon Horman <horms@verge.net.au> ]
Signed-off-by: Hannes Eder <heder@google.com>
Signed-off-by: Simon Horman <horms@verge.net.au>
---
include/net/ip_vs.h | 2
net/netfilter/ipvs/Kconfig | 2
net/netfilter/ipvs/ip_vs_app.c | 43 ---------
net/netfilter/ipvs/ip_vs_core.c | 1
net/netfilter/ipvs/ip_vs_ftp.c | 174 ++++++++++++++++++++++++++++++++++++---
5 files changed, 164 insertions(+), 58 deletions(-)
v2.6
* pointer arguments for %pI4
v2.5
* Use nf_ct_is_untracked(ct) instead of nf_ct_is_untracked(),
the latter is blatantly incorrect
* Return 0 (and thus drop the packet) if mangling wasn't attempted
v2.4
As suggested by Patrick McHardy
* Use nf_conntrack_untracked() instead of &nf_conntrack_untracked
* Fix ip_vs_ftp_out logic
- Don't call nf_nat_mangle_tcp_packet() unless ct is valid and tracked
- Only call ip_vs_expect_relatedi() if nf_nat_mangle_tcp_packet()
succeeds
- Note that packets are dropped if mangling fails
Other
* Drop unrelated cosmetic change to sizing of buf in ip_vs_ftp_out()
v2.3
* Up-port
* Drop buf_len = snprintf() change - its a separate, cosmetic, fix
As suggested by Patrick McHardy
* Use %pI4 instead of NIPQUAD
v2.2
* No change
v2.1
* Up-port
Index: nf-next-2.6/include/net/ip_vs.h
===================================================================
--- nf-next-2.6.orig/include/net/ip_vs.h 2010-07-11 17:30:19.000000000 +0900
+++ nf-next-2.6/include/net/ip_vs.h 2010-07-11 17:33:33.000000000 +0900
@@ -736,8 +736,6 @@ extern void ip_vs_app_inc_put(struct ip_
extern int ip_vs_app_pkt_out(struct ip_vs_conn *, struct sk_buff *skb);
extern int ip_vs_app_pkt_in(struct ip_vs_conn *, struct sk_buff *skb);
-extern int ip_vs_skb_replace(struct sk_buff *skb, gfp_t pri,
- char *o_buf, int o_len, char *n_buf, int n_len);
extern int ip_vs_app_init(void);
extern void ip_vs_app_cleanup(void);
Index: nf-next-2.6/net/netfilter/ipvs/Kconfig
===================================================================
--- nf-next-2.6.orig/net/netfilter/ipvs/Kconfig 2010-07-11 17:33:06.000000000 +0900
+++ nf-next-2.6/net/netfilter/ipvs/Kconfig 2010-07-11 17:33:33.000000000 +0900
@@ -235,7 +235,7 @@ comment 'IPVS application helper'
config IP_VS_FTP
tristate "FTP protocol helper"
- depends on IP_VS_PROTO_TCP
+ depends on IP_VS_PROTO_TCP && NF_NAT
---help---
FTP is a protocol that transfers IP address and/or port number in
the payload. In the virtual server via Network Address Translation,
Index: nf-next-2.6/net/netfilter/ipvs/ip_vs_app.c
===================================================================
--- nf-next-2.6.orig/net/netfilter/ipvs/ip_vs_app.c 2010-07-11 17:30:19.000000000 +0900
+++ nf-next-2.6/net/netfilter/ipvs/ip_vs_app.c 2010-07-11 17:33:33.000000000 +0900
@@ -569,49 +569,6 @@ static const struct file_operations ip_v
};
#endif
-
-/*
- * Replace a segment of data with a new segment
- */
-int ip_vs_skb_replace(struct sk_buff *skb, gfp_t pri,
- char *o_buf, int o_len, char *n_buf, int n_len)
-{
- int diff;
- int o_offset;
- int o_left;
-
- EnterFunction(9);
-
- diff = n_len - o_len;
- o_offset = o_buf - (char *)skb->data;
- /* The length of left data after o_buf+o_len in the skb data */
- o_left = skb->len - (o_offset + o_len);
-
- if (diff <= 0) {
- memmove(o_buf + n_len, o_buf + o_len, o_left);
- memcpy(o_buf, n_buf, n_len);
- skb_trim(skb, skb->len + diff);
- } else if (diff <= skb_tailroom(skb)) {
- skb_put(skb, diff);
- memmove(o_buf + n_len, o_buf + o_len, o_left);
- memcpy(o_buf, n_buf, n_len);
- } else {
- if (pskb_expand_head(skb, skb_headroom(skb), diff, pri))
- return -ENOMEM;
- skb_put(skb, diff);
- memmove(skb->data + o_offset + n_len,
- skb->data + o_offset + o_len, o_left);
- skb_copy_to_linear_data_offset(skb, o_offset, n_buf, n_len);
- }
-
- /* must update the iph total length here */
- ip_hdr(skb)->tot_len = htons(skb->len);
-
- LeaveFunction(9);
- return 0;
-}
-
-
int __init ip_vs_app_init(void)
{
/* we will replace it with proc_net_ipvs_create() soon */
Index: nf-next-2.6/net/netfilter/ipvs/ip_vs_core.c
===================================================================
--- nf-next-2.6.orig/net/netfilter/ipvs/ip_vs_core.c 2010-07-11 17:33:06.000000000 +0900
+++ nf-next-2.6/net/netfilter/ipvs/ip_vs_core.c 2010-07-11 17:33:33.000000000 +0900
@@ -54,7 +54,6 @@
EXPORT_SYMBOL(register_ip_vs_scheduler);
EXPORT_SYMBOL(unregister_ip_vs_scheduler);
-EXPORT_SYMBOL(ip_vs_skb_replace);
EXPORT_SYMBOL(ip_vs_proto_name);
EXPORT_SYMBOL(ip_vs_conn_new);
EXPORT_SYMBOL(ip_vs_conn_in_get);
Index: nf-next-2.6/net/netfilter/ipvs/ip_vs_ftp.c
===================================================================
--- nf-next-2.6.orig/net/netfilter/ipvs/ip_vs_ftp.c 2010-07-11 17:30:19.000000000 +0900
+++ nf-next-2.6/net/netfilter/ipvs/ip_vs_ftp.c 2010-07-11 18:01:58.000000000 +0900
@@ -20,6 +20,17 @@
*
* Author: Wouter Gadeyne
*
+ *
+ * Code for ip_vs_expect_related and ip_vs_expect_callback is taken from
+ * http://www.ssi.bg/~ja/nfct/:
+ *
+ * ip_vs_nfct.c: Netfilter connection tracking support for IPVS
+ *
+ * Portions Copyright (C) 2001-2002
+ * Antefacto Ltd, 181 Parnell St, Dublin 1, Ireland.
+ *
+ * Portions Copyright (C) 2003-2008
+ * Julian Anastasov
*/
#define KMSG_COMPONENT "IPVS"
@@ -32,6 +43,9 @@
#include <linux/in.h>
#include <linux/ip.h>
#include <linux/netfilter.h>
+#include <net/netfilter/nf_conntrack.h>
+#include <net/netfilter/nf_conntrack_expect.h>
+#include <net/netfilter/nf_nat_helper.h>
#include <linux/gfp.h>
#include <net/protocol.h>
#include <net/tcp.h>
@@ -43,6 +57,16 @@
#define SERVER_STRING "227 Entering Passive Mode ("
#define CLIENT_STRING "PORT "
+#define FMT_TUPLE "%pI4:%u->%pI4:%u/%u"
+#define ARG_TUPLE(T) &(T)->src.u3.ip, ntohs((T)->src.u.all), \
+ &(T)->dst.u3.ip, ntohs((T)->dst.u.all), \
+ (T)->dst.protonum
+
+#define FMT_CONN "%pI4:%u->%pI4:%u->%pI4:%u/%u:%u"
+#define ARG_CONN(C) &((C)->caddr.ip), ntohs((C)->cport), \
+ &((C)->vaddr.ip), ntohs((C)->vport), \
+ &((C)->daddr.ip), ntohs((C)->dport), \
+ (C)->protocol, (C)->state
/*
* List of ports (up to IP_VS_APP_MAX_PORTS) to be handled by helper
@@ -123,6 +147,119 @@ static int ip_vs_ftp_get_addrport(char *
return 1;
}
+/*
+ * Called from init_conntrack() as expectfn handler.
+ */
+static void
+ip_vs_expect_callback(struct nf_conn *ct,
+ struct nf_conntrack_expect *exp)
+{
+ struct nf_conntrack_tuple *orig, new_reply;
+ struct ip_vs_conn *cp;
+
+ if (exp->tuple.src.l3num != PF_INET)
+ return;
+
+ /*
+ * We assume that no NF locks are held before this callback.
+ * ip_vs_conn_out_get and ip_vs_conn_in_get should match their
+ * expectations even if they use wildcard values, now we provide the
+ * actual values from the newly created original conntrack direction.
+ * The conntrack is confirmed when packet reaches IPVS hooks.
+ */
+
+ /* RS->CLIENT */
+ orig = &ct->tuplehash[IP_CT_DIR_ORIGINAL].tuple;
+ cp = ip_vs_conn_out_get(exp->tuple.src.l3num, orig->dst.protonum,
+ &orig->src.u3, orig->src.u.tcp.port,
+ &orig->dst.u3, orig->dst.u.tcp.port);
+ if (cp) {
+ /* Change reply CLIENT->RS to CLIENT->VS */
+ new_reply = ct->tuplehash[IP_CT_DIR_REPLY].tuple;
+ IP_VS_DBG(7, "%s(): ct=%p, status=0x%lX, tuples=" FMT_TUPLE ", "
+ FMT_TUPLE ", found inout cp=" FMT_CONN "\n",
+ __func__, ct, ct->status,
+ ARG_TUPLE(orig), ARG_TUPLE(&new_reply),
+ ARG_CONN(cp));
+ new_reply.dst.u3 = cp->vaddr;
+ new_reply.dst.u.tcp.port = cp->vport;
+ IP_VS_DBG(7, "%s(): ct=%p, new tuples=" FMT_TUPLE ", " FMT_TUPLE
+ ", inout cp=" FMT_CONN "\n",
+ __func__, ct,
+ ARG_TUPLE(orig), ARG_TUPLE(&new_reply),
+ ARG_CONN(cp));
+ goto alter;
+ }
+
+ /* CLIENT->VS */
+ cp = ip_vs_conn_in_get(exp->tuple.src.l3num, orig->dst.protonum,
+ &orig->src.u3, orig->src.u.tcp.port,
+ &orig->dst.u3, orig->dst.u.tcp.port);
+ if (cp) {
+ /* Change reply VS->CLIENT to RS->CLIENT */
+ new_reply = ct->tuplehash[IP_CT_DIR_REPLY].tuple;
+ IP_VS_DBG(7, "%s(): ct=%p, status=0x%lX, tuples=" FMT_TUPLE ", "
+ FMT_TUPLE ", found outin cp=" FMT_CONN "\n",
+ __func__, ct, ct->status,
+ ARG_TUPLE(orig), ARG_TUPLE(&new_reply),
+ ARG_CONN(cp));
+ new_reply.src.u3 = cp->daddr;
+ new_reply.src.u.tcp.port = cp->dport;
+ IP_VS_DBG(7, "%s(): ct=%p, new tuples=" FMT_TUPLE ", "
+ FMT_TUPLE ", outin cp=" FMT_CONN "\n",
+ __func__, ct,
+ ARG_TUPLE(orig), ARG_TUPLE(&new_reply),
+ ARG_CONN(cp));
+ goto alter;
+ }
+
+ IP_VS_DBG(7, "%s(): ct=%p, status=0x%lX, tuple=" FMT_TUPLE
+ " - unknown expect\n",
+ __func__, ct, ct->status, ARG_TUPLE(orig));
+ return;
+
+alter:
+ /* Never alter conntrack for non-NAT conns */
+ if (IP_VS_FWD_METHOD(cp) == IP_VS_CONN_F_MASQ)
+ nf_conntrack_alter_reply(ct, &new_reply);
+ ip_vs_conn_put(cp);
+ return;
+}
+
+/*
+ * Create NF conntrack expectation with wildcard (optional) source port.
+ * Then the default callback function will alter the reply and will confirm
+ * the conntrack entry when the first packet comes.
+ */
+static void
+ip_vs_expect_related(struct sk_buff *skb, struct nf_conn *ct,
+ struct ip_vs_conn *cp, u_int8_t proto,
+ const __be16 *port, int from_rs)
+{
+ struct nf_conntrack_expect *exp;
+
+ BUG_ON(!ct || ct == &nf_conntrack_untracked);
+
+ exp = nf_ct_expect_alloc(ct);
+ if (!exp)
+ return;
+
+ if (from_rs)
+ nf_ct_expect_init(exp, NF_CT_EXPECT_CLASS_DEFAULT,
+ nf_ct_l3num(ct), &cp->daddr, &cp->caddr,
+ proto, port, &cp->cport);
+ else
+ nf_ct_expect_init(exp, NF_CT_EXPECT_CLASS_DEFAULT,
+ nf_ct_l3num(ct), &cp->caddr, &cp->vaddr,
+ proto, port, &cp->vport);
+
+ exp->expectfn = ip_vs_expect_callback;
+
+ IP_VS_DBG(7, "%s(): ct=%p, expect tuple=" FMT_TUPLE "\n",
+ __func__, ct, ARG_TUPLE(&exp->tuple));
+ nf_ct_expect_related(exp);
+ nf_ct_expect_put(exp);
+}
/*
* Look at outgoing ftp packets to catch the response to a PASV command
@@ -149,7 +286,9 @@ static int ip_vs_ftp_out(struct ip_vs_ap
struct ip_vs_conn *n_cp;
char buf[24]; /* xxx.xxx.xxx.xxx,ppp,ppp\000 */
unsigned buf_len;
- int ret;
+ int ret = 0;
+ enum ip_conntrack_info ctinfo;
+ struct nf_conn *ct;
#ifdef CONFIG_IP_VS_IPV6
/* This application helper doesn't work with IPv6 yet,
@@ -219,19 +358,26 @@ static int ip_vs_ftp_out(struct ip_vs_ap
buf_len = strlen(buf);
+ ct = nf_ct_get(skb, &ctinfo);
+ if (ct && !nf_ct_is_untracked(ct)) {
+ /* If mangling fails this function will return 0
+ * which will cause the packet to be dropped.
+ * Mangling can only fail under memory pressure,
+ * hopefully it will succeed on the retransmitted
+ * packet.
+ */
+ ret = nf_nat_mangle_tcp_packet(skb, ct, ctinfo,
+ start-data, end-start,
+ buf, buf_len);
+ if (ret)
+ ip_vs_expect_related(skb, ct, n_cp,
+ IPPROTO_TCP, NULL, 0);
+ }
+
/*
- * Calculate required delta-offset to keep TCP happy
+ * Not setting 'diff' is intentional, otherwise the sequence
+ * would be adjusted twice.
*/
- *diff = buf_len - (end-start);
-
- if (*diff == 0) {
- /* simply replace it with new passive address */
- memcpy(start, buf, buf_len);
- ret = 1;
- } else {
- ret = !ip_vs_skb_replace(skb, GFP_ATOMIC, start,
- end-start, buf, buf_len);
- }
cp->app_data = NULL;
ip_vs_tcp_conn_listen(n_cp);
@@ -263,6 +409,7 @@ static int ip_vs_ftp_in(struct ip_vs_app
union nf_inet_addr to;
__be16 port;
struct ip_vs_conn *n_cp;
+ struct nf_conn *ct;
#ifdef CONFIG_IP_VS_IPV6
/* This application helper doesn't work with IPv6 yet,
@@ -349,6 +496,11 @@ static int ip_vs_ftp_in(struct ip_vs_app
ip_vs_control_add(n_cp, cp);
}
+ ct = (struct nf_conn *)skb->nfct;
+ if (ct && ct != &nf_conntrack_untracked)
+ ip_vs_expect_related(skb, ct, n_cp,
+ IPPROTO_TCP, &n_cp->dport, 1);
+
/*
* Move tunnel to listen state
*/
^ permalink raw reply
* [patch v2.6 2/4] IPVS: make friends with nf_conntrack
From: horms, Simon Horman @ 2010-07-11 9:03 UTC (permalink / raw)
To: lvs-devel, netdev, linux-kernel, netfilter, netfilter-devel
Cc: Malcolm Turnbull, Wensong Zhang, Julius Volz, Patrick McHardy,
David S. Miller, Hannes Eder
In-Reply-To: <20100711090342.035149543@vergenet.net>
[-- Attachment #1: IPVS-make-friends-with-nf_conntrack.patch --]
[-- Type: text/plain, Size: 5649 bytes --]
From: Hannes Eder <heder@google.com>
Update the nf_conntrack tuple in reply direction, as we will see
traffic from the real server (RIP) to the client (CIP). Once this is
done we can use netfilters SNAT in POSTROUTING, especially with
xt_ipvs, to do source NAT, e.g.:
% iptables -t nat -A POSTROUTING -m ipvs --vaddr 192.168.100.30/32 --vport 80 \
> -j SNAT --to-source 192.168.10.10
[ minor fixes by Simon Horman <horms@verge.net.au> ]
Signed-off-by: Hannes Eder <heder@google.com>
Signed-off-by: Simon Horman <horms@verge.net.au>
---
net/netfilter/ipvs/Kconfig | 2 +-
net/netfilter/ipvs/ip_vs_core.c | 36 ------------------------------------
net/netfilter/ipvs/ip_vs_xmit.c | 29 +++++++++++++++++++++++++++++
3 files changed, 30 insertions(+), 37 deletions(-)
v2.4
As per advice from Patrick McHardy
* Use nf_conntrack_untracked() instead of &nf_conntrack_untracked
v2.1, v2.2, v2.3
No change
Index: nf-next-2.6/net/netfilter/ipvs/Kconfig
===================================================================
--- nf-next-2.6.orig/net/netfilter/ipvs/Kconfig 2010-07-07 13:24:31.000000000 +0900
+++ nf-next-2.6/net/netfilter/ipvs/Kconfig 2010-07-07 13:38:23.000000000 +0900
@@ -3,7 +3,7 @@
#
menuconfig IP_VS
tristate "IP virtual server support"
- depends on NET && INET && NETFILTER
+ depends on NET && INET && NETFILTER && NF_CONNTRACK
---help---
IP Virtual Server support will let you build a high-performance
virtual server based on cluster of two or more real servers. This
Index: nf-next-2.6/net/netfilter/ipvs/ip_vs_core.c
===================================================================
--- nf-next-2.6.orig/net/netfilter/ipvs/ip_vs_core.c 2010-07-07 13:23:37.000000000 +0900
+++ nf-next-2.6/net/netfilter/ipvs/ip_vs_core.c 2010-07-07 13:38:23.000000000 +0900
@@ -536,26 +536,6 @@ int ip_vs_leave(struct ip_vs_service *sv
return NF_DROP;
}
-
-/*
- * It is hooked before NF_IP_PRI_NAT_SRC at the NF_INET_POST_ROUTING
- * chain, and is used for VS/NAT.
- * It detects packets for VS/NAT connections and sends the packets
- * immediately. This can avoid that iptable_nat mangles the packets
- * for VS/NAT.
- */
-static unsigned int ip_vs_post_routing(unsigned int hooknum,
- struct sk_buff *skb,
- const struct net_device *in,
- const struct net_device *out,
- int (*okfn)(struct sk_buff *))
-{
- if (!skb->ipvs_property)
- return NF_ACCEPT;
- /* The packet was sent from IPVS, exit this chain */
- return NF_STOP;
-}
-
__sum16 ip_vs_checksum_complete(struct sk_buff *skb, int offset)
{
return csum_fold(skb_checksum(skb, offset, skb->len - offset, 0));
@@ -1499,14 +1479,6 @@ static struct nf_hook_ops ip_vs_ops[] __
.hooknum = NF_INET_FORWARD,
.priority = 99,
},
- /* Before the netfilter connection tracking, exit from POST_ROUTING */
- {
- .hook = ip_vs_post_routing,
- .owner = THIS_MODULE,
- .pf = PF_INET,
- .hooknum = NF_INET_POST_ROUTING,
- .priority = NF_IP_PRI_NAT_SRC-1,
- },
#ifdef CONFIG_IP_VS_IPV6
/* After packet filtering, forward packet through VS/DR, VS/TUN,
* or VS/NAT(change destination), so that filtering rules can be
@@ -1535,14 +1507,6 @@ static struct nf_hook_ops ip_vs_ops[] __
.hooknum = NF_INET_FORWARD,
.priority = 99,
},
- /* Before the netfilter connection tracking, exit from POST_ROUTING */
- {
- .hook = ip_vs_post_routing,
- .owner = THIS_MODULE,
- .pf = PF_INET6,
- .hooknum = NF_INET_POST_ROUTING,
- .priority = NF_IP6_PRI_NAT_SRC-1,
- },
#endif
};
Index: nf-next-2.6/net/netfilter/ipvs/ip_vs_xmit.c
===================================================================
--- nf-next-2.6.orig/net/netfilter/ipvs/ip_vs_xmit.c 2010-07-07 13:23:37.000000000 +0900
+++ nf-next-2.6/net/netfilter/ipvs/ip_vs_xmit.c 2010-07-07 13:42:22.000000000 +0900
@@ -28,6 +28,7 @@
#include <net/ip6_route.h>
#include <linux/icmpv6.h>
#include <linux/netfilter.h>
+#include <net/netfilter/nf_conntrack.h>
#include <linux/netfilter_ipv4.h>
#include <net/ip_vs.h>
@@ -348,6 +349,30 @@ ip_vs_bypass_xmit_v6(struct sk_buff *skb
}
#endif
+static void
+ip_vs_update_conntrack(struct sk_buff *skb, struct ip_vs_conn *cp)
+{
+ struct nf_conn *ct = (struct nf_conn *)skb->nfct;
+ struct nf_conntrack_tuple new_tuple;
+
+ if (ct == NULL || nf_ct_is_untracked(ct) || nf_ct_is_confirmed(ct))
+ return;
+
+ /*
+ * The connection is not yet in the hashtable, so we update it.
+ * CIP->VIP will remain the same, so leave the tuple in
+ * IP_CT_DIR_ORIGINAL untouched. When the reply comes back from the
+ * real-server we will see RIP->DIP.
+ */
+ new_tuple = ct->tuplehash[IP_CT_DIR_REPLY].tuple;
+ new_tuple.src.u3 = cp->daddr;
+ /*
+ * This will also take care of UDP and other protocols.
+ */
+ new_tuple.src.u.tcp.port = cp->dport;
+ nf_conntrack_alter_reply(ct, &new_tuple);
+}
+
/*
* NAT transmitter (only for outside-to-inside nat forwarding)
* Not used for related ICMP
@@ -403,6 +428,8 @@ ip_vs_nat_xmit(struct sk_buff *skb, stru
IP_VS_DBG_PKT(10, pp, skb, 0, "After DNAT");
+ ip_vs_update_conntrack(skb, cp);
+
/* FIXME: when application helper enlarges the packet and the length
is larger than the MTU of outgoing device, there will be still
MTU problem. */
@@ -479,6 +506,8 @@ ip_vs_nat_xmit_v6(struct sk_buff *skb, s
IP_VS_DBG_PKT(10, pp, skb, 0, "After DNAT");
+ ip_vs_update_conntrack(skb, cp);
+
/* FIXME: when application helper enlarges the packet and the length
is larger than the MTU of outgoing device, there will be still
MTU problem. */
^ permalink raw reply
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox