* Re: ipcomp regression in 2.6.24
From: Herbert Xu @ 2008-01-31 5:32 UTC (permalink / raw)
To: Marco Berizzi; +Cc: davem, Daniel.Beschorner, netdev
In-Reply-To: <BAY103-DAV204C57458B3E41DE1AA365B2360@phx.gbl>
On Wed, Jan 30, 2008 at 10:14:46AM +0100, Marco Berizzi wrote:
>
> Sorry for bother you again.
> I have applied to 2.6.24, but ipcomp doesn't work anyway.
> I have patched a clean 2.6.24 tree and I did a complete
> rebuild.
> With tcpdump I see both the esp packets going in/out but
> I don't see the clear packets on the interface.
After testing it here it looks like there is this little typo
which means that you can't actually use IPComp for anything
that's not compressible :)
[IPCOMP]: Fix reception of incompressible packets
I made a silly typo by entering IPPROTO_IP (== 0) instead of
IPPROTO_IPIP (== 4). This broke the reception of incompressible
packets.
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Cheers,
--
Visit Openswan at http://www.openswan.org/
Email: Herbert Xu ~{PmV>HI~} <herbert@gondor.apana.org.au>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt
--
diff --git a/net/ipv4/xfrm4_tunnel.c b/net/ipv4/xfrm4_tunnel.c
index 3268451..152f83d 100644
--- a/net/ipv4/xfrm4_tunnel.c
+++ b/net/ipv4/xfrm4_tunnel.c
@@ -50,7 +50,7 @@ static struct xfrm_type ipip_type = {
static int xfrm_tunnel_rcv(struct sk_buff *skb)
{
- return xfrm4_rcv_spi(skb, IPPROTO_IP, ip_hdr(skb)->saddr);
+ return xfrm4_rcv_spi(skb, IPPROTO_IPIP, ip_hdr(skb)->saddr);
}
static int xfrm_tunnel_err(struct sk_buff *skb, u32 info)
^ permalink raw reply related
* RE: e1000 full-duplex TCP performance well below wire speed
From: Brandeburg, Jesse @ 2008-01-31 5:43 UTC (permalink / raw)
To: Bruce Allen; +Cc: netdev, Carsten Aulbert, Henning Fehrmann, Bruce Allen
In-Reply-To: <Pine.LNX.4.63.0801301635470.19938@trinity.phys.uwm.edu>
Bruce Allen wrote:
> Hi Jesse,
>
> It's good to be talking directly to one of the e1000 developers and
> maintainers. Although at this point I am starting to think that the
> issue may be TCP stack related and nothing to do with the NIC. Am I
> correct that these are quite distinct parts of the kernel?
Yes, quite.
> Important note: we ARE able to get full duplex wire speed (over 900
> Mb/s simulaneously in both directions) using UDP. The problems occur
> only with TCP connections.
That eliminates bus bandwidth issues, probably, but small packets take
up a lot of extra descriptors, bus bandwidth, CPU, and cache resources.
>>> The test was done with various mtu sizes ranging from 1500 to 9000,
>>> with ethernet flow control switched on and off, and using reno and
>>> cubic as a TCP congestion control.
>>
>> As asked in LKML thread, please post the exact netperf command used
>> to start the client/server, whether or not you're using irqbalanced
>> (aka irqbalance) and what cat /proc/interrupts looks like (you ARE
>> using MSI, right?)
>
> I have to wait until Carsten or Henning wake up tomorrow (now 23:38 in
> Germany). So we'll provide this info in ~10 hours.
I would suggest you try TCP_RR with a command line something like this:
netperf -t TCP_RR -H <hostname> -C -c -- -b 4 -r 64K
I think you'll have to compile netperf with burst mode support enabled.
> I assume that the interrupt load is distributed among all four cores
> -- the default affinity is 0xff, and I also assume that there is some
> type of interrupt aggregation taking place in the driver. If the
> CPUs were not able to service the interrupts fast enough, I assume
> that we would also see loss of performance with UDP testing.
>
>> One other thing you can try with e1000 is disabling the dynamic
>> interrupt moderation by loading the driver with
>> InterruptThrottleRate=8000,8000,... (the number of commas depends on
>> your number of ports) which might help in your particular benchmark.
>
> OK. Is 'dynamic interrupt moderation' another name for 'interrupt
> aggregation'? Meaning that if more than one interrupt is generated
> in a given time interval, then they are replaced by a single
> interrupt?
Yes, InterruptThrottleRate=8000 means there will be no more than 8000
ints/second from that adapter, and if interrupts are generated faster
than that they are "aggregated."
Interestingly since you are interested in ultra low latency, and may be
willing to give up some cpu for it during bulk transfers you should try
InterruptThrottleRate=1 (can generate up to 70000 ints/s)
>> just for completeness can you post the dump of ethtool -e eth0 and
>> lspci -vvv?
>
> Yup, we'll give that info also.
>
> Thanks again!
Welcome, its an interesting discussion. Hope we can come to a good
conclusion.
Jesse
^ permalink raw reply
* Re: ipcomp regression in 2.6.24
From: David Miller @ 2008-01-31 5:48 UTC (permalink / raw)
To: herbert; +Cc: pupilla, Daniel.Beschorner, netdev
In-Reply-To: <20080131053221.GA4739@gondor.apana.org.au>
From: Herbert Xu <herbert@gondor.apana.org.au>
Date: Thu, 31 Jan 2008 16:32:21 +1100
> [IPCOMP]: Fix reception of incompressible packets
>
> I made a silly typo by entering IPPROTO_IP (== 0) instead of
> IPPROTO_IPIP (== 4). This broke the reception of incompressible
> packets.
>
> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Applied, thanks!
^ permalink raw reply
* Re: net-2.6.25 is no more...
From: Jeff Garzik @ 2008-01-31 6:09 UTC (permalink / raw)
To: David Miller
Cc: dlezcano-NmTC/0ZBporQT0dZR+AlfA, netdev-u79uwXL29TY76Z2rM5mHXA,
linux-wireless-u79uwXL29TY76Z2rM5mHXA, John Linville
In-Reply-To: <20080130.054807.182676684.davem-fT/PcQaiUtIeIZ0/mPfg9Q@public.gmane.org>
David Miller wrote:
> From: Daniel Lezcano <dlezcano-NmTC/0ZBporQT0dZR+AlfA@public.gmane.org>
> Date: Wed, 30 Jan 2008 10:03:09 +0100
>
>> David Miller wrote:
>>> Now that the bulk has been merged over and we are
>>> actively working alongside Linus's tree I have moved
>>> all current patch applying to net-2.6 instead of net-2.6.25,
>>> so the current tree to use is:
>>>
>>> kernel.org:/pub/scm/linux/kernel/git/davem/net-2.6.git
>> This tree is for fixes only, right ? or shall we send enhancement
>> patches to net-2.6 until net-2.6.26 appears ?
>
> The latter.
So... what to do about changes which are not bug fixes? Such changes
should not go upstream immediately, because that's the standard rule for
merge windows.
And queueing them on our side just re-creates the same old situation we
just changed away from, after all.
Jeff
^ permalink raw reply
* Re: net-2.6.25 is no more...
From: David Miller @ 2008-01-31 6:21 UTC (permalink / raw)
To: jeff-o2qLIJkoznsdnm+yROfE0A
Cc: dlezcano-NmTC/0ZBporQT0dZR+AlfA, netdev-u79uwXL29TY76Z2rM5mHXA,
linux-wireless-u79uwXL29TY76Z2rM5mHXA,
linville-2XuSBdqkA4R54TAoqtyWWQ
In-Reply-To: <47A16611.3000602-o2qLIJkoznsdnm+yROfE0A@public.gmane.org>
From: Jeff Garzik <jeff-o2qLIJkoznsdnm+yROfE0A@public.gmane.org>
Date: Thu, 31 Jan 2008 01:09:21 -0500
> David Miller wrote:
> > The latter.
>
> So... what to do about changes which are not bug fixes? Such changes
> should not go upstream immediately, because that's the standard rule for
> merge windows.
>
> And queueing them on our side just re-creates the same old situation we
> just changed away from, after all.
I'm saying that everything should go to the net-2.6 tree right now.
When the merge window closes, net-2.6 will be for bug fixes only and
I'll hold off on creating net-2.6.26 for about a week so that people
concentrate on regression fixes.
Yes, things will pile up but I think it's appropriate to hold off on
merging features into the 2.6.26 queue for just one week.
^ permalink raw reply
* Re: [PATCH] [1/1] Deprecate tcp_tw_{reuse,recycle}
From: Ben Greear @ 2008-01-31 6:37 UTC (permalink / raw)
To: Andi Kleen; +Cc: netdev
In-Reply-To: <200801310359.07362.ak@suse.de>
Andi Kleen wrote:
> On Wednesday 30 January 2008 20:22, Ben Greear wrote:
>
>
>> We use these features to enable creating very high numbers of short-lived
>> TCP connections, primarily used as a test tool for other network
>> devices.
>>
>
> Hopefully these other network devices don't do any NAT then
> or don't otherwise violate the IP-matches-PAWS assumption.
> Most likely they do actually, so enabling TW recycle
> for testing is probably not even safe for you.
>
> Modern systems have a lot of RAM so even without tw recycle
> you should be able to get a very high number of connections.
> An timewait socket is around 128 bytes on 64bit; this means
> with a GB of memory you can already support > 8 Million TW sockets.
> On 32bit it's even more.
>
I believe the problem was that all of my ports were used up with
TIME_WAIT sockets and so it couldn't create more. My test
case was similar to this:
1 Have one machine B listen for connections on one interface (one IP).
2 Have one machine A make a connection to B, and close connection
immediately or soon after
it was established.
goto 2
The goal was to make a maximum number of TCP connections per second.
The data passed
is just filler, and for the fastest settings, we don't pass data at all.
Without setting
tcp_tw_recycle to 1, the system could do only a few thousand connections
per second. With
it set to 1, I think I was getting around 10,000. Either way, it was
significantly faster than
w/out recycle enabled.
So, is there a better way to max out the connections per second without
having to use tcp_tw_recycle?
Thanks,
Ben
--
Ben Greear <greearb@candelatech.com>
Candela Technologies Inc http://www.candelatech.com
^ permalink raw reply
* Re: [PATCH] [1/1] Deprecate tcp_tw_{reuse,recycle}
From: Andi Kleen @ 2008-01-31 6:55 UTC (permalink / raw)
To: Ben Greear; +Cc: netdev
In-Reply-To: <47A16CBB.3000409@candelatech.com>
> I believe the problem was that all of my ports were used up with
> TIME_WAIT sockets and so it couldn't create more. My test
> case was similar to this:
Ah that's simple to solve then :- use more IP addresses and bind
to them in RR in your user program.
Arguably the Linux TCP code should be able to do this by itself
when enough IP addresses are available, but it's not very hard
to do in user space using bind(2)
BTW it's also an very unusual case -- in most cases there are more
remote IP addresses
> So, is there a better way to max out the connections per second without
> having to use tcp_tw_recycle?
Well did you profile where the bottle necks were?
Perhaps also just increase the memory allowed for TCP sockets.
-Andi
^ permalink raw reply
* [PATCH] [VLAN] vlan_dev: Initialize dev pointer only when it is being used
From: Benjamin Li @ 2008-01-31 6:55 UTC (permalink / raw)
To: kaber, netdev; +Cc: Benjamin Li
Signed-off-by: Benjamin Li <ben@wenli.org>
---
net/8021q/vlan_dev.c | 3 ++-
1 files changed, 2 insertions(+), 1 deletions(-)
diff --git a/net/8021q/vlan_dev.c b/net/8021q/vlan_dev.c
index 8059fa4..2fa5d68 100644
--- a/net/8021q/vlan_dev.c
+++ b/net/8021q/vlan_dev.c
@@ -49,7 +49,7 @@
*/
static int vlan_dev_rebuild_header(struct sk_buff *skb)
{
- struct net_device *dev = skb->dev;
+ struct net_device *dev;
struct vlan_ethhdr *veth = (struct vlan_ethhdr *)(skb->data);
switch (veth->h_vlan_encapsulated_proto) {
@@ -60,6 +60,7 @@ static int vlan_dev_rebuild_header(struct sk_buff *skb)
return arp_find(veth->h_dest, skb);
#endif
default:
+ dev = skb->dev;
pr_debug("%s: unable to resolve type %X addresses.\n",
dev->name, ntohs(veth->h_vlan_encapsulated_proto));
--
1.5.3.8
^ permalink raw reply related
* [PATCH][XFRM]: Fix statistics.
From: Masahide NAKAMURA @ 2008-01-31 7:49 UTC (permalink / raw)
To: davem; +Cc: herbert, netdev, nakam
Hello,
I found some XFRM statistics updates which should be applied.
This patch change user-public header defines introduced at
net-2.6.25 and I hope to make it to be applied to linux 2.6.25.
[PATCH][XFRM]: Fix statistics.
o Outbound sequence number overflow error status
is counted as XfrmOutStateSeqError.
o Additionaly, it changes inbound sequence number replay
error name from XfrmInSeqOutOfWindow to XfrmInStateSeqError
to apply name scheme above.
o Inbound IPv4 UDP encapsuling type mismatch error is wrongly
mapped to XfrmInStateInvalid then this patch fiex the error
to XfrmInStateMismatch.
Signed-off-by: Masahide NAKAMURA <nakam@linux-ipv6.org>
---
Documentation/networking/xfrm_proc.txt | 8 ++++++--
include/linux/snmp.h | 3 ++-
net/xfrm/xfrm_input.c | 4 ++--
net/xfrm/xfrm_output.c | 1 +
net/xfrm/xfrm_proc.c | 3 ++-
5 files changed, 13 insertions(+), 6 deletions(-)
diff --git a/Documentation/networking/xfrm_proc.txt b/Documentation/networking/xfrm_proc.txt
index 53c1a58..d0d8baf 100644
--- a/Documentation/networking/xfrm_proc.txt
+++ b/Documentation/networking/xfrm_proc.txt
@@ -26,8 +26,9 @@ XfrmInStateProtoError:
e.g. SA key is wrong
XfrmInStateModeError:
Transformation mode specific error
-XfrmInSeqOutOfWindow:
- Sequence out of window
+XfrmInStateSeqError:
+ Sequence error
+ i.e. Sequence number is out of window
XfrmInStateExpired:
State is expired
XfrmInStateMismatch:
@@ -60,6 +61,9 @@ XfrmOutStateProtoError:
Transformation protocol specific error
XfrmOutStateModeError:
Transformation mode specific error
+XfrmOutStateSeqError:
+ Sequence error
+ i.e. Sequence number overflow
XfrmOutStateExpired:
State is expired
XfrmOutPolBlock:
diff --git a/include/linux/snmp.h b/include/linux/snmp.h
index 86d3eff..5df62ef 100644
--- a/include/linux/snmp.h
+++ b/include/linux/snmp.h
@@ -227,7 +227,7 @@ enum
LINUX_MIB_XFRMINNOSTATES, /* XfrmInNoStates */
LINUX_MIB_XFRMINSTATEPROTOERROR, /* XfrmInStateProtoError */
LINUX_MIB_XFRMINSTATEMODEERROR, /* XfrmInStateModeError */
- LINUX_MIB_XFRMINSEQOUTOFWINDOW, /* XfrmInSeqOutOfWindow */
+ LINUX_MIB_XFRMINSTATESEQERROR, /* XfrmInStateSeqError */
LINUX_MIB_XFRMINSTATEEXPIRED, /* XfrmInStateExpired */
LINUX_MIB_XFRMINSTATEMISMATCH, /* XfrmInStateMismatch */
LINUX_MIB_XFRMINSTATEINVALID, /* XfrmInStateInvalid */
@@ -241,6 +241,7 @@ enum
LINUX_MIB_XFRMOUTNOSTATES, /* XfrmOutNoStates */
LINUX_MIB_XFRMOUTSTATEPROTOERROR, /* XfrmOutStateProtoError */
LINUX_MIB_XFRMOUTSTATEMODEERROR, /* XfrmOutStateModeError */
+ LINUX_MIB_XFRMOUTSTATESEQERROR, /* XfrmOutStateSeqError */
LINUX_MIB_XFRMOUTSTATEEXPIRED, /* XfrmOutStateExpired */
LINUX_MIB_XFRMOUTPOLBLOCK, /* XfrmOutPolBlock */
LINUX_MIB_XFRMOUTPOLDEAD, /* XfrmOutPolDead */
diff --git a/net/xfrm/xfrm_input.c b/net/xfrm/xfrm_input.c
index 039e701..c32b50b 100644
--- a/net/xfrm/xfrm_input.c
+++ b/net/xfrm/xfrm_input.c
@@ -160,12 +160,12 @@ int xfrm_input(struct sk_buff *skb, int nexthdr, __be32 spi, int encap_type)
}
if ((x->encap ? x->encap->encap_type : 0) != encap_type) {
- XFRM_INC_STATS(LINUX_MIB_XFRMINSTATEINVALID);
+ XFRM_INC_STATS(LINUX_MIB_XFRMINSTATEMISMATCH);
goto drop_unlock;
}
if (x->props.replay_window && xfrm_replay_check(x, skb, seq)) {
- XFRM_INC_STATS(LINUX_MIB_XFRMINSEQOUTOFWINDOW);
+ XFRM_INC_STATS(LINUX_MIB_XFRMINSTATESEQERROR);
goto drop_unlock;
}
diff --git a/net/xfrm/xfrm_output.c b/net/xfrm/xfrm_output.c
index f4a1047..fc69036 100644
--- a/net/xfrm/xfrm_output.c
+++ b/net/xfrm/xfrm_output.c
@@ -64,6 +64,7 @@ static int xfrm_output_one(struct sk_buff *skb, int err)
if (x->type->flags & XFRM_TYPE_REPLAY_PROT) {
XFRM_SKB_CB(skb)->seq = ++x->replay.oseq;
if (unlikely(x->replay.oseq == 0)) {
+ XFRM_INC_STATS(LINUX_MIB_XFRMOUTSTATESEQERROR);
x->replay.oseq--;
xfrm_audit_state_replay_overflow(x, skb);
err = -EOVERFLOW;
diff --git a/net/xfrm/xfrm_proc.c b/net/xfrm/xfrm_proc.c
index 31d0354..2b0db13 100644
--- a/net/xfrm/xfrm_proc.c
+++ b/net/xfrm/xfrm_proc.c
@@ -22,7 +22,7 @@ static struct snmp_mib xfrm_mib_list[] = {
SNMP_MIB_ITEM("XfrmInNoStates", LINUX_MIB_XFRMINNOSTATES),
SNMP_MIB_ITEM("XfrmInStateProtoError", LINUX_MIB_XFRMINSTATEPROTOERROR),
SNMP_MIB_ITEM("XfrmInStateModeError", LINUX_MIB_XFRMINSTATEMODEERROR),
- SNMP_MIB_ITEM("XfrmInSeqOutOfWindow", LINUX_MIB_XFRMINSEQOUTOFWINDOW),
+ SNMP_MIB_ITEM("XfrmInStateSeqError", LINUX_MIB_XFRMINSTATESEQERROR),
SNMP_MIB_ITEM("XfrmInStateExpired", LINUX_MIB_XFRMINSTATEEXPIRED),
SNMP_MIB_ITEM("XfrmInStateMismatch", LINUX_MIB_XFRMINSTATEMISMATCH),
SNMP_MIB_ITEM("XfrmInStateInvalid", LINUX_MIB_XFRMINSTATEINVALID),
@@ -36,6 +36,7 @@ static struct snmp_mib xfrm_mib_list[] = {
SNMP_MIB_ITEM("XfrmOutNoStates", LINUX_MIB_XFRMOUTNOSTATES),
SNMP_MIB_ITEM("XfrmOutStateProtoError", LINUX_MIB_XFRMOUTSTATEPROTOERROR),
SNMP_MIB_ITEM("XfrmOutStateModeError", LINUX_MIB_XFRMOUTSTATEMODEERROR),
+ SNMP_MIB_ITEM("XfrmOutStateSeqError", LINUX_MIB_XFRMOUTSTATESEQERROR),
SNMP_MIB_ITEM("XfrmOutStateExpired", LINUX_MIB_XFRMOUTSTATEEXPIRED),
SNMP_MIB_ITEM("XfrmOutPolBlock", LINUX_MIB_XFRMOUTPOLBLOCK),
SNMP_MIB_ITEM("XfrmOutPolDead", LINUX_MIB_XFRMOUTPOLDEAD),
--
1.4.4.2
^ permalink raw reply related
* RE: e1000 full-duplex TCP performance well below wire speed
From: Bruce Allen @ 2008-01-31 8:31 UTC (permalink / raw)
To: Brandeburg, Jesse; +Cc: netdev, Carsten Aulbert, Henning Fehrmann, Bruce Allen
In-Reply-To: <36D9DB17C6DE9E40B059440DB8D95F52044F8BA3@orsmsx418.amr.corp.intel.com>
Hi Jesse,
>> It's good to be talking directly to one of the e1000 developers and
>> maintainers. Although at this point I am starting to think that the
>> issue may be TCP stack related and nothing to do with the NIC. Am I
>> correct that these are quite distinct parts of the kernel?
>
> Yes, quite.
OK. I hope that there is also someone knowledgable about the TCP stack
who is following this thread. (Perhaps you also know this part of the
kernel, but I am assuming that your expertise is on the e1000/NIC bits.)
>> Important note: we ARE able to get full duplex wire speed (over 900
>> Mb/s simulaneously in both directions) using UDP. The problems occur
>> only with TCP connections.
>
> That eliminates bus bandwidth issues, probably, but small packets take
> up a lot of extra descriptors, bus bandwidth, CPU, and cache resources.
I see. Your concern is the extra ACK packets associated with TCP. Even
those these represent a small volume of data (around 5% with MTU=1500, and
less at larger MTU) they double the number of packets that must be handled
by the system compared to UDP transmission at the same data rate. Is that
correct?
>> I have to wait until Carsten or Henning wake up tomorrow (now 23:38 in
>> Germany). So we'll provide this info in ~10 hours.
>
> I would suggest you try TCP_RR with a command line something like this:
> netperf -t TCP_RR -H <hostname> -C -c -- -b 4 -r 64K
>
> I think you'll have to compile netperf with burst mode support enabled.
I just saw Carsten a few minutes ago. He has to take part in a
'Baubesprechung' meeting this morning, after which he will start answering
the technical questions and doing additional testing as suggested by you
and others. If you are on the US west coast, he should have some answers
and results posted by Thursday morning Pacific time.
>> I assume that the interrupt load is distributed among all four cores
>> -- the default affinity is 0xff, and I also assume that there is some
>> type of interrupt aggregation taking place in the driver. If the
>> CPUs were not able to service the interrupts fast enough, I assume
>> that we would also see loss of performance with UDP testing.
>>
>>> One other thing you can try with e1000 is disabling the dynamic
>>> interrupt moderation by loading the driver with
>>> InterruptThrottleRate=8000,8000,... (the number of commas depends on
>>> your number of ports) which might help in your particular benchmark.
>>
>> OK. Is 'dynamic interrupt moderation' another name for 'interrupt
>> aggregation'? Meaning that if more than one interrupt is generated
>> in a given time interval, then they are replaced by a single
>> interrupt?
>
> Yes, InterruptThrottleRate=8000 means there will be no more than 8000
> ints/second from that adapter, and if interrupts are generated faster
> than that they are "aggregated."
>
> Interestingly since you are interested in ultra low latency, and may be
> willing to give up some cpu for it during bulk transfers you should try
> InterruptThrottleRate=1 (can generate up to 70000 ints/s)
I'm not sure it's quite right to say that we are interested in ultra low
latency. Most of our network transfers involve bulk data movement (a few
MB or more). We don't care so much about low latency (meaning how long it
takes the FIRST byte of data to travel from sender to receiver). We care
about aggregate bandwidth: once the pipe is full, how fast can data be
moved through it. Sow we don't care so much if getting the pipe full takes
20 us or 50 us. We just want the data to flow fast once the pipe IS full.
> Welcome, its an interesting discussion. Hope we can come to a good
> conclusion.
Thank you. Carsten will post more info and answers later today.
Cheers,
Bruce
^ permalink raw reply
* When Love Comes Knocking
From: hamilton @ 2008-01-31 8:44 UTC (permalink / raw)
To: netdev
Our Journey http://69.228.44.95/
^ permalink raw reply
* Re: ipcomp regression in 2.6.24
From: Marco Berizzi @ 2008-01-31 8:37 UTC (permalink / raw)
To: Herbert Xu; +Cc: davem, Daniel.Beschorner, netdev
In-Reply-To: <20080131053221.GA4739@gondor.apana.org.au>
Herbert Xu wrote:
> On Wed, Jan 30, 2008 at 10:14:46AM +0100, Marco Berizzi wrote:
> >
> > Sorry for bother you again.
> > I have applied to 2.6.24, but ipcomp doesn't work anyway.
> > I have patched a clean 2.6.24 tree and I did a complete
> > rebuild.
> > With tcpdump I see both the esp packets going in/out but
> > I don't see the clear packets on the interface.
>
> After testing it here it looks like there is this little typo
> which means that you can't actually use IPComp for anything
> that's not compressible :)
applied and tested to 2.6.24: ipcomp is working now.
As always, thanks a lot Herbert for fixing this.
^ permalink raw reply
* xfrm_lookup() and XFRM_POLICY_ICMP
From: Andy Johnson @ 2008-01-31 8:42 UTC (permalink / raw)
To: netdev
Hello,
A question about XFRM_POLICY_ICMP:
I had tried to understand this check in __xfrm_lookup() method in
net/xfrm/xfrm_policy.c (the recent 2.6 git dave miller tree):
...
...
if ((flags & XFRM_LOOKUP_ICMP) && !(policy->flags & XFRM_POLICY_ICMP))
goto error;
...
...
Why is the check for XFRM_POLICY_ICMP? I had grepped under the kernel tree,
and the only place where XFRM_POLICY_ICMP appears is here (except its definition
in xfrm.h).
I also grepped under openswan tree, and could not find XFRM_POLICY_ICMP.
(the struct xfrm_userpolicy_info in openswan includes XFRM_POLICY_ALLOW and
XFRM_POLICY_BLOCK and XFRM_POLICY_LOCALOK, but not XFRM_POLICY_ICMP).
I also grepped under iproute2 tree (from git) and there is no XFRM_POLICY_ICMP.
So is this there a way at all to set XFRM_POLICY_ICMP? and if not - maybe this
check is not needed at all ?
Regards,
Andy
^ permalink raw reply
* Re: e1000 full-duplex TCP performance well below wire speed
From: Bruce Allen @ 2008-01-31 8:52 UTC (permalink / raw)
To: SANGTAE HA; +Cc: Linux Kernel Mailing List, netdev, Stephen Hemminger
In-Reply-To: <649aecc70801301617m6331bcb8i8ce60366e182c739@mail.gmail.com>
Hi Sangtae,
Thanks for joining this discussion -- it's good to a CUBIC author and
expert here!
>> In our application (cluster computing) we use a very tightly coupled
>> high-speed low-latency network. There is no 'wide area traffic'. So
>> it's hard for me to understand why any networking components or
>> software layers should take more than milliseconds to ramp up or back
>> off in speed. Perhaps we should be asking for a TCP congestion
>> avoidance algorithm which is designed for a data center environment
>> where there are very few hops and typical packet delivery times are
>> tens or hundreds of microseconds. It's very different than delivering
>> data thousands of km across a WAN.
> If your network latency is low, regardless of type of protocols should
> give you more than 900Mbps.
Yes, this is also what I had thought.
In the graph that we posted, the two machines are connected by an ethernet
crossover cable. The total RTT of the two machines is probably AT MOST a
couple of hundred microseconds. Typically it takes 20 or 30 microseconds
to get the first packet out the NIC. Travel across the wire is a few
nanoseconds. Then getting the packet into the receiving NIC might be
another 20 or 30 microseconds. The ACK should fly back in about the same
time.
> I can guess the RTT of two machines is less than 4ms in your case and I
> remember the throughputs of all high-speed protocols (including
> tcp-reno) were more than 900Mbps with 4ms RTT. So, my question which
> kernel version did you use with your broadcomm NIC and got more than
> 900Mbps?
We are going to double-check this (we did the broadcom testing about two
months ago). Carsten is going to re-run the broadcomm experiments later
today and will then post the results.
You can see results from some testing on crossover-cable wired systems
with broadcomm NICs, that I did about 2 years ago, here:
http://www.lsc-group.phys.uwm.edu/beowulf/nemo/design/SMC_8508T_Performance.html
You'll notice that total TCP throughput on the crossover cable was about
220 MB/sec. With TCP overhead this is very close to 2Gb/s.
> I have two machines connected by a gig switch and I can see what happens
> in my environment. Could you post what parameters did you use for
> netperf testing?
Carsten will post these in the next few hours. If you want to simplify
further, you can even take away the gig switch and just use a crossover
cable.
> and also if you set any parameters for your testing, please post them
> here so that I can see that happens to me as well.
Carsten will post all the sysctl and ethtool parameters shortly.
Thanks again for chiming in. I am sure that with help from you, Jesse, and
Rick, we can figure out what is going on here, and get it fixed.
Cheers,
Bruce
^ permalink raw reply
* Re: e1000 full-duplex TCP performance well below wire speed
From: Andi Kleen @ 2008-01-31 9:17 UTC (permalink / raw)
To: Bruce Allen
Cc: Brandeburg, Jesse, netdev, Carsten Aulbert, Henning Fehrmann,
Bruce Allen
In-Reply-To: <Pine.LNX.4.63.0801301635470.19938@trinity.phys.uwm.edu>
Bruce Allen <ballen@gravity.phys.uwm.edu> writes:
>
> Important note: we ARE able to get full duplex wire speed (over 900
> Mb/s simulaneously in both directions) using UDP. The problems occur
> only with TCP connections.
Another issue with full duplex TCP not mentioned yet is that if TSO is used
the output will be somewhat bursty and might cause problems with the
TCP ACK clock of the other direction because the ACKs would need
to squeeze in between full TSO bursts.
You could try disabling TSO with ethtool.
-Andi
^ permalink raw reply
* Re: e1000 full-duplex TCP performance well below wire speed
From: Bruce Allen @ 2008-01-31 9:59 UTC (permalink / raw)
To: Andi Kleen
Cc: Brandeburg, Jesse, netdev, Carsten Aulbert, Henning Fehrmann,
Bruce Allen
In-Reply-To: <p738x265p7p.fsf@bingen.suse.de>
Hi Andi!
>> Important note: we ARE able to get full duplex wire speed (over 900
>> Mb/s simulaneously in both directions) using UDP. The problems occur
>> only with TCP connections.
>
> Another issue with full duplex TCP not mentioned yet is that if TSO is used
> the output will be somewhat bursty and might cause problems with the
> TCP ACK clock of the other direction because the ACKs would need
> to squeeze in between full TSO bursts.
>
> You could try disabling TSO with ethtool.
Noted. We'll try this also.
Cheers,
Bruce
^ permalink raw reply
* Re: Strange commit 42a73808ed4f30b739eb52bcbb33a02fe62ceef5
From: Pavel Emelyanov @ 2008-01-31 10:27 UTC (permalink / raw)
To: Adrian Bunk, David S. Miller; +Cc: netdev, linux-kernel, Denis Lunev
In-Reply-To: <20080130175508.GC29368@does.not.exist>
Adrian Bunk wrote:
> Commit 42a73808ed4f30b739eb52bcbb33a02fe62ceef5
> ("[RAW]: Consolidate proc interface.") did not only change raw6_seq_ops
> (including adding 3 EXPORT_SYMBOL_GPL's to net/ipv4/raw.c for accessing
> functions from there), it also removed the only user of raw6_seq_ops...
The commit is not strange it is wrong :( Sorry David, when I checked
the according proc files, I saw that both files show sockets, but
overlooked that the raw6 one shows the ipv4 part of the ipv6 socket.
Denis noticed that this morning and has already prepared a fix.
So please, do not revert the commit, the fix will be at your mailbox today.
Thanks, Adrian.
> cu
> Adrian
>
^ permalink raw reply
* Re: [2.6 patch] make net/802/tr.c:sysctl_tr_rif_timeout static
From: Pavel Emelyanov @ 2008-01-31 10:35 UTC (permalink / raw)
To: Adrian Bunk; +Cc: David S. Miller, netdev, linux-kernel
In-Reply-To: <20080130200250.GB29368@does.not.exist>
Adrian Bunk wrote:
> sysctl_tr_rif_timeout can now become static.
>
> Signed-off-by: Adrian Bunk <bunk@kernel.org>
Acked-by: Pavel Emelyanov <xemul@openvz.org>
> ---
> e5accd81b924224d40a3f38204c08cf6896ed79b
> diff --git a/net/802/tr.c b/net/802/tr.c
> index 3f16b17..18c6647 100644
> --- a/net/802/tr.c
> +++ b/net/802/tr.c
> @@ -76,7 +76,7 @@ static DEFINE_SPINLOCK(rif_lock);
>
> static struct timer_list rif_timer;
>
> -int sysctl_tr_rif_timeout = 60*10*HZ;
> +static int sysctl_tr_rif_timeout = 60*10*HZ;
>
> static inline unsigned long rif_hash(const unsigned char *addr)
> {
>
>
^ permalink raw reply
* Re: [2.6 patch] make struct ipv4_devconf static
From: Pavel Emelyanov @ 2008-01-31 10:37 UTC (permalink / raw)
To: Adrian Bunk; +Cc: David S. Miller, netdev, linux-kernel
In-Reply-To: <20080130200254.GC29368@does.not.exist>
Adrian Bunk wrote:
> struct ipv4_devconf can now become static.
>
> Signed-off-by: Adrian Bunk <bunk@kernel.org>
Acked-by: Pavel Emelyanov <xemul@openvz.org>
> ---
>
> include/linux/inetdevice.h | 2 --
> net/ipv4/devinet.c | 2 +-
> 2 files changed, 1 insertion(+), 3 deletions(-)
>
> 20262a3317069b1bdbf2b37f4002fa5322445914
> diff --git a/include/linux/inetdevice.h b/include/linux/inetdevice.h
> index 8d9eaae..fc4e3db 100644
> --- a/include/linux/inetdevice.h
> +++ b/include/linux/inetdevice.h
> @@ -17,8 +17,6 @@ struct ipv4_devconf
> DECLARE_BITMAP(state, __NET_IPV4_CONF_MAX - 1);
> };
>
> -extern struct ipv4_devconf ipv4_devconf;
> -
> struct in_device
> {
> struct net_device *dev;
> diff --git a/net/ipv4/devinet.c b/net/ipv4/devinet.c
> index 21f71bf..5ab5acc 100644
> --- a/net/ipv4/devinet.c
> +++ b/net/ipv4/devinet.c
> @@ -64,7 +64,7 @@
> #include <net/rtnetlink.h>
> #include <net/net_namespace.h>
>
> -struct ipv4_devconf ipv4_devconf = {
> +static struct ipv4_devconf ipv4_devconf = {
> .data = {
> [NET_IPV4_CONF_ACCEPT_REDIRECTS - 1] = 1,
> [NET_IPV4_CONF_SEND_REDIRECTS - 1] = 1,
>
>
^ permalink raw reply
* Re: [2.6 patch] make nf_ct_path[] static
From: Pavel Emelyanov @ 2008-01-31 10:38 UTC (permalink / raw)
To: Adrian Bunk
Cc: Patrick McHardy, David S. Miller, netfilter-devel, netdev,
linux-kernel
In-Reply-To: <20080130200259.GE29368@does.not.exist>
Adrian Bunk wrote:
> This patch makes the needlessly global nf_ct_path[] static.
>
> Signed-off-by: Adrian Bunk <bunk@kernel.org>
Acked-by: Pavel Emelyanov <xemul@openvz.org>
Thanks, Adrian!
> ---
> 6396fbcebe3eb61f7e6eb1a671920a515912b005
> diff --git a/net/netfilter/nf_conntrack_standalone.c b/net/netfilter/nf_conntrack_standalone.c
> index 696074a..5bd38a6 100644
> --- a/net/netfilter/nf_conntrack_standalone.c
> +++ b/net/netfilter/nf_conntrack_standalone.c
> @@ -380,7 +380,7 @@ static ctl_table nf_ct_netfilter_table[] = {
> { .ctl_name = 0 }
> };
>
> -struct ctl_path nf_ct_path[] = {
> +static struct ctl_path nf_ct_path[] = {
> { .procname = "net", .ctl_name = CTL_NET, },
> { }
> };
>
>
^ permalink raw reply
* Re: [PATCH net-2.6.25][NETNS]: Fix race between put_net() and netlink_kernel_create().
From: Pavel Emelyanov @ 2008-01-31 11:05 UTC (permalink / raw)
To: David Miller; +Cc: netdev
In-Reply-To: <20080130.193128.136832319.davem@davemloft.net>
David Miller wrote:
> From: Pavel Emelyanov <xemul@openvz.org>
> Date: Thu, 24 Jan 2008 16:15:13 +0300
>
>> The comment about "race free view of the set of network
>> namespaces" was a bit hasty. Look (there even can be only
>> one CPU, as discovered by Alexey Dobriyan and Denis Lunev):
> ...
>> Instead, I propose to crate the socket inside an init_net
>> namespace and then re-attach it to the desired one right
>> after the socket is created.
>>
>> After doing this, we also have to be careful on error paths
>> not to drop the reference on the namespace, we didn't get
>> the one on.
>>
>> Signed-off-by: Pavel Emelyanov <xemul@openvz.org>
>> Acked-by: Denis Lunev <den@openvz.org>
>
> Applied, thanks.
>
Thanks, David.
I have one more patch pending in netdev@ and some more to be sent
(cleanups, small fixes and net namespaces). Do I have to wait till
net-2.6.26, or can I start (re-)sending them while 2.6.25 merge
window is open?
Thanks,
Pavel
^ permalink raw reply
* [PATCH 0/3] [RAW]: proc output cleanups.
From: Denis V. Lunev @ 2008-01-31 11:32 UTC (permalink / raw)
To: David Miller; +Cc: Adrian Bunk, Linux Kernel Mailing List, netdev, devel, xemul
yesterday Adrian Bunk noticed, that the commit
commit 42a73808ed4f30b739eb52bcbb33a02fe62ceef5
Author: Pavel Emelyanov <xemul@openvz.org>
Date: Mon Nov 19 22:38:33 2007 -0800
is somewhat strange. Basically, the commit is obviously wrong as the
content of the /proc/net/raw6 is incorrect due to it.
This series of patches fixes original problem and slightly cleanups the
code around.
Signed-off-by: Denis V. Lunev <den@openvz.org>
^ permalink raw reply
* [PATCH 2/3] [RAW]: Cleanup IPv4 raw_seq_show.
From: Denis V. Lunev @ 2008-01-31 11:34 UTC (permalink / raw)
To: davem; +Cc: netdev, linux-kernel, devel, xemul, bunk, Denis V. Lunev
In-Reply-To: <47A1B1E4.2010804@sw.ru>
There is no need to use 128 bytes on the stack at all. Clean the code in
the IPv6 style.
Signed-off-by: Denis V. Lunev <den@openvz.org>
---
net/ipv4/raw.c | 24 +++++++-----------------
1 files changed, 7 insertions(+), 17 deletions(-)
diff --git a/net/ipv4/raw.c b/net/ipv4/raw.c
index 507cbfe..830f19e 100644
--- a/net/ipv4/raw.c
+++ b/net/ipv4/raw.c
@@ -927,7 +927,7 @@ void raw_seq_stop(struct seq_file *seq, void *v)
}
EXPORT_SYMBOL_GPL(raw_seq_stop);
-static __inline__ char *get_raw_sock(struct sock *sp, char *tmpbuf, int i)
+static void raw_sock_seq_show(struct seq_file *seq, struct sock *sp, int i)
{
struct inet_sock *inet = inet_sk(sp);
__be32 dest = inet->daddr,
@@ -935,33 +935,23 @@ static __inline__ char *get_raw_sock(struct sock *sp, char *tmpbuf, int i)
__u16 destp = 0,
srcp = inet->num;
- sprintf(tmpbuf, "%4d: %08X:%04X %08X:%04X"
+ seq_printf(seq, "%4d: %08X:%04X %08X:%04X"
" %02X %08X:%08X %02X:%08lX %08X %5d %8d %lu %d %p %d",
i, src, srcp, dest, destp, sp->sk_state,
atomic_read(&sp->sk_wmem_alloc),
atomic_read(&sp->sk_rmem_alloc),
0, 0L, 0, sock_i_uid(sp), 0, sock_i_ino(sp),
atomic_read(&sp->sk_refcnt), sp, atomic_read(&sp->sk_drops));
- return tmpbuf;
}
-#define TMPSZ 128
-
static int raw_seq_show(struct seq_file *seq, void *v)
{
- char tmpbuf[TMPSZ+1];
-
if (v == SEQ_START_TOKEN)
- seq_printf(seq, "%-*s\n", TMPSZ-1,
- " sl local_address rem_address st tx_queue "
- "rx_queue tr tm->when retrnsmt uid timeout "
- "inode drops");
- else {
- struct raw_iter_state *state = raw_seq_private(seq);
-
- seq_printf(seq, "%-*s\n", TMPSZ-1,
- get_raw_sock(v, tmpbuf, state->bucket));
- }
+ seq_printf(seq, " sl local_address rem_address st tx_queue "
+ "rx_queue tr tm->when retrnsmt uid timeout "
+ "inode drops\n");
+ else
+ raw_sock_seq_show(seq, v, raw_seq_private(seq)->bucket);
return 0;
}
--
1.5.3.rc5
^ permalink raw reply related
* [PATCH 3/3] [RAW]: Wrong content of the /proc/net/raw6.
From: Denis V. Lunev @ 2008-01-31 11:34 UTC (permalink / raw)
To: davem; +Cc: netdev, linux-kernel, devel, xemul, bunk, Denis V. Lunev
In-Reply-To: <47A1B1E4.2010804@sw.ru>
The address of IPv6 raw sockets was shown in the wrong format, from IPv4 ones.
The problem has been introduced by the
commit 42a73808ed4f30b739eb52bcbb33a02fe62ceef5
Author: Pavel Emelyanov <xemul@openvz.org>
Date: Mon Nov 19 22:38:33 2007 -0800
Thanks to Adrian Bunk who originally noticed the problem.
Signed-off-by: Denis V. Lunev <den@openvz.org>
---
include/net/raw.h | 3 ++-
net/ipv4/raw.c | 8 ++++----
net/ipv6/raw.c | 2 +-
3 files changed, 7 insertions(+), 6 deletions(-)
diff --git a/include/net/raw.h b/include/net/raw.h
index c7ea7a2..1828f81 100644
--- a/include/net/raw.h
+++ b/include/net/raw.h
@@ -48,7 +48,8 @@ struct raw_iter_state {
void *raw_seq_start(struct seq_file *seq, loff_t *pos);
void *raw_seq_next(struct seq_file *seq, void *v, loff_t *pos);
void raw_seq_stop(struct seq_file *seq, void *v);
-int raw_seq_open(struct inode *ino, struct file *file, struct raw_hashinfo *h);
+int raw_seq_open(struct inode *ino, struct file *file,
+ struct raw_hashinfo *h, const struct seq_operations *ops);
#endif
diff --git a/net/ipv4/raw.c b/net/ipv4/raw.c
index 830f19e..a3002fe 100644
--- a/net/ipv4/raw.c
+++ b/net/ipv4/raw.c
@@ -962,13 +962,13 @@ static const struct seq_operations raw_seq_ops = {
.show = raw_seq_show,
};
-int raw_seq_open(struct inode *ino, struct file *file, struct raw_hashinfo *h)
+int raw_seq_open(struct inode *ino, struct file *file,
+ struct raw_hashinfo *h, const struct seq_operations *ops)
{
int err;
struct raw_iter_state *i;
- err = seq_open_net(ino, file, &raw_seq_ops,
- sizeof(struct raw_iter_state));
+ err = seq_open_net(ino, file, ops, sizeof(struct raw_iter_state));
if (err < 0)
return err;
@@ -980,7 +980,7 @@ EXPORT_SYMBOL_GPL(raw_seq_open);
static int raw_v4_seq_open(struct inode *inode, struct file *file)
{
- return raw_seq_open(inode, file, &raw_v4_hashinfo);
+ return raw_seq_open(inode, file, &raw_v4_hashinfo, &raw_seq_ops);
}
static const struct file_operations raw_seq_fops = {
diff --git a/net/ipv6/raw.c b/net/ipv6/raw.c
index a2cf499..8897ccf 100644
--- a/net/ipv6/raw.c
+++ b/net/ipv6/raw.c
@@ -1262,7 +1262,7 @@ static const struct seq_operations raw6_seq_ops = {
static int raw6_seq_open(struct inode *inode, struct file *file)
{
- return raw_seq_open(inode, file, &raw_v6_hashinfo);
+ return raw_seq_open(inode, file, &raw_v6_hashinfo, &raw6_seq_ops);
}
static const struct file_operations raw6_seq_fops = {
--
1.5.3.rc5
^ permalink raw reply related
* [PATCH 1/3] [RAW]: Family check in the /proc/net/raw[6] is extra.
From: Denis V. Lunev @ 2008-01-31 11:34 UTC (permalink / raw)
To: davem; +Cc: netdev, linux-kernel, devel, xemul, bunk, Denis V. Lunev
In-Reply-To: <47A1B1E4.2010804@sw.ru>
Different hashtables are used for IPv6 and IPv4 raw sockets, so no need to
check the socket family in the iterator over hashtables. Clean this out.
Signed-off-by: Denis V. Lunev <den@openvz.org>
---
include/net/raw.h | 4 +---
net/ipv4/raw.c | 12 ++++--------
net/ipv6/raw.c | 2 +-
3 files changed, 6 insertions(+), 12 deletions(-)
diff --git a/include/net/raw.h b/include/net/raw.h
index cca81d8..c7ea7a2 100644
--- a/include/net/raw.h
+++ b/include/net/raw.h
@@ -41,7 +41,6 @@ extern void raw_proc_exit(void);
struct raw_iter_state {
struct seq_net_private p;
int bucket;
- unsigned short family;
struct raw_hashinfo *h;
};
@@ -49,8 +48,7 @@ struct raw_iter_state {
void *raw_seq_start(struct seq_file *seq, loff_t *pos);
void *raw_seq_next(struct seq_file *seq, void *v, loff_t *pos);
void raw_seq_stop(struct seq_file *seq, void *v);
-int raw_seq_open(struct inode *ino, struct file *file, struct raw_hashinfo *h,
- unsigned short family);
+int raw_seq_open(struct inode *ino, struct file *file, struct raw_hashinfo *h);
#endif
diff --git a/net/ipv4/raw.c b/net/ipv4/raw.c
index f863c3d..507cbfe 100644
--- a/net/ipv4/raw.c
+++ b/net/ipv4/raw.c
@@ -862,8 +862,7 @@ static struct sock *raw_get_first(struct seq_file *seq)
struct hlist_node *node;
sk_for_each(sk, node, &state->h->ht[state->bucket])
- if (sk->sk_net == state->p.net &&
- sk->sk_family == state->family)
+ if (sk->sk_net == state->p.net)
goto found;
}
sk = NULL;
@@ -879,8 +878,7 @@ static struct sock *raw_get_next(struct seq_file *seq, struct sock *sk)
sk = sk_next(sk);
try_again:
;
- } while (sk && sk->sk_net != state->p.net &&
- sk->sk_family != state->family);
+ } while (sk && sk->sk_net != state->p.net);
if (!sk && ++state->bucket < RAW_HTABLE_SIZE) {
sk = sk_head(&state->h->ht[state->bucket]);
@@ -974,8 +972,7 @@ static const struct seq_operations raw_seq_ops = {
.show = raw_seq_show,
};
-int raw_seq_open(struct inode *ino, struct file *file, struct raw_hashinfo *h,
- unsigned short family)
+int raw_seq_open(struct inode *ino, struct file *file, struct raw_hashinfo *h)
{
int err;
struct raw_iter_state *i;
@@ -987,14 +984,13 @@ int raw_seq_open(struct inode *ino, struct file *file, struct raw_hashinfo *h,
i = raw_seq_private((struct seq_file *)file->private_data);
i->h = h;
- i->family = family;
return 0;
}
EXPORT_SYMBOL_GPL(raw_seq_open);
static int raw_v4_seq_open(struct inode *inode, struct file *file)
{
- return raw_seq_open(inode, file, &raw_v4_hashinfo, PF_INET);
+ return raw_seq_open(inode, file, &raw_v4_hashinfo);
}
static const struct file_operations raw_seq_fops = {
diff --git a/net/ipv6/raw.c b/net/ipv6/raw.c
index d61c63d..a2cf499 100644
--- a/net/ipv6/raw.c
+++ b/net/ipv6/raw.c
@@ -1262,7 +1262,7 @@ static const struct seq_operations raw6_seq_ops = {
static int raw6_seq_open(struct inode *inode, struct file *file)
{
- return raw_seq_open(inode, file, &raw_v6_hashinfo, PF_INET6);
+ return raw_seq_open(inode, file, &raw_v6_hashinfo);
}
static const struct file_operations raw6_seq_fops = {
--
1.5.3.rc5
^ permalink raw reply related
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox