* Re: [PATCH] compile error in ieee80211_ioctl.c
From: John W. Linville @ 2006-04-26 20:49 UTC (permalink / raw)
To: Jiri Benc; +Cc: Alex Davis, netdev
In-Reply-To: <20060426183734.4ac5dc6c@griffin.suse.cz>
On Wed, Apr 26, 2006 at 06:37:34PM +0200, Jiri Benc wrote:
> On Wed, 26 Apr 2006 09:29:46 -0700 (PDT), Alex Davis wrote:
> > Here is an updated patch which addresses Randy's issues. I'm currently running
> > this with no problems:
> > [...]
> > +module_param(ieee80211_regdom, int, 0666);
> > MODULE_PARM_DESC(ieee80211_regdom, "IEEE 802.11 regulatory domain; 64=MKK");
>
> NAK. Those parameters should not be writable yet.
>
> Please see
> http://marc.theaimsgroup.com/?l=linux-netdev&m=114565040832451&w=2 for
> the correct patch (hopefully John will pull it soon).
This has now been addressed.
Thanks!
John
--
John W. Linville
linville@tuxdriver.com
^ permalink raw reply
* Re: Fw: Bug: PPP dropouts in >=2.6.16
From: Sven Schuster @ 2006-04-26 21:25 UTC (permalink / raw)
To: Nuri Jawad; +Cc: Andi Kleen, Jesse Brandeburg, Andrew Morton, netdev
In-Reply-To: <Pine.LNX.4.64.0604260126020.12542@pc>
[-- Attachment #1: Type: text/plain, Size: 931 bytes --]
On Wed, Apr 26, 2006 at 02:36:18AM +0200, Nuri Jawad told us:
> Did you create a high load on the system in the manner I described?
> The bug once only appeared after about 6 hours here when line + CPU had
> been mostly idle. But that was the longest time between failures. Can you
> test with one of the 2.6.16 kernels I tried (latest was .9)? Can't say
Unfortunately it seems like plain 2.6.16.x doesn't like the ide
controller on my (VIA) mainboard, I'm getting I/O errrors on hda
when booting this kernel (but hard drive works ok with -mm) :-(
actually I haven't been running a plain stable kernel for a while,
I've been running -mm kernels for ages...
Sven
> for sure if CPU load is a factor, load on the connection seems to be.
--
Linux zion.homelinux.com 2.6.17-rc1-mm1_31 #31 Sat Apr 8 16:18:23 CEST 2006 i686 athlon i386 GNU/Linux
23:16:15 up 3:58, 2 users, load average: 0.83, 0.82, 0.79
[-- Attachment #2: Type: application/pgp-signature, Size: 191 bytes --]
^ permalink raw reply
* Re: [PATCH 0/17] d80211 patches
From: Ivo van Doorn @ 2006-04-26 21:27 UTC (permalink / raw)
To: John W. Linville; +Cc: Jiri Benc, Michael Buesch, netdev
In-Reply-To: <20060426193907.GB7922@tuxdriver.com>
[-- Attachment #1: Type: text/plain, Size: 1147 bytes --]
On Wednesday 26 April 2006 21:39, John W. Linville wrote:
> On Fri, Apr 21, 2006 at 10:52:10PM +0200, Jiri Benc wrote:
> > On Fri, 21 Apr 2006 22:52:08 +0200, Michael Buesch wrote:
> > > Can you please send your hacky patch for the bcm43xx
> > > to me, so I can come up with a clean one?
> >
> > Sure, actually I planned to do it in a few minutes :-)
>
> Hacky or not, I'm applying this patch to keep the bcm43xx driver
> from breaking. I don't suppose you have a patch for the rt2x00 driver?
Hi,
I had promised to create this patch before, but due to shortage of time
hadn't managed to do so. At the moment a compatibility fix for the updates
is already present in our CVS tree.
I am already working on a patch series to send for wireless-dev to bring rt2x00
up to date. The change for compatibility for the stack is update is also amongst the
patches. I am still working on the final patches for the series, but will be able to
send all (+/- 30 patches) tomorow.
After that I hope to send new patches for rt2x00 on a more regular basis,
to prevent large patch series like this one to be send in a single day.
IvD
[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]
^ permalink raw reply
* Re: tune back idle cwnd closing?
From: David S. Miller @ 2006-04-26 21:45 UTC (permalink / raw)
To: jheffner; +Cc: zach.brown, netdev
In-Reply-To: <444E31D9.1010705@psc.edu>
From: John Heffner <jheffner@psc.edu>
Date: Tue, 25 Apr 2006 10:27:37 -0400
> Yours is the first complaint of this kind I recall seeing, but I've
> expected for a while someone would have this type of problem. RFC2861
> seems conceptually nice at first, but there are a few things about it
> that bother me. One thing in particular is that a naturally bursty
> application (like yours) will actually perform better by padding its
> connection with junk data whenever it doesn't have real data to send.
> Or equivalently, it's punished for not sending data when it doesn't need
> to. I also think it may not do much good when there are connections
> with significantly different RTTs.
>
> Given that RFC2681 is Experimental (and I'm not aware of any current
> efforts in the IETF to push it to the standard track), IHMO it would not
> be inappropriate to make this behavior controlled via sysctl.
I have to respectfully disagree.
This is the price you pay when the network's congestion is being
measured by probing, information becomes stale over time if you don't
send any probes.
And this change of congestion state is real and happens frequently for
most end to end users.
When you're bursty application is not sending, other flows can take up
the pipe space you are not using, and you must reprobe to figure that
out.
^ permalink raw reply
* Re: Fw: Bug: PPP dropouts in >=2.6.16
From: Andrew Morton @ 2006-04-26 22:07 UTC (permalink / raw)
To: Sven Schuster; +Cc: lkml, ak, jesse.brandeburg, netdev
In-Reply-To: <20060426212542.GA9287@zion.homelinux.com>
Sven Schuster <schuster.sven@gmx.de> wrote:
>
> On Wed, Apr 26, 2006 at 02:36:18AM +0200, Nuri Jawad told us:
> > Did you create a high load on the system in the manner I described?
> > The bug once only appeared after about 6 hours here when line + CPU had
> > been mostly idle. But that was the longest time between failures. Can you
> > test with one of the 2.6.16 kernels I tried (latest was .9)? Can't say
>
> Unfortunately it seems like plain 2.6.16.x doesn't like the ide
> controller on my (VIA) mainboard, I'm getting I/O errrors on hda
> when booting this kernel (but hard drive works ok with -mm) :-(
> actually I haven't been running a plain stable kernel for a while,
> I've been running -mm kernels for ages...
>
So there's something in -mm which fixes your kernel? It's usually the
other way around ;)
And it sounds like something which has been in -mm for a long time, so it
might not be a patch which I was planning on sending upstream.
Can you think of a way in which we can identify which patch does the good
deed?
^ permalink raw reply
* [RFC] e1000 performance patch
From: Robin Humble @ 2006-04-26 22:13 UTC (permalink / raw)
To: netdev
[-- Attachment #1: Type: text/plain, Size: 2298 bytes --]
[I sent this to the e1000-devel folks, and they suggested netdev might
have opinions too. the below text has changed a little bit to reflect
feedback from Auke Kok]
attached is a small patch for e1000 that dynamically changes Interrupt
Throttle Rate for best performance - both latency and bandwidth.
it makes e1000 look really good on netpipe with a ~28 us latency and
890 Mbit/s bandwidth.
the basic idea is that high InterruptThrottleRate (~200k) is best for
small messages, whilst low ITR (~15k) is best for large messages.
leaving the ITR high for large messages burns outrageous amounts of cpu,
and any less than ~15k ITR is bad for bandwidth.
so this patch creates a new "performance dynamic" mode
InterruptThrottleRate=2 (2,2 for dual NICS)
which changes the ITR on the fly. the patch is based on the existing
"dynamic" mode (ITR=1) which seems to be optimised for low cpu usage
with little concern for performance.
hopefully the thresholds chosen for ITR changeovers will be ok on other
people's hardware too, but I really have no idea how universal it'll be.
we've been running it for a few months on our cluster and it appears stable.
10M 20M 100M as thresholds for changing between the 200k 90k 30 15k ITRs
were set pretty much by eye - by doing a bunch of netpipe runs and
trying to minimise cpu usage (ITR) for a target latency/bandwidth.
I've done an analysis of performance on this page:
http://www.cita.utoronto.ca/mediawiki/index.php/E1000_performance_patch
our hardware details are there too.
there's also a link to another analysis of how the patch affects routing
performance and cpu usage (surprisingly better).
despite the netpipe improvements, I haven't seen much in the way of real
world code differences (either +ve or -ve) from a regular 15k ITR. I've
seen an improvement in one code, and a slight degradation (~1%) in HPL
(top500.org benchmark). it should probably make the most difference for
codes that consistantly send small (< 1k) messages.
one possible improvement would be if the watchdog routine was called
more than once every 2 seconds - that would allow the ITR to adapt more
often.
ideally (I think) for traffic with mixed packet sizes the ITR would be
adapted 100's of times a second, but I'm not sure how practical that is.
cheers,
robin
[-- Attachment #2: rjh-performance-e1000-7.0.33.patch --]
[-- Type: text/plain, Size: 3074 bytes --]
diff -ru e1000-7.0.33/src/e1000_main.c e1000-7.0.33-rjh-performance/src/e1000_main.c
--- e1000-7.0.33/src/e1000_main.c 2006-02-03 16:53:41.000000000 -0500
+++ e1000-7.0.33-rjh-performance/src/e1000_main.c 2006-04-01 21:44:21.000000000 -0500
@@ -1732,7 +1732,7 @@
if (hw->mac_type >= e1000_82540) {
E1000_WRITE_REG(hw, RADV, adapter->rx_abs_int_delay);
- if (adapter->itr > 1)
+ if (adapter->itr > 2)
E1000_WRITE_REG(hw, ITR,
1000000000 / (adapter->itr * 256));
}
@@ -2394,17 +2394,30 @@
}
}
- /* Dynamic mode for Interrupt Throttle Rate (ITR) */
- if (adapter->hw.mac_type >= e1000_82540 && adapter->itr == 1) {
- /* Symmetric Tx/Rx gets a reduced ITR=2000; Total
- * asymmetrical Tx or Rx gets ITR=8000; everyone
- * else is between 2000-8000. */
- uint32_t goc = (adapter->gotcl + adapter->gorcl) / 10000;
- uint32_t dif = (adapter->gotcl > adapter->gorcl ?
- adapter->gotcl - adapter->gorcl :
- adapter->gorcl - adapter->gotcl) / 10000;
- uint32_t itr = goc > 0 ? (dif * 6000 / goc + 2000) : 8000;
- E1000_WRITE_REG(&adapter->hw, ITR, 1000000000 / (itr * 256));
+ /* Dynamic modes for Interrupt Throttle Rate (ITR) */
+ if (adapter->hw.mac_type >= e1000_82540) {
+ if (adapter->itr == 1) {
+ /* Symmetric Tx/Rx gets a reduced ITR=2000; Total
+ * asymmetrical Tx or Rx gets ITR=8000; everyone
+ * else is between 2000-8000. */
+ uint32_t goc = (adapter->gotcl + adapter->gorcl) / 10000;
+ uint32_t dif = (adapter->gotcl > adapter->gorcl ?
+ adapter->gotcl - adapter->gorcl :
+ adapter->gorcl - adapter->gotcl) / 10000;
+ uint32_t itr = goc > 0 ? (dif * 6000 / goc + 2000) : 8000;
+ E1000_WRITE_REG(&adapter->hw, ITR, 1000000000 / (itr * 256));
+ }
+ else if (adapter->itr == 2) { /* low latency, high bandwidth, moderate cpu usage */
+ /* range from high itr at low cl, to low itr at high cl
+ * < 10M => large itr
+ * 10M to 20M => 90k itr
+ * 20M to 100M => 30k itr
+ * > 100M => 15k itr */
+ uint32_t goc = max(adapter->gotcl, adapter->gorcl) / 1000000;
+ uint32_t itr = goc > 10 ? (goc > 20 ? (goc > 100 ? 15000: 30000): 90000): 200000;
+ /* DPRINTK(PROBE, INFO, "e1000 ITR %d - [tr]cl min/ave/max %dm / %dm/ %dm\n", itr, min(adapter->gotcl, adapter->gorcl) / 1000000, (adapter->gotcl + adapter->gorcl) / 2000000, max(adapter->gotcl, adapter->gorcl) / 1000000 ); */
+ E1000_WRITE_REG(&adapter->hw, ITR, 1000000000 / (itr * 256));
+ }
}
/* Cause software interrupt to ensure rx ring is cleaned */
diff -ru e1000-7.0.33/src/e1000_param.c e1000-7.0.33-rjh-performance/src/e1000_param.c
--- e1000-7.0.33/src/e1000_param.c 2006-02-03 16:53:41.000000000 -0500
+++ e1000-7.0.33-rjh-performance/src/e1000_param.c 2006-03-29 21:42:00.000000000 -0500
@@ -538,6 +538,10 @@
DPRINTK(PROBE, INFO, "%s set to dynamic mode\n",
opt.name);
break;
+ case 2:
+ DPRINTK(PROBE, INFO, "%s set to performance dynamic mode\n",
+ opt.name);
+ break;
default:
e1000_validate_option(&adapter->itr, &opt,
adapter);
^ permalink raw reply
* Re: tune back idle cwnd closing?
From: Rick Jones @ 2006-04-26 22:16 UTC (permalink / raw)
To: David S. Miller; +Cc: jheffner, zach.brown, netdev
In-Reply-To: <20060426.144540.39973302.davem@davemloft.net>
> When you're bursty application is not sending, other flows can take up
> the pipe space you are not using, and you must reprobe to figure that
> out.
If the "restarted" connection does normal slow-start, one of two things
will happen yes? Either it will grow its cwnd to >= the receiver's
window, or it will have to stop before then because it triggered a
packet loss.
In the first case, seems it would have been just as good to let the
connection burst.
In the second case, is the effect on other connections really any better
than if the connection just started-up from where it was before?
BTW, is the RFC 2681? I looked that one up on ietf.org and the RFC by
that number was a different beast entirely - at least at a very quick
glance.
rick jones
^ permalink raw reply
* ixp2000: handle enp2611s with two gigabit ports
From: Lennert Buytenhek @ 2006-04-26 22:24 UTC (permalink / raw)
To: jgarzik; +Cc: netdev
The ixp2000 driver for the enp2611 was developed on a board with
three gigabit ports, but some enp2611 models only have two ports
(and only one onboard PM3386.) The current driver assumes there
are always three ports and so it doesn't work on the two-port
version of the board at all.
This patch adds a bit of logic to the enp2611 driver to limit the
number of ports to 2 if the second PM3386 isn't detected.
Signed-off-by: Lennert Buytenhek <buytenh@wantstofly.org>
diff -urN linux-2.6.17-rc2.orig/drivers/net/ixp2000/enp2611.c linux-2.6.17-rc2/drivers/net/ixp2000/enp2611.c
--- linux-2.6.17-rc2.orig/drivers/net/ixp2000/enp2611.c 2006-04-19 21:54:58.000000000 +0200
+++ linux-2.6.17-rc2/drivers/net/ixp2000/enp2611.c 2006-04-19 22:03:05.000000000 +0200
@@ -149,6 +149,8 @@
int status;
dev = nds[i];
+ if (dev == NULL)
+ continue;
status = pm3386_is_link_up(i);
if (status && !netif_carrier_ok(dev)) {
@@ -191,6 +193,7 @@
static int __init enp2611_init_module(void)
{
+ int ports;
int i;
if (!machine_is_enp2611())
@@ -199,7 +202,8 @@
caleb_reset();
pm3386_reset();
- for (i = 0; i < 3; i++) {
+ ports = pm3386_port_count();
+ for (i = 0; i < ports; i++) {
nds[i] = ixpdev_alloc(i, sizeof(struct enp2611_ixpdev_priv));
if (nds[i] == NULL) {
while (--i >= 0)
@@ -215,8 +219,8 @@
ixp2400_msf_init(&enp2611_msf_parameters);
- if (ixpdev_init(3, nds, enp2611_set_port_admin_status)) {
- for (i = 0; i < 3; i++)
+ if (ixpdev_init(ports, nds, enp2611_set_port_admin_status)) {
+ for (i = 0; i < ports; i++)
free_netdev(nds[i]);
return -EINVAL;
}
@@ -236,8 +240,10 @@
del_timer_sync(&link_check_timer);
ixpdev_deinit();
- for (i = 0; i < 3; i++)
- free_netdev(nds[i]);
+ for (i = 0; i < 3; i++) {
+ if (nds[i] != NULL)
free_netdev(nds[i]);
+ }
}
module_init(enp2611_init_module);
diff -urN linux-2.6.17-rc2.orig/drivers/net/ixp2000/pm3386.c linux-2.6.17-rc2/drivers/net/ixp2000/pm3386.c
--- linux-2.6.17-rc2.orig/drivers/net/ixp2000/pm3386.c 2006-03-20 06:53:29.000000000 +0100
+++ linux-2.6.17-rc2/drivers/net/ixp2000/pm3386.c 2006-04-19 22:01:39.000000000 +0200
@@ -86,40 +86,53 @@
pm3386_reg_write(port >> 1, reg, value);
}
+int pm3386_secondary_present(void)
+{
+ return pm3386_reg_read(1, 0) == 0x3386;
+}
void pm3386_reset(void)
{
u8 mac[3][6];
+ int secondary;
+
+ secondary = pm3386_secondary_present();
/* Save programmed MAC addresses. */
pm3386_get_mac(0, mac[0]);
pm3386_get_mac(1, mac[1]);
- pm3386_get_mac(2, mac[2]);
+ if (secondary)
+ pm3386_get_mac(2, mac[2]);
/* Assert analog and digital reset. */
pm3386_reg_write(0, 0x002, 0x0060);
- pm3386_reg_write(1, 0x002, 0x0060);
+ if (secondary)
+ pm3386_reg_write(1, 0x002, 0x0060);
mdelay(1);
/* Deassert analog reset. */
pm3386_reg_write(0, 0x002, 0x0062);
- pm3386_reg_write(1, 0x002, 0x0062);
+ if (secondary)
+ pm3386_reg_write(1, 0x002, 0x0062);
mdelay(10);
/* Deassert digital reset. */
pm3386_reg_write(0, 0x002, 0x0063);
- pm3386_reg_write(1, 0x002, 0x0063);
+ if (secondary)
+ pm3386_reg_write(1, 0x002, 0x0063);
mdelay(10);
/* Restore programmed MAC addresses. */
pm3386_set_mac(0, mac[0]);
pm3386_set_mac(1, mac[1]);
- pm3386_set_mac(2, mac[2]);
+ if (secondary)
+ pm3386_set_mac(2, mac[2]);
/* Disable carrier on all ports. */
pm3386_set_carrier(0, 0);
pm3386_set_carrier(1, 0);
- pm3386_set_carrier(2, 0);
+ if (secondary)
+ pm3386_set_carrier(2, 0);
}
static u16 swaph(u16 x)
@@ -127,6 +140,11 @@
return ((x << 8) | (x >> 8)) & 0xffff;
}
+int pm3386_port_count(void)
+{
+ return 2 + !!pm3386_secondary_present();
+}
+
void pm3386_init_port(int port)
{
int pm = port >> 1;
diff -urN linux-2.6.17-rc2.orig/drivers/net/ixp2000/pm3386.h linux-2.6.17-rc2/drivers/net/ixp2000/pm3386.h
--- linux-2.6.17-rc2.orig/drivers/net/ixp2000/pm3386.h 2006-03-20 06:53:29.000000000 +0100
+++ linux-2.6.17-rc2/drivers/net/ixp2000/pm3386.h 2006-04-19 21:56:55.000000000 +0200
@@ -13,6 +13,7 @@
#define __PM3386_H
void pm3386_reset(void);
+int pm3386_port_count(void);
void pm3386_init_port(int port);
void pm3386_get_mac(int port, u8 *mac);
void pm3386_set_mac(int port, u8 *mac);
^ permalink raw reply
* Re: [RFC] e1000 performance patch
From: Rick Jones @ 2006-04-26 22:26 UTC (permalink / raw)
To: Robin Humble; +Cc: netdev
In-Reply-To: <20060426221353.GA22143@lemming.cita.utoronto.ca>
Robin Humble wrote:
> [I sent this to the e1000-devel folks, and they suggested netdev might
> have opinions too. the below text has changed a little bit to reflect
> feedback from Auke Kok]
>
> attached is a small patch for e1000 that dynamically changes Interrupt
> Throttle Rate for best performance - both latency and bandwidth.
> it makes e1000 look really good on netpipe with a ~28 us latency and
> 890 Mbit/s bandwidth.
>
> the basic idea is that high InterruptThrottleRate (~200k) is best for
> small messages,
Best for small numbers of small messages? If one is looking to have
high aggregate small packet rates, the higher throttle rate may degrade
the peak PPS one can achieve.
> I've done an analysis of performance on this page:
> http://www.cita.utoronto.ca/mediawiki/index.php/E1000_performance_patch
> our hardware details are there too.
> there's also a link to another analysis of how the patch affects routing
> performance and cpu usage (surprisingly better).
>
> despite the netpipe improvements, I haven't seen much in the way of real
> world code differences (either +ve or -ve) from a regular 15k ITR. I've
> seen an improvement in one code, and a slight degradation (~1%) in HPL
> (top500.org benchmark). it should probably make the most difference for
> codes that consistantly send small (< 1k) messages.
Tweaking interrupt coalescing parameters was rather common in SPECweb
benchmarking. If you examine some of the results on www.spec.org you may
see examples. IIRC the last ones I submitted used an interrupt throttle
rate of something like 700. It was a small but non-trivial percentage
difference in the SPECweb result.
rick jones
It is a bit rough/messy as a writeup, but here is what I've seen wrt the
latency vs throughput tradeoffs:
ftp://ftp.cup.hp.com/dist/networking/briefs/nic_latency_vs_tput.txt
^ permalink raw reply
* Re: [RFC] bridge: partial rtnetlink hooks
From: Francois Romieu @ 2006-04-26 22:24 UTC (permalink / raw)
To: Stephen Hemminger; +Cc: netdev
In-Reply-To: <20060426104521.44682924@localhost.localdomain>
Stephen Hemminger <shemminger@osdl.org> :
[...]
> --- /dev/null
> +++ bridge-2.6/net/bridge/br_netlink.c
[...]
> +static int br_fill_ifinfo(struct sk_buff *skb, const struct net_bridge_port *port,
> + u32 pid, u32 seq, int event, unsigned int flags)
> +{
[...]
> +nlmsg_failure:
> +rtattr_failure:
> +
> + skb_trim(skb, b - skb->data);
> + return -1;
return -EINVAL;
(see below)
> +}
> +
> +
> +void br_ifinfo_notify(int event, struct net_bridge_port *port)
> +{
> + struct sk_buff *skb;
> +
> + printk(KERN_DEBUG "bridge notify event=%d\n", event);
> + skb = alloc_skb(NLMSG_SPACE(sizeof(struct ifinfomsg) + 128),
> + GFP_ATOMIC);
> + if (!skb) {
> + netlink_set_err(rtnl, 0, RTNLGRP_BRIDGE_IFINFO, ENOBUFS);
> + return;
> + }
> + if (br_fill_ifinfo(skb, port, current->pid, 0, event, 0) < 0) {
> + kfree_skb(skb);
> + netlink_set_err(rtnl, 0, RTNLGRP_BRIDGE_IFINFO, EINVAL);
> + return;
> + }
> + NETLINK_CB(skb).dst_group = RTNLGRP_IPV6_IFINFO;
> + netlink_broadcast(rtnl, skb, 0, RTNLGRP_BRIDGE_IFINFO, GFP_ATOMIC);
> +}
void br_ifinfo_notify(int event, struct net_bridge_port *port)
{
struct sk_buff *skb;
int err = -ENOBUFS;
printk(KERN_DEBUG "bridge notify event=%d\n", event);
skb = alloc_skb(NLMSG_SPACE(sizeof(struct ifinfomsg) + 128),
GFP_ATOMIC);
if (!skb)
goto err_out;
err = br_fill_ifinfo(skb, port, current->pid, 0, event, 0);
if (unlikely((err < 0))
goto err_kfree;
NETLINK_CB(skb).dst_group = RTNLGRP_IPV6_IFINFO;
netlink_broadcast(rtnl, skb, 0, RTNLGRP_BRIDGE_IFINFO, GFP_ATOMIC);
return;
err_kfree:
kfree_skb(skb);
err_out:
netlink_set_err(rtnl, 0, RTNLGRP_BRIDGE_IFINFO, -err);
}
--
Ueimor
^ permalink raw reply
* Re: tune back idle cwnd closing?
From: Stephen Hemminger @ 2006-04-26 22:27 UTC (permalink / raw)
To: Rick Jones; +Cc: David S. Miller, jheffner, zach.brown, netdev
In-Reply-To: <444FF132.2080505@hp.com>
On Wed, 26 Apr 2006 15:16:18 -0700
Rick Jones <rick.jones2@hp.com> wrote:
> > When you're bursty application is not sending, other flows can take up
> > the pipe space you are not using, and you must reprobe to figure that
> > out.
>
> If the "restarted" connection does normal slow-start, one of two things
> will happen yes? Either it will grow its cwnd to >= the receiver's
> window, or it will have to stop before then because it triggered a
> packet loss.
>
> In the first case, seems it would have been just as good to let the
> connection burst.
>
> In the second case, is the effect on other connections really any better
> than if the connection just started-up from where it was before?
>
> BTW, is the RFC 2681? I looked that one up on ietf.org and the RFC by
> that number was a different beast entirely - at least at a very quick
> glance.
>
> rick jones
> -
> To unsubscribe from this list: send the line "unsubscribe netdev" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
http://www.faqs.org/rfcs/rfc2861.html
Long periods when the sender is application-limited can lead to the
invalidation of the congestion window. During periods when the TCP
sender is network-limited, the value of the congestion window is
repeatedly "revalidated" by the successful transmission of a window
of data without loss. When the TCP sender is network-limited, there
is an incoming stream of acknowledgements that "clocks out" new data,
giving concrete evidence of recent available bandwidth in the
network. In contrast, during periods when the TCP sender is
application-limited, the estimate of available capacity represented
by the congestion window may become steadily less accurate over time.
In particular, capacity that had once been used by the network-
limited connection might now be used by other traffic.
^ permalink raw reply
* [resend PATCH 0/1] b44: fix ethtool settings support
From: Gary Zambrano @ 2006-04-26 15:31 UTC (permalink / raw)
To: jgarzik; +Cc: netdev
Please disregard the previous patch. This one is cleaner.
^ permalink raw reply
* [resend PATCH 1/1] b44: fix ethool link settings support
From: Gary Zambrano @ 2006-04-26 15:31 UTC (permalink / raw)
To: jgarzik; +Cc: netdev
The b44 driver support for setting link speed/duplex using ethtool was broken.
This patch contains the following driver changes for link settings:
-clear the b44 flags of the previous settings before setting the phy
-allow access to get/set_settings() if interface is down
-set "advertised autoneg" setting dependent on autoneg enabled
-return unknown speed/duplex if the interface is down
Signed-off-by: Gary Zambrano <zambrano@broadcom.com>
diff --git a/drivers/net/b44.c b/drivers/net/b44.c
index 3d30668..7b3dfc4 100644
--- a/drivers/net/b44.c
+++ b/drivers/net/b44.c
@@ -1612,8 +1612,6 @@ static int b44_get_settings(struct net_d
{
struct b44 *bp = netdev_priv(dev);
- if (!netif_running(dev))
- return -EAGAIN;
cmd->supported = (SUPPORTED_Autoneg);
cmd->supported |= (SUPPORTED_100baseT_Half |
SUPPORTED_100baseT_Full |
@@ -1641,6 +1639,12 @@ static int b44_get_settings(struct net_d
XCVR_INTERNAL : XCVR_EXTERNAL;
cmd->autoneg = (bp->flags & B44_FLAG_FORCE_LINK) ?
AUTONEG_DISABLE : AUTONEG_ENABLE;
+ if (cmd->autoneg == AUTONEG_ENABLE)
+ cmd->advertising |= ADVERTISED_Autoneg;
+ if (!netif_running(dev)){
+ cmd->speed = 0;
+ cmd->duplex = 0xff;
+ }
cmd->maxtxpkt = 0;
cmd->maxrxpkt = 0;
return 0;
@@ -1650,47 +1654,42 @@ static int b44_set_settings(struct net_d
{
struct b44 *bp = netdev_priv(dev);
- if (!netif_running(dev))
- return -EAGAIN;
-
- /* We do not support gigabit. */
- if (cmd->autoneg == AUTONEG_ENABLE) {
- if (cmd->advertising &
- (ADVERTISED_1000baseT_Half |
- ADVERTISED_1000baseT_Full))
- return -EINVAL;
- } else if ((cmd->speed != SPEED_100 &&
- cmd->speed != SPEED_10) ||
- (cmd->duplex != DUPLEX_HALF &&
- cmd->duplex != DUPLEX_FULL)) {
- return -EINVAL;
- }
-
spin_lock_irq(&bp->lock);
if (cmd->autoneg == AUTONEG_ENABLE) {
- bp->flags &= ~B44_FLAG_FORCE_LINK;
- bp->flags &= ~(B44_FLAG_ADV_10HALF |
+ bp->flags &= ~(B44_FLAG_FORCE_LINK |
+ B44_FLAG_100_BASE_T |
+ B44_FLAG_FULL_DUPLEX |
+ B44_FLAG_ADV_10HALF |
B44_FLAG_ADV_10FULL |
B44_FLAG_ADV_100HALF |
B44_FLAG_ADV_100FULL);
- if (cmd->advertising & ADVERTISE_10HALF)
- bp->flags |= B44_FLAG_ADV_10HALF;
- if (cmd->advertising & ADVERTISE_10FULL)
- bp->flags |= B44_FLAG_ADV_10FULL;
- if (cmd->advertising & ADVERTISE_100HALF)
- bp->flags |= B44_FLAG_ADV_100HALF;
- if (cmd->advertising & ADVERTISE_100FULL)
- bp->flags |= B44_FLAG_ADV_100FULL;
+ if (cmd->advertising & ADVERTISE_ALL) {
+ if (cmd->advertising & ADVERTISE_10HALF)
+ bp->flags |= B44_FLAG_ADV_10HALF;
+ if (cmd->advertising & ADVERTISE_10FULL)
+ bp->flags |= B44_FLAG_ADV_10FULL;
+ if (cmd->advertising & ADVERTISE_100HALF)
+ bp->flags |= B44_FLAG_ADV_100HALF;
+ if (cmd->advertising & ADVERTISE_100FULL)
+ bp->flags |= B44_FLAG_ADV_100FULL;
+ } else {
+ bp->flags |= (B44_FLAG_ADV_10HALF |
+ B44_FLAG_ADV_10FULL |
+ B44_FLAG_ADV_100HALF |
+ B44_FLAG_ADV_100FULL);
+ }
} else {
bp->flags |= B44_FLAG_FORCE_LINK;
+ bp->flags &= ~(B44_FLAG_100_BASE_T | B44_FLAG_FULL_DUPLEX);
if (cmd->speed == SPEED_100)
bp->flags |= B44_FLAG_100_BASE_T;
if (cmd->duplex == DUPLEX_FULL)
bp->flags |= B44_FLAG_FULL_DUPLEX;
}
- b44_setup_phy(bp);
+ if(netif_running(dev))
+ b44_setup_phy(bp);
spin_unlock_irq(&bp->lock);
^ permalink raw reply related
* Re: tune back idle cwnd closing?
From: David S. Miller @ 2006-04-26 22:33 UTC (permalink / raw)
To: rick.jones2; +Cc: jheffner, zach.brown, netdev
In-Reply-To: <444FF132.2080505@hp.com>
From: Rick Jones <rick.jones2@hp.com>
Date: Wed, 26 Apr 2006 15:16:18 -0700
> BTW, is the RFC 2681? I looked that one up on ietf.org and the RFC by
> that number was a different beast entirely - at least at a very quick
> glance.
Congestion window validation is the correct RFC.
^ permalink raw reply
* Re: [PATCH 1/3] Rough VJ Channel Implementation - vj_core.patch
From: David S. Miller @ 2006-04-26 22:35 UTC (permalink / raw)
To: caitlinb; +Cc: jeff, kelly, netdev, rusty
In-Reply-To: <54AD0F12E08D1541B826BE97C98F99F143AE7F@NT-SJCA-0751.brcm.ad.broadcom.com>
From: "Caitlin Bestler" <caitlinb@broadcom.com>
Date: Wed, 26 Apr 2006 13:20:50 -0700
> If you are dropping all packets from IP X, then how was the connection
> established? Obviously we are only dealing with connections that
> were established before the rule to drop all packets from IP X
> was created.
The problem is listening TCP connections that you don't
want anyone in the world to be able to connect to.
^ permalink raw reply
* Re: [PATCH 1/3] Rough VJ Channel Implementation - vj_core.patch
From: David S. Miller @ 2006-04-26 22:40 UTC (permalink / raw)
To: jeff; +Cc: caitlinb, kelly, netdev, rusty
In-Reply-To: <444FCE32.2010207@garzik.org>
From: Jeff Garzik <jeff@garzik.org>
Date: Wed, 26 Apr 2006 15:46:58 -0400
> Oh, there are plenty of examples of filtering within an established
> connection: input rules. I've seen "drop all packets from <these> IPs"
> type rules frequently. Victims of DoS use those kinds of rules to stop
> packets as early as possible.
Yes, good point, but this applies to listening connections.
We'll need to figure out a way to deal with this.
It occurs to me that for established connections, netfilter can simply
remove all matching entries from the netchannel lookup tables.
But that still leaves the thorny listening socket issue. This may
by itself make netfilter netchannel support important and that brings
up a lot of issues about classifier algorithms.
All of this I wanted to avoid as we start this work :-)
We can think about how to approach these other problems and start
with something simple meanwhile. That seems to me to be the best
approach moving forward.
It's important to start really simple else we'll just keep getting
bogged down in complexity and details and never implement anything.
^ permalink raw reply
* Re: [RFC] bridge: partial rtnetlink hooks
From: David S. Miller @ 2006-04-26 22:43 UTC (permalink / raw)
To: shemminger; +Cc: netdev
In-Reply-To: <20060426104521.44682924@localhost.localdomain>
From: Stephen Hemminger <shemminger@osdl.org>
Date: Wed, 26 Apr 2006 10:45:21 -0700
> +struct brifinfo {
> + __u8 state;
> + __u32 cost;
> +};
> +
Maybe put the __u32 first and explicitly pad out the 3
bytes after the __u8? Just to be safe.
I know you use an assignment initializer, so your current
code won't leak kernel data into userspace, but a safer
layout might help provide even more protection for future
code using this data structure.
^ permalink raw reply
* Re: tune back idle cwnd closing?
From: Rick Jones @ 2006-04-26 22:44 UTC (permalink / raw)
To: Stephen Hemminger; +Cc: David S. Miller, jheffner, zach.brown, netdev
In-Reply-To: <20060426152738.3a50d645@localhost.localdomain>
>>BTW, is the RFC 2681? I looked that one up on ietf.org and the RFC by
>>that number was a different beast entirely - at least at a very quick
>>glance.
>>
>>rick jones
>>-
>>To unsubscribe from this list: send the line "unsubscribe netdev" in
>>the body of a message to majordomo@vger.kernel.org
>>More majordomo info at http://vger.kernel.org/majordomo-info.html
>
>
> http://www.faqs.org/rfcs/rfc2861.html
thanks.
> Long periods when the sender is application-limited can lead to the
> invalidation of the congestion window. During periods when the TCP
> sender is network-limited, the value of the congestion window is
> repeatedly "revalidated" by the successful transmission of a window
> of data without loss. When the TCP sender is network-limited, there
> is an incoming stream of acknowledgements that "clocks out" new data,
> giving concrete evidence of recent available bandwidth in the
> network. In contrast, during periods when the TCP sender is
> application-limited, the estimate of available capacity represented
> by the congestion window may become steadily less accurate over time.
> In particular, capacity that had once been used by the network-
> limited connection might now be used by other traffic.
May, might, could... :)
What concerned me the most was section 5, where the experiments were for
dial-up connections and an interactive user then cat'ing a large file to
the screen. How often does someone "list a moderately large file"
without using less or more? And the bit about the second experiment
with the real modem bank not showing any difference in what the user
experienced because the bank had buffering was interesting. It suggests
(to me anyway) that perhaps the TCP receive window was too large for a
modem connection in the first place. Leaves me wondering what effect
Linux's moderated receive window would have on that experiment.
rick jones
^ permalink raw reply
* how to change classful netem loss probability?
From: George Nychis @ 2006-04-26 22:46 UTC (permalink / raw)
To: LARTC, netdev
Hi,
I am using netem to add loss and then adding another qdisc within netem
according to the wiki. Then i want to change the netem drop probability
without having to delete the qdisc and recreate it. I try it but I get
invalid argument:
thorium-ini hedpe # tc qdisc add dev ath0 root handle 1:0 netem drop 1%
thorium-ini hedpe # tc qdisc add dev ath0 parent 1:1 handle 10: xcp
capacity 54Mbit limit 500
thorium-ini hedpe # tc -s qdisc ls dev ath0
qdisc netem 1: limit 1000 loss 1%
Sent 0 bytes 0 pkts (dropped 0, overlimits 0)
qdisc xcp 10: parent 1:1 capacity 52734Kbit limit 500p
Sent 0 bytes 0 pkts (dropped 0, overlimits 0)
thorium-ini hedpe # tc qdisc change dev ath0 root handle 1:0 netem drop 1%
RTNETLINK answers: Invalid argument
thorium-ini hedpe # tc qdisc change dev ath0 root netem drop 1%
RTNETLINK answers: Invalid argument
any ideas?
Thanks!
George
^ permalink raw reply
* RE: [PATCH 1/3] Rough VJ Channel Implementation - vj_core.patch
From: Caitlin Bestler @ 2006-04-26 22:53 UTC (permalink / raw)
To: David S. Miller, jeff; +Cc: kelly, netdev, rusty
David S. Miller wrote:
> From: Jeff Garzik <jeff@garzik.org>
> Date: Wed, 26 Apr 2006 15:46:58 -0400
>
>> Oh, there are plenty of examples of filtering within an established
>> connection: input rules. I've seen "drop all packets from <these>
>> IPs" type rules frequently. Victims of DoS use those kinds of rules
>> to stop packets as early as possible.
>
> Yes, good point, but this applies to listening connections.
>
> We'll need to figure out a way to deal with this.
>
> It occurs to me that for established connections, netfilter
> can simply remove all matching entries from the netchannel lookup
> tables.
>
> But that still leaves the thorny listening socket issue.
> This may by itself make netfilter netchannel support
> important and that brings up a lot of issues about classifier
> algorithms.
>
> All of this I wanted to avoid as we start this work :-)
>
> We can think about how to approach these other problems and
> start with something simple meanwhile. That seems to me to
> be the best approach moving forward.
>
> It's important to start really simple else we'll just keep
> getting bogged down in complexity and details and never
> implement anything.
How does this sound?
The netchannel qualifiers should only deal with TCP packets
for established connections. Listens would continue to be
dealt with by the existing stack logic, vj_channelizing
only occurring when the the connection was accepted.
The vj_netchannel qualifiers would conceptually take place
before the netfilter rules (to avoid making deployment
of netchannels dependent on netfilter) but their creation
would have to be approved by netfilter (if netfiler was
active). Netfilter could also revoke vj_channel qualifiers.
If the rule is that "if a vj_netchannel rule exists then it
must be ok with netfilter" is actually very easy to implement.
During early development you simply tell the testers "hey,
don't set up any netchannels that netfilter would reject"
and defer implementing enforcement until after the netchannels
code actually works. After all, if it is isn't actually successfully
transmitting or receiving packets yet it can't really be acting
contrary to netfilter policy.
^ permalink raw reply
* Re: [PATCH 1/3] Rough VJ Channel Implementation - vj_core.patch
From: David S. Miller @ 2006-04-26 22:59 UTC (permalink / raw)
To: caitlinb; +Cc: jeff, kelly, netdev, rusty
In-Reply-To: <54AD0F12E08D1541B826BE97C98F99F143AEC1@NT-SJCA-0751.brcm.ad.broadcom.com>
From: "Caitlin Bestler" <caitlinb@broadcom.com>
Date: Wed, 26 Apr 2006 15:53:44 -0700
> The netchannel qualifiers should only deal with TCP packets
> for established connections. Listens would continue to be
> dealt with by the existing stack logic, vj_channelizing
> only occurring when the the connection was accepted.
I consider netchannel support for listening TCP sockets
to be absolutely essential.
^ permalink raw reply
* Re: tune back idle cwnd closing?
From: Zach Brown @ 2006-04-26 23:25 UTC (permalink / raw)
To: David S. Miller; +Cc: jheffner, netdev
In-Reply-To: <20060426.144540.39973302.davem@davemloft.net>
>> Given that RFC2681 is Experimental (and I'm not aware of any current
>> efforts in the IETF to push it to the standard track), IHMO it would not
>> be inappropriate to make this behavior controlled via sysctl.
>
> I have to respectfully disagree.
OK, thanks for taking the time to look at it.
- z
^ permalink raw reply
* Re: e1000_down and tx_timeout worker race cleaning the transmit buffers
From: Shaw @ 2006-04-27 0:14 UTC (permalink / raw)
To: Andy Gospodarek
Cc: Michael Chan, Herbert Xu, netdev, auke-jan.h.kok, davem, jgarzik
In-Reply-To: <bdfc5d6e0604211346n50b15f56g4ebc2fe5fe88a63a@mail.gmail.com>
[-- Attachment #1: Type: text/plain, Size: 1255 bytes --]
On 4/21/06, Andy Gospodarek <andy@greyhouse.net> wrote:
> On 4/21/06, Michael Chan <mchan@broadcom.com> wrote:
> > On Fri, 2006-04-21 at 16:01 -0400, Andy Gospodarek wrote:
> >
> > > I just hate to see extra resources used to solve problems that good
> > > coding can solve (not that my suggestion is necessarily a 'good' one),
> > > so I was trying to think of a way to resolve this without explicitly
> > > adding another workqueue.
> >
> > If you don't want to add another workqueue, then look at tg3, bnx2, and
> > one of the smc drivers on how to effectively wait for the driver's
> > workqueue task to finish without deadlocking with linkwatch_event.
> >
>
> I agree 100%. I just hope others can manage to figure that out too.
Ok, here's another attempt. The goal here is to serialize attempts to
clean the tx and rx buffers, and ensure that e1000_close is called
after the tx_timeout_task has completed running and/or that the task
is safe to run after e1000_close hasrun.
I'm concerned about the addition of the netif_running check to
e1000_down. While something like this is needed, I'm not familiar
enough w/ the code to know if this is okay.
All explanations and comments are greatly appreciated.
Thanks,
Shaw
[-- Attachment #2: e1000.patch --]
[-- Type: text/x-patch, Size: 2736 bytes --]
diff -u -uprN -X linux-2.6.16.11/Documentation/dontdiff linux-2.6.16.11/drivers/net/e1000/e1000.h linux-2.6.16.11.e1000_patch/drivers/net/e1000/e1000.h
--- linux-2.6.16.11/drivers/net/e1000/e1000.h 2006-04-24 13:20:24.000000000 -0700
+++ linux-2.6.16.11.e1000_patch/drivers/net/e1000/e1000.h 2006-04-26 16:23:46.475842000 -0700
@@ -358,5 +358,8 @@ struct e1000_adapter {
#ifdef CONFIG_PCI_MSI
boolean_t have_msi;
#endif
+ uint32_t flags;
+#define E1000_CLEANING 0x00000001
+ spinlock_t clean_lock;
};
#endif /* _E1000_H_ */
diff -u -uprN -X linux-2.6.16.11/Documentation/dontdiff linux-2.6.16.11/drivers/net/e1000/e1000_main.c linux-2.6.16.11.e1000_patch/drivers/net/e1000/e1000_main.c
--- linux-2.6.16.11/drivers/net/e1000/e1000_main.c 2006-04-24 13:20:24.000000000 -0700
+++ linux-2.6.16.11.e1000_patch/drivers/net/e1000/e1000_main.c 2006-04-26 16:59:48.742905000 -0700
@@ -525,6 +525,16 @@ e1000_down(struct e1000_adapter *adapter
boolean_t mng_mode_enabled = (adapter->hw.mac_type >= e1000_82571) &&
e1000_check_mng_mode(&adapter->hw);
+ spin_lock_bh(&adapter->clean_lock);
+ adapter->flags |= E1000_CLEANING;
+
+ if (!netif_running(netdev)) {
+ adapter->flags &= ~E1000_CLEANING;
+ spin_unlock_bh(&adapter->clean_lock);
+ return;
+ }
+ spin_unlock_bh(&adapter->clean_lock);
+
e1000_irq_disable(adapter);
#ifdef CONFIG_E1000_MQ
while (atomic_read(&adapter->rx_sched_call_data.count) != 0);
@@ -549,8 +559,12 @@ e1000_down(struct e1000_adapter *adapter
netif_stop_queue(netdev);
e1000_reset(adapter);
+
+ spin_lock_bh(&adapter->clean_lock);
e1000_clean_all_tx_rings(adapter);
e1000_clean_all_rx_rings(adapter);
+ adapter->flags &= ~E1000_CLEANING;
+ spin_unlock_bh(&adapter->clean_lock);
/* Power down the PHY so no link is implied when interface is down *
* The PHY cannot be powered down if any of the following is TRUE *
@@ -1109,6 +1123,8 @@ e1000_sw_init(struct e1000_adapter *adap
atomic_set(&adapter->irq_sem, 1);
spin_lock_init(&adapter->stats_lock);
+ spin_lock_init(&adapter->clean_lock);
+ adapter->flags = 0;
return 0;
}
@@ -1269,10 +1285,18 @@ e1000_close(struct net_device *netdev)
{
struct e1000_adapter *adapter = netdev_priv(netdev);
+ /* Calling flush_scheduled_work() may deadlock because
+ * linkwatch_event() may be on the workqueue and it will
+ * try to get the rtnl_lock which we are holding. */
+ while (adapter->flags & E1000_CLEANING)
+ msleep(1);
+
e1000_down(adapter);
+ spin_lock_bh(&adapter->clean_lock);
e1000_free_all_tx_resources(adapter);
e1000_free_all_rx_resources(adapter);
+ spin_unlock_bh(&adapter->clean_lock);
if ((adapter->hw.mng_cookie.status &
E1000_MNG_DHCP_COOKIE_STATUS_VLAN_SUPPORT)) {
^ permalink raw reply
* Re: [RFC] Geographical/regulatory information for ieee80211
From: Larry Finger @ 2006-04-27 0:54 UTC (permalink / raw)
To: Rick Jones, Christoph Hellwig, netdev
In-Reply-To: <4443D694.8090809@hp.com>
Rick Jones wrote:
> Christoph Hellwig wrote:
>> On Thu, Apr 13, 2006 at 07:59:21PM -0500, Larry Finger wrote:
>>
>>> I am planning on writing a new routine to be added to
>>> net/ieee80211/ieee80211_geo.c that will populate an ieee80211_geo
>>> object given a country code. The new routine will eliminate the need
>>> for each driver to do their own.
>>
>>
>> This sounds like a generally good idea, but the question is: do we want
>> this inside a kernel module or in userspace, either like the regulartory
>> daemon intel has (unfortunately in binary only form) or as a simple init
>> script. I really don't want to recompile my kernel just because
>> regulations
>> changed, and they seems to do that quite often.
>
> Yet I would expect the regulatory bodies to look less favorably on
> something more easily maleable by the end-user.
I don't think it would make that much difference as the user could easily lie about their locality
and get any set of parameters that they wanted. Intel avoids this problem by hiding the locality in
an EEPROM (ipw2200) or by combining the EEPROM information with the binary-only daemon (3945).
I am leaning toward putting the geographical information into a userland daemon. That way we won't
have to patch the kernel every time a country modifies its regulations. In addition, the kernel will
be smaller. The downside is that the daemon will have to be updated and supplied in some convenient
form, perhaps as part of a wireless tools package.
Larry
^ permalink raw reply
* RE: [PATCH 1/3] Rough VJ Channel Implementation - vj_core.patch
From: Caitlin Bestler @ 2006-04-27 1:02 UTC (permalink / raw)
To: David S. Miller; +Cc: jeff, kelly, netdev, rusty
netdev-owner@vger.kernel.org wrote:
> From: "Caitlin Bestler" <caitlinb@broadcom.com>
> Date: Wed, 26 Apr 2006 15:53:44 -0700
>
>> The netchannel qualifiers should only deal with TCP packets for
>> established connections. Listens would continue to be dealt with by
>> the existing stack logic, vj_channelizing only occurring when the the
>> connection was accepted.
>
> I consider netchannel support for listening TCP sockets to be
> absolutely essential. -
Meaning that inbound SYNs would be placed in a net channel
for processing by a Consumer at the other end of the ring?
If so the rules filtering SYNs would have to be applied either
before it went into the ring, or when the consumer end takes
them out. The latter makes more sense to me, because the rules
about what remote hosts can initiate a connection request to
a given TCP port can be fairly complex for a variety of
legitimate reasons.
Would it be reasonable to state that a net channel carrying
SYNs should not be set up when the consumer is a user mode
process?
^ permalink raw reply
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox