* Re: Problem wit route cache
From: Paweł Staszewski @ 2010-02-08 14:16 UTC (permalink / raw)
To: Eric Dumazet; +Cc: Linux Network Development list
In-Reply-To: <1265638014.3048.20.camel@edumazet-laptop>
W dniu 2010-02-08 15:06, Eric Dumazet pisze:
> Le lundi 08 février 2010 à 14:59 +0100, Paweł Staszewski a écrit :
>
>> W dniu 2010-02-08 14:51, Eric Dumazet pisze:
>>
>>> Le lundi 08 février 2010 à 14:33 +0100, Paweł Staszewski a écrit :
>>>
>>>
>>>
>>>>>
>>>>>
>>>> Yes this is x86_64 kernel
>>>> i kernels 2.6.32.2 / 2.6.32.7 and now 2.6.33-rc6-git5 and on all
>>>> kernels the same thing happens.
>>>> grep . /proc/sys/net/ipv4/route/*
>>>> /proc/sys/net/ipv4/route/error_burst:1250
>>>> /proc/sys/net/ipv4/route/error_cost:250
>>>> grep: /proc/sys/net/ipv4/route/flush: Permission denied
>>>> /proc/sys/net/ipv4/route/gc_elasticity:2
>>>> /proc/sys/net/ipv4/route/gc_interval:2
>>>> /proc/sys/net/ipv4/route/gc_min_interval:0
>>>> /proc/sys/net/ipv4/route/gc_min_interval_ms:500
>>>> /proc/sys/net/ipv4/route/gc_thresh:65535
>>>> /proc/sys/net/ipv4/route/gc_timeout:300
>>>> /proc/sys/net/ipv4/route/max_size:524288
>>>> /proc/sys/net/ipv4/route/min_adv_mss:256
>>>> /proc/sys/net/ipv4/route/min_pmtu:552
>>>> /proc/sys/net/ipv4/route/mtu_expires:600
>>>> /proc/sys/net/ipv4/route/redirect_load:5
>>>> /proc/sys/net/ipv4/route/redirect_number:9
>>>> /proc/sys/net/ipv4/route/redirect_silence:5120
>>>> /proc/sys/net/ipv4/route/secret_interval:2
>>>>
>>>> This happens not all the time.
>>>> I have this info only when there are "internet rush hours" - thn there
>>>> is about 700Mbit/s TX + 700Mbit/s RX forwarded traffic
>>>>
>>>>
>>>>
>>> I dont understand your settings, they are very very small for your
>>> setup. You want to flush cache every 2 seconds...
>>>
>>> With 12GB of ram, you could have
>>>
>>> /proc/sys/net/ipv4/route/gc_thresh:524288
>>> /proc/sys/net/ipv4/route/max_size:8388608
>>> /proc/sys/net/ipv4/route/secret_interval:3600
>>> /proc/sys/net/ipv4/route/gc_elasticity:4
>>> /proc/sys/net/ipv4/route/gc_interval:1
>>>
>>> That would allow about 2 million entries in your route cache, using 768
>>> Mbytes of ram, and a good cache hit ratio.
>>>
>>>
>>>
>>>
>> Yes as i write i change this settings after i see first info
>> "secret_interval" - from 3600 to 2
>> To check if this resolve the problem.
>> Also my normal settings are:
>>
>> /proc/sys/net/ipv4/route/gc_thresh:256000
>> /proc/sys/net/ipv4/route/max_size:1048576
>> /proc/sys/net/ipv4/route/secret_interval:3600
>> /proc/sys/net/ipv4/route/gc_interval:2
>> /proc/sys/net/ipv4/route/gc_elasticity:2
>>
>> And with this setting i was have this info:
>> Route hash chain too long!
>> Adjust your secret_interval!
>>
>>
>>
>> Now i put Your settings as You suggest ... and we will see but i dont know it will help.
>> Because i try many of different settings.
>>
>>
> One important point is the size of hash table, you want something big
> for your router.
>
> # dmesg | grep 'IP route'
> ... IP route cache hash table entries: 524288 (order: 10, 4194304
> bytes)
>
>
On my machine it is also the same:
dmesg | grep 'IP route'
IP route cache hash table entries: 524288 (order: 10, 4194304 bytes)
> Then if it is correctly sized, dont change gc_thresh or max_size, as
> defaults are good.
>
> I would only change gc_interval to 1, to perform a smooth gc
>
> And eventually gc_elasticity to 4, 5 or 6 if I had less ram than your
> machine.
>
>
Some day ago after info about route cache i was have also this info:
Feb 4 13:12:40 TM_01_C1 ------------[ cut here ]------------
Feb 4 13:12:40 TM_01_C1 WARNING: at net/sched/sch_generic.c:261
dev_watchdog+0x130/0x1d6()
Feb 4 13:12:40 TM_01_C1 Hardware name: X7DCT
Feb 4 13:12:40 TM_01_C1 NETDEV WATCHDOG: eth0 (e1000e): transmit queue
0 timed out
Feb 4 13:12:40 TM_01_C1 Modules linked in: oprofile
Feb 4 13:12:40 TM_01_C1 Pid: 0, comm: swapper Not tainted 2.6.32 #1
Feb 4 13:12:40 TM_01_C1 Call Trace:
Feb 4 13:12:40 TM_01_C1 <IRQ> [<ffffffff812fcaf7>] ?
dev_watchdog+0x130/0x1d6
Feb 4 13:12:40 TM_01_C1 [<ffffffff812fcaf7>] ? dev_watchdog+0x130/0x1d6
Feb 4 13:12:40 TM_01_C1 [<ffffffff81038811>] ?
warn_slowpath_common+0x77/0xa3
Feb 4 13:12:40 TM_01_C1 [<ffffffff81038899>] ? warn_slowpath_fmt+0x51/0x59
Feb 4 13:12:40 TM_01_C1 [<ffffffff8102897e>] ? activate_task+0x3f/0x4e
Feb 4 13:12:40 TM_01_C1 [<ffffffff81034fe5>] ? try_to_wake_up+0x1eb/0x1f8
Feb 4 13:12:40 TM_01_C1 [<ffffffff812eb768>] ? netdev_drivername+0x3b/0x40
Feb 4 13:12:40 TM_01_C1 [<ffffffff812fcaf7>] ? dev_watchdog+0x130/0x1d6
Feb 4 13:12:40 TM_01_C1 [<ffffffff8102d1e3>] ? __wake_up+0x30/0x44
Feb 4 13:12:40 TM_01_C1 [<ffffffff812fc9c7>] ? dev_watchdog+0x0/0x1d6
Feb 4 13:12:40 TM_01_C1 [<ffffffff810448c4>] ?
run_timer_softirq+0x1ff/0x29d
Feb 4 13:12:40 TM_01_C1 [<ffffffff810556ab>] ? ktime_get+0x5f/0xb7
Feb 4 13:12:40 TM_01_C1 [<ffffffff8103e0fd>] ? __do_softirq+0xd7/0x196
Feb 4 13:12:40 TM_01_C1 [<ffffffff8100be7c>] ? call_softirq+0x1c/0x28
Feb 4 13:12:40 TM_01_C1 [<ffffffff8100d645>] ? do_softirq+0x31/0x66
Feb 4 13:12:40 TM_01_C1 [<ffffffff8101b148>] ?
smp_apic_timer_interrupt+0x87/0x95
Feb 4 13:12:40 TM_01_C1 [<ffffffff8100b873>] ?
apic_timer_interrupt+0x13/0x20
Feb 4 13:12:40 TM_01_C1 <EOI> [<ffffffff810111f5>] ? mwait_idle+0x9b/0xa0
Feb 4 13:12:40 TM_01_C1 [<ffffffff8100a236>] ? cpu_idle+0x49/0x7c
Feb 4 13:12:40 TM_01_C1 ---[ end trace c670a6a17be040e5 ]---
And after change kernel to 2.6.33-rc6 another different inf:
BUG: soft lockup - CPU#1 stuck for 61s!
[events/1:28]
Modules linked in:
CPU 1
Pid: 28, comm: events/1 Not tainted 2.6.33-rc6-git5 #1 X7DCT/X7DCT
RIP: 0010:[<ffffffff810a3d89>] [<ffffffff810a3d89>]
kmem_cache_free+0x11b/0x11c
RSP: 0018:ffff880028243e50 EFLAGS: 00000292
RAX: 0000000000000032 RBX: 000000000000007d RCX: ffff8803190683c0
RDX: 0000000000000031 RSI: ffff8803190683c0 RDI: ffff88031f83e680
RBP: ffffffff81002893 R08: 0000000000000000 R09: 000000000000007c
R10: ffff88030d776800 R11: ffff88030d7768a0 R12: ffff880028243dd0
R13: ffffc900008b2f80 R14: ffff88031fa7c800 R15: ffffffff81012da7
FS: 0000000000000000(0000) GS:ffff880028240000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
CR2: 00007fd61d5bd000 CR3: 000000031e55c000 CR4: 00000000000006a0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Process events/1 (pid: 28, threadinfo ffff88031f9c8000, task ffff88031f9a4f80)
Stack:
ffffffff8126826f ffff88031faa4600 ffffffff8126834a 000096ba00000023
<0> 01ffc90000000024 ffff88031fbb4000 ffff88031faa4600 0000000000000040
<0> 0000000000000040 ffff88031faa4788 ffff88031faa4600 0000000000000740
Call Trace:
<IRQ>
[<ffffffff8126826f>] ? e1000_put_txbuf+0x62/0x74
[<ffffffff8126834a>] ? e1000_clean_tx_irq+0xc9/0x235
[<ffffffff8126b71b>] ? e1000_clean+0x5c/0x21c
[<ffffffff812f29a3>] ? net_rx_action+0x71/0x15d
[<ffffffff81035311>] ? __do_softirq+0xd7/0x196
[<ffffffff81002dac>] ? call_softirq+0x1c/0x28
[<ffffffff812f768f>] ? dst_gc_task+0x0/0x1a7
[<ffffffff81002dac>] ? call_softirq+0x1c/0x28
<EOI>
[<ffffffff81004599>] ? do_softirq+0x31/0x63
[<ffffffff81034ec1>] ? local_bh_enable_ip+0x75/0x86
[<ffffffff812f768f>] ? dst_gc_task+0x0/0x1a7
[<ffffffff812f775d>] ? dst_gc_task+0xce/0x1a7
[<ffffffff8136b08c>] ? schedule+0x82c/0x906
[<ffffffff8103c44f>] ? lock_timer_base+0x26/0x4b
[<ffffffff810a41d6>] ? cache_reap+0x0/0x11d
[<ffffffff81044c38>] ? worker_thread+0x14c/0x1dc
[<ffffffff81047dcd>] ? autoremove_wake_function+0x0/0x2e
[<ffffffff81044aec>] ? worker_thread+0x0/0x1dc
[<ffffffff810479bd>] ? kthread+0x79/0x81
[<ffffffff81002cb4>] ? kernel_thread_helper+0x4/0x10
[<ffffffff81047944>] ? kthread+0x0/0x81
[<ffffffff81002cb0>] ? kernel_thread_helper+0x0/0x10
Code: fe 79 4c 00 48 85 db 74 14 48 8b 74 24 10 48 89 ef ff 13 48 83 c3 08 48
83 3b 00 eb ea 48 83 c4 18 5b 5d 41 5c 41 5d 41 5e 41 5f<c3> 55 48 89 f5 53 48
89 fb 48 83 ec 08 48 8b 76 18 48 2b 75 10
Call Trace:
<IRQ> [<ffffffff8126826f>] ? e1000_put_txbuf+0x62/0x74
[<ffffffff8126834a>] ? e1000_clean_tx_irq+0xc9/0x235
[<ffffffff8126b71b>] ? e1000_clean+0x5c/0x21c
[<ffffffff812f29a3>] ? net_rx_action+0x71/0x15d
[<ffffffff81035311>] ? __do_softirq+0xd7/0x196
[<ffffffff81002dac>] ? call_softirq+0x1c/0x28
[<ffffffff812f768f>] ? dst_gc_task+0x0/0x1a7
[<ffffffff81002dac>] ? call_softirq+0x1c/0x28
<EOI> [<ffffffff81004599>] ? do_softirq+0x31/0x63
[<ffffffff81034ec1>] ? local_bh_enable_ip+0x75/0x86
[<ffffffff812f768f>] ? dst_gc_task+0x0/0x1a7
[<ffffffff812f775d>] ? dst_gc_task+0xce/0x1a7
[<ffffffff8136b08c>] ? schedule+0x82c/0x906
[<ffffffff8103c44f>] ? lock_timer_base+0x26/0x4b
[<ffffffff810a41d6>] ? cache_reap+0x0/0x11d
[<ffffffff81044c38>] ? worker_thread+0x14c/0x1dc
[<ffffffff81047dcd>] ? autoremove_wake_function+0x0/0x2e
[<ffffffff81044aec>] ? worker_thread+0x0/0x1dc
[<ffffffff810479bd>] ? kthread+0x79/0x81
[<ffffffff81002cb4>] ? kernel_thread_helper+0x4/0x10
[<ffffffff81047944>] ? kthread+0x0/0x81
[<ffffffff81002cb0>] ? kernel_thread_helper+0x0/0x10
>
>
> --
> To unsubscribe from this list: send the line "unsubscribe netdev" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
>
>
>
^ permalink raw reply
* RE: [PATCH] [V2] net: emaclite: adding MDIO and phy lib support
From: John Linn @ 2010-02-08 14:09 UTC (permalink / raw)
To: John Linn, netdev, linuxppc-dev, jgarzik, grant.likely, jwboyer
Cc: john.williams, Sadanand Mutyala
In-Reply-To: <1265409646-11882-1-git-send-email-john.linn@xilinx.com>
> -----Original Message-----
> From: John Linn [mailto:john.linn@xilinx.com]
> Sent: Friday, February 05, 2010 3:41 PM
> To: netdev@vger.kernel.org; linuxppc-dev@ozlabs.org;
jgarzik@pobox.com; grant.likely@secretlab.ca;
> jwboyer@linux.vnet.ibm.com
> Cc: john.williams@petalogix.com; John Linn; Sadanand Mutyala
> Subject: [PATCH] [V2] net: emaclite: adding MDIO and phy lib support
>
> These changes add MDIO and phy lib support to the driver as the
> IP core now supports the MDIO bus.
>
> The MDIO bus and phy are added as a child to the emaclite in the
device
> tree as illustrated below.
>
> mdio {
> #address-cells = <1>;
> #size-cells = <0>;
> compatible = "xlnx,emaclite-mdio";
> phy0: phy@7 {
> reg = <7>;
> } ;
> }
>
> Signed-off-by: Sadanand Mutyala <Sadanand.Mutyala@xilinx.com>
> Signed-off-by: John Linn <john.linn@xilinx.com>
>
> ---
>
> V2 - updated it for Grant's comments, except I couldn't find any tabs
> converted to white space issue, let's see if V2 has it also
> ---
> drivers/net/Kconfig | 1 +
> drivers/net/xilinx_emaclite.c | 393
++++++++++++++++++++++++++++++++++++-----
> 2 files changed, 351 insertions(+), 43 deletions(-)
>
> diff --git a/drivers/net/Kconfig b/drivers/net/Kconfig
> index dd9a09c..9509a36 100644
> --- a/drivers/net/Kconfig
> +++ b/drivers/net/Kconfig
> @@ -1939,6 +1939,7 @@ config ATL2
> config XILINX_EMACLITE
> tristate "Xilinx 10/100 Ethernet Lite support"
> depends on PPC32 || MICROBLAZE
> + select PHYLIB
> help
> This driver supports the 10/100 Ethernet Lite from Xilinx.
>
> diff --git a/drivers/net/xilinx_emaclite.c
b/drivers/net/xilinx_emaclite.c
> index 8c777ba..02f18dd 100644
> --- a/drivers/net/xilinx_emaclite.c
> +++ b/drivers/net/xilinx_emaclite.c
> @@ -22,11 +22,17 @@
>
> #include <linux/of_device.h>
> #include <linux/of_platform.h>
> +#include <linux/of_mdio.h>
> +#include <linux/phy.h>
>
<snip>
> +static int xemaclite_mdio_read(struct mii_bus *bus, int phy_id, int
reg)
> +{
> + struct net_local *lp = bus->priv;
> + u32 ctrl_reg;
> + u32 rc;
> +
> + mutex_lock(&lp->mdio_mutex);
> +
> + if (xemaclite_mdio_wait(lp))
> + return -ETIMEDOUT;
I already see a problem with this patch as when there's a timeout
waiting for the mdio the mutex won't get unlocked. This is true
anywhere the timeout happens (other places also).
I tested it with normal operation, but don't have a great way to test
the timeout paths.
I'll wait for other comments then spin it again to unlock the mutex when
there's a timeout.
Thanks,
John
This email and any attachments are intended for the sole use of the named recipient(s) and contain(s) confidential information that may be proprietary, privileged or copyrighted under applicable law. If you are not the intended recipient, do not read, copy, or forward this email message or any attachments. Delete this email message and any attachments immediately.
^ permalink raw reply
* Re: Problem wit route cache
From: Eric Dumazet @ 2010-02-08 14:06 UTC (permalink / raw)
To: Paweł Staszewski; +Cc: Linux Network Development list
In-Reply-To: <4B7018DF.8060600@itcare.pl>
Le lundi 08 février 2010 à 14:59 +0100, Paweł Staszewski a écrit :
> W dniu 2010-02-08 14:51, Eric Dumazet pisze:
> > Le lundi 08 février 2010 à 14:33 +0100, Paweł Staszewski a écrit :
> >
> >
> >>>
> >>>
> >> Yes this is x86_64 kernel
> >> i kernels 2.6.32.2 / 2.6.32.7 and now 2.6.33-rc6-git5 and on all
> >> kernels the same thing happens.
> >> grep . /proc/sys/net/ipv4/route/*
> >> /proc/sys/net/ipv4/route/error_burst:1250
> >> /proc/sys/net/ipv4/route/error_cost:250
> >> grep: /proc/sys/net/ipv4/route/flush: Permission denied
> >> /proc/sys/net/ipv4/route/gc_elasticity:2
> >> /proc/sys/net/ipv4/route/gc_interval:2
> >> /proc/sys/net/ipv4/route/gc_min_interval:0
> >> /proc/sys/net/ipv4/route/gc_min_interval_ms:500
> >> /proc/sys/net/ipv4/route/gc_thresh:65535
> >> /proc/sys/net/ipv4/route/gc_timeout:300
> >> /proc/sys/net/ipv4/route/max_size:524288
> >> /proc/sys/net/ipv4/route/min_adv_mss:256
> >> /proc/sys/net/ipv4/route/min_pmtu:552
> >> /proc/sys/net/ipv4/route/mtu_expires:600
> >> /proc/sys/net/ipv4/route/redirect_load:5
> >> /proc/sys/net/ipv4/route/redirect_number:9
> >> /proc/sys/net/ipv4/route/redirect_silence:5120
> >> /proc/sys/net/ipv4/route/secret_interval:2
> >>
> >> This happens not all the time.
> >> I have this info only when there are "internet rush hours" - thn there
> >> is about 700Mbit/s TX + 700Mbit/s RX forwarded traffic
> >>
> >>
> > I dont understand your settings, they are very very small for your
> > setup. You want to flush cache every 2 seconds...
> >
> > With 12GB of ram, you could have
> >
> > /proc/sys/net/ipv4/route/gc_thresh:524288
> > /proc/sys/net/ipv4/route/max_size:8388608
> > /proc/sys/net/ipv4/route/secret_interval:3600
> > /proc/sys/net/ipv4/route/gc_elasticity:4
> > /proc/sys/net/ipv4/route/gc_interval:1
> >
> > That would allow about 2 million entries in your route cache, using 768
> > Mbytes of ram, and a good cache hit ratio.
> >
> >
> >
> Yes as i write i change this settings after i see first info
> "secret_interval" - from 3600 to 2
> To check if this resolve the problem.
> Also my normal settings are:
>
> /proc/sys/net/ipv4/route/gc_thresh:256000
> /proc/sys/net/ipv4/route/max_size:1048576
> /proc/sys/net/ipv4/route/secret_interval:3600
> /proc/sys/net/ipv4/route/gc_interval:2
> /proc/sys/net/ipv4/route/gc_elasticity:2
>
> And with this setting i was have this info:
> Route hash chain too long!
> Adjust your secret_interval!
>
>
>
> Now i put Your settings as You suggest ... and we will see but i dont know it will help.
> Because i try many of different settings.
>
One important point is the size of hash table, you want something big
for your router.
# dmesg | grep 'IP route'
... IP route cache hash table entries: 524288 (order: 10, 4194304
bytes)
Then if it is correctly sized, dont change gc_thresh or max_size, as
defaults are good.
I would only change gc_interval to 1, to perform a smooth gc
And eventually gc_elasticity to 4, 5 or 6 if I had less ram than your
machine.
^ permalink raw reply
* Re: Problem wit route cache
From: Paweł Staszewski @ 2010-02-08 13:59 UTC (permalink / raw)
To: Eric Dumazet, Linux Network Development list
In-Reply-To: <1265637067.3048.14.camel@edumazet-laptop>
W dniu 2010-02-08 14:51, Eric Dumazet pisze:
> Le lundi 08 février 2010 à 14:33 +0100, Paweł Staszewski a écrit :
>
>
>>>
>>>
>> Yes this is x86_64 kernel
>> i kernels 2.6.32.2 / 2.6.32.7 and now 2.6.33-rc6-git5 and on all
>> kernels the same thing happens.
>> grep . /proc/sys/net/ipv4/route/*
>> /proc/sys/net/ipv4/route/error_burst:1250
>> /proc/sys/net/ipv4/route/error_cost:250
>> grep: /proc/sys/net/ipv4/route/flush: Permission denied
>> /proc/sys/net/ipv4/route/gc_elasticity:2
>> /proc/sys/net/ipv4/route/gc_interval:2
>> /proc/sys/net/ipv4/route/gc_min_interval:0
>> /proc/sys/net/ipv4/route/gc_min_interval_ms:500
>> /proc/sys/net/ipv4/route/gc_thresh:65535
>> /proc/sys/net/ipv4/route/gc_timeout:300
>> /proc/sys/net/ipv4/route/max_size:524288
>> /proc/sys/net/ipv4/route/min_adv_mss:256
>> /proc/sys/net/ipv4/route/min_pmtu:552
>> /proc/sys/net/ipv4/route/mtu_expires:600
>> /proc/sys/net/ipv4/route/redirect_load:5
>> /proc/sys/net/ipv4/route/redirect_number:9
>> /proc/sys/net/ipv4/route/redirect_silence:5120
>> /proc/sys/net/ipv4/route/secret_interval:2
>>
>> This happens not all the time.
>> I have this info only when there are "internet rush hours" - thn there
>> is about 700Mbit/s TX + 700Mbit/s RX forwarded traffic
>>
>>
> I dont understand your settings, they are very very small for your
> setup. You want to flush cache every 2 seconds...
>
> With 12GB of ram, you could have
>
> /proc/sys/net/ipv4/route/gc_thresh:524288
> /proc/sys/net/ipv4/route/max_size:8388608
> /proc/sys/net/ipv4/route/secret_interval:3600
> /proc/sys/net/ipv4/route/gc_elasticity:4
> /proc/sys/net/ipv4/route/gc_interval:1
>
> That would allow about 2 million entries in your route cache, using 768
> Mbytes of ram, and a good cache hit ratio.
>
>
>
Yes as i write i change this settings after i see first info
"secret_interval" - from 3600 to 2
To check if this resolve the problem.
Also my normal settings are:
/proc/sys/net/ipv4/route/gc_thresh:256000
/proc/sys/net/ipv4/route/max_size:1048576
/proc/sys/net/ipv4/route/secret_interval:3600
/proc/sys/net/ipv4/route/gc_interval:2
/proc/sys/net/ipv4/route/gc_elasticity:2
And with this setting i was have this info:
Route hash chain too long!
Adjust your secret_interval!
Now i put Your settings as You suggest ... and we will see but i dont know it will help.
Because i try many of different settings.
>
>
>
>
>
^ permalink raw reply
* Re: smsc911x suspend/resume
From: Mike Rapoport @ 2010-02-08 13:55 UTC (permalink / raw)
To: Steve.Glendinning
Cc: Daniel Mack, LAKML, netdev, Ian.Saturley, Vlad.Lyalikov, davem
In-Reply-To: <OFE2B2FE18.A8D1D506-ON802576C4.003FCA39-802576C4.0040B683@smsc.com>
Steve.Glendinning@smsc.com wrote:
> Hi Mike,
>
>>>> I'm trying to make suspend/resume work on OMAP3-based system and I'm
>>>> encountering issues with resume of SMSC 9220 chip.
>>>> After resume the interface is unusable unless I do 'ifconfig eth0
> down'
>>>> and 'ifconfig eth0 up'.
>>>> I would expect that 'ping <some host>' would work right after resume
>>>> without need to bring the interface down and up, but probably I miss
>>>> something.
>>>> Any help would be appreciated.
>>> What happens to your supply voltages when going to suspend? When I
>>> implemented the code for the smsc driver, I could only test scenarios
>>> where AVDD remains stable during suspend. So the driver might need
> some
>>> tweaks if that assumption is not true in your case. The SMSC datasheet
>>> is quite comprehensive about this topic IIRC.
>> By AVDD you mean VDD33A?
>> Anyway, I have all the supplies except VDDVARIO shut down...
>> And the datasheet is indeed comprehensively describes chip power states
>> but I haven't found there anything about supplies required to stay on
>> during suspend...
>
> I don't have the code in front of me (I'm travelling today) but IIRC
> the driver doesn't place the LAN9220 in a specific power saving state,
> so as far as the LAN9220 is concerned it's not actually suspended.
>
> It's hard to do this in a "generic" way: some people expect their link
> to stay up during suspend, some want suspend to completely power down
> the ethernet chip, and some want to use one of several WoL modes.
Indeed a "generic" implementation may be tough task... Probably we can
add some platform_data flags for suspend modes and use them to decide
what low power mode should be selected?
> David: What's the "done thing" here for similar non-pci drivers?
> Obviously "doesn't work after suspend" isn't the intended behaviour, so
> there's definitely room for improvement!
>
>>> Also have a look at the Raumfeld device patches which use exactly this
>>> chip and which can suspend and resume just fine. You'll need to check
>>> out Eric's devel branch for that.
>
> Daniel: I haven't seen this, do you have a location for the git tree?
>
> Thanks,
> Steve
--
Sincerely yours,
Mike.
^ permalink raw reply
* Flooded with bonding: bond0: doing slave updates when interface is down.
From: Ferenc Wagner @ 2010-02-08 13:20 UTC (permalink / raw)
To: netdev
Hi,
On a system running Linux 2.6.32.7 I use the following initramfs script
to bring up some interfaces before mounting the root filesystem:
---------- /etc/initramfs-tools/scripts/init-premount/net_up ----------
#!/bin/sh
PREREQ="udev"
prereqs()
{
echo "$PREREQ"
}
case $1 in
prereqs)
prereqs
exit 0
;;
esac
. /scripts/functions
maybe_break pre-netup
# Arguments: DEV IP/MASKLEN
ifup()
{
ip addr add $2 dev $1
ip link set up dev $1
echo "Configured $1 with address $2"
}
BOND=bond0
modprobe bonding
cd /sys/class/net/$BOND/bonding
echo active-backup >mode
echo +eth0 >slaves
echo +eth1 >slaves
echo eth0 >primary
echo +10.0.0.1 >arp_ip_target
echo +10.0.0.2 >arp_ip_target
echo 1000 >arp_interval
vconfig set_name_type VLAN_PLUS_VID_NO_PAD
vconfig add $BOND 39
eval $(getprops) || { panic "MAC of eth0 unknown, aborting."; halt; }
ifup $BOND $MYVLAN2
ifup vlan39 $MYVLAN39
ip addr
maybe_break post-netup
-----------------------------------------------------------------------
The eval $(getprops) command sets the MYVLANx variables depending on the
MAC address of eth0. This stuff mostly works as expected, but sometimes
I get this on the console:
[ 27.792746] Ethernet Channel Bonding Driver: v3.5.0 (November 4, 2008)
[ 27.831788] bonding: Warning: either miimon or arp_interval and arp_ip_target module parameters must be specified, otherwise bonding will not detect link failures! see bonding.txt for details.
[ 27.935640] bonding: bond0: setting mode to active-backup (1).
[ 27.970565] bonding: bond0: doing slave updates when interface is down.
[ 28.010110] bonding: bond0: Adding slave eth0.
[ 28.036651] bonding bond0: master_dev is not up in bond_enslave
[ 28.137410] bonding: bond0: Warning: failed to get speed and duplex from eth0, assumed to be 100Mb/sec and Full.
[ 28.198298] bonding: bond0: making interface eth0 the new active one.
[ 28.236806] bonding: bond0: first active interface up!
[ 28.267515] bonding: bond0: enslaving eth0 as an active interface with an up link.
[ 28.312847] bonding: bond0: doing slave updates when interface is down.
[ 28.352397] bonding: bond0: doing slave updates when interface is down.
[ 28.391941] bonding: bond0: doing slave updates when interface is down.
[ 28.431482] bonding: bond0: doing slave updates when interface is down.
[ 28.471023] bonding: bond0: doing slave updates when interface is down.
[ 28.510566] bonding: bond0: doing slave updates when interface is down.
[ 28.550111] bonding: bond0: doing slave updates when interface is down.
[ 28.589652] bonding: bond0: doing slave updates when interface is down.
[ 28.629197] bonding: bond0: doing slave updates when interface is down.
[ 28.668740] bonding: bond0: doing slave updates when interface is down.
[ 28.708282] bonding: bond0: doing slave updates when interface is down.
[ 28.749653] bonding: bond0: doing slave updates when interface is down.
[ 28.789195] bonding: bond0: doing slave updates when interface is down.
[ 28.828738] bonding: bond0: doing slave updates when interface is down.
[ 28.868280] bonding: bond0: doing slave updates when interface is down.
[ 28.907823] bonding: bond0: doing slave updates when interface is down.
[ 28.947364] bonding: bond0: doing slave updates when interface is down.
[ 28.986907] bonding: bond0: doing slave updates when interface is down.
[ 29.026450] bonding: bond0: doing slave updates when interface is down.
[ 29.065994] bonding: bond0: doing slave updates when interface is down.
[ 29.105535] bonding: bond0: doing slave updates when interface is down.
[ 29.145172] tg3: eth0: Link is up at 1000 Mbps, full duplex.
[ 29.178990] tg3: eth0: Flow control is off for TX and off for RX.
[ 29.215415] bonding: bond0: doing slave updates when interface is down.
[ 29.254956] bonding: bond0: doing slave updates when interface is down.
[ 29.294496] bonding: bond0: doing slave updates when interface is down.
[ 29.334038] bonding: bond0: doing slave updates when interface is down.
[ 29.373581] bonding: bond0: doing slave updates when interface is down.
[ 29.413122] bonding: bond0: doing slave updates when interface is down.
[ 29.452663] bonding: bond0: doing slave updates when interface is down.
[ 29.492205] bonding: bond0: doing slave updates when interface is down.
[ 29.531748] bonding: bond0: doing slave updates when interface is down.
[...]
[ 78.580920] bonding: bond0: doing slave updates when interface is down.
[ 78.620463] bonding: bond0: doing slave updates when interface is down.
[ 78.660009] bonding: bond0: doing slave updates when interface is down.
[ 78.699825] bonding: bond0: doing slave updates when interface is down.
[ 78.739373] bonding: bond0: Adding slave eth1.
[ 78.765914] bonding bond0: master_dev is not up in bond_enslave
[ 78.817517] tg3 0000:05:01.1: firmware: requesting tigon/tg3_tso.bin
[ 78.919759] bonding: bond0: Warning: failed to get speed and duplex from eth1, assumed to be 100Mb/sec and Full.
[ 78.980658] bonding: bond0: enslaving eth1 as a backup interface with an up link.
[ 79.025492] bonding: bond0: Setting eth0 as primary slave.
[ 79.058351] bonding: bond0: adding ARP target 10.0.0.1.
[ 79.089601] bonding: bond0: adding ARP target 10.0.0.2.
[ 79.120855] bonding: bond0: Setting ARP monitoring interval to 1000.
[ 79.167280] 802.1Q VLAN Support v1.8 Ben Greear <greearb@candelatech.com>
[ 79.208621] All bugs added by David S. Miller <davem@redhat.com>
[ 79.259000] bonding: bond0: no route to arp_ip_target 10.0.0.1
In the end, everything seems to be all right, but this occasional
interlude is disturbing and seems to indicate that something isn't quite
right. Which may well be my abuse of the bonding driver, but then
please enlighten me, as I'd like to eliminate this 50-second delay from
the boot procedure. I don't mind the couple of "doing slave updates
when interface is down" warnings which appear during the normal course
of actions, but the above is way too much in my opinion.
--
Thanks,
Feri.
^ permalink raw reply
* Re: Problem wit route cache
From: Eric Dumazet @ 2010-02-08 13:51 UTC (permalink / raw)
To: Paweł Staszewski; +Cc: Linux Network Development list
In-Reply-To: <4B7012BC.9000702@itcare.pl>
Le lundi 08 février 2010 à 14:33 +0100, Paweł Staszewski a écrit :
> >
> Yes this is x86_64 kernel
> i kernels 2.6.32.2 / 2.6.32.7 and now 2.6.33-rc6-git5 and on all
> kernels the same thing happens.
> grep . /proc/sys/net/ipv4/route/*
> /proc/sys/net/ipv4/route/error_burst:1250
> /proc/sys/net/ipv4/route/error_cost:250
> grep: /proc/sys/net/ipv4/route/flush: Permission denied
> /proc/sys/net/ipv4/route/gc_elasticity:2
> /proc/sys/net/ipv4/route/gc_interval:2
> /proc/sys/net/ipv4/route/gc_min_interval:0
> /proc/sys/net/ipv4/route/gc_min_interval_ms:500
> /proc/sys/net/ipv4/route/gc_thresh:65535
> /proc/sys/net/ipv4/route/gc_timeout:300
> /proc/sys/net/ipv4/route/max_size:524288
> /proc/sys/net/ipv4/route/min_adv_mss:256
> /proc/sys/net/ipv4/route/min_pmtu:552
> /proc/sys/net/ipv4/route/mtu_expires:600
> /proc/sys/net/ipv4/route/redirect_load:5
> /proc/sys/net/ipv4/route/redirect_number:9
> /proc/sys/net/ipv4/route/redirect_silence:5120
> /proc/sys/net/ipv4/route/secret_interval:2
>
> This happens not all the time.
> I have this info only when there are "internet rush hours" - thn there
> is about 700Mbit/s TX + 700Mbit/s RX forwarded traffic
>
I dont understand your settings, they are very very small for your
setup. You want to flush cache every 2 seconds...
With 12GB of ram, you could have
/proc/sys/net/ipv4/route/gc_thresh:524288
/proc/sys/net/ipv4/route/max_size:8388608
/proc/sys/net/ipv4/route/secret_interval:3600
/proc/sys/net/ipv4/route/gc_elasticity:4
/proc/sys/net/ipv4/route/gc_interval:1
That would allow about 2 million entries in your route cache, using 768
Mbytes of ram, and a good cache hit ratio.
^ permalink raw reply
* Re: [PATCH for 2.6.33] conntrack: restrict runtime hashsize modifications
From: Patrick McHardy @ 2010-02-08 13:34 UTC (permalink / raw)
To: Alexey Dobriyan; +Cc: davem, jonathan, eric.dumazet, netdev, netfilter-devel
In-Reply-To: <20100205220407.GA27953@x200>
Alexey Dobriyan wrote:
> On Fri, Feb 05, 2010 at 11:00:03AM +0100, Patrick McHardy wrote:
>> Actually it doesn't seem like much more work to allow changing
>> table size, the main problem is that sysfs module parameters
>> don't seem to fit into the network namespace model at all.
>
> Well, they "fit" as they're global because modules are global.
> So we can make every netns hashtable size equals to module param,
> or make it bounded by module param
> or make initial hashtable size and not bounded
> or million other things.
Feel free to suggest your preferred method.
>> Please be more specific about your suspected slowdowns.
>
> I meant net->ct.htable_size in hash functions _if_ you're not allowing
> changing it from inside netns.
>
>> What's "everything"? What's different about the hashsize
>> compared to the many members we already moved to per-netns
>> structs?
>
> But whatever. I think per-netns hashtable size shouldn't be done
> that late.
This is a regression. You can not simply disable resizing and call
it a bugfix.
^ permalink raw reply
* Re: Problem wit route cache
From: Paweł Staszewski @ 2010-02-08 13:33 UTC (permalink / raw)
To: Eric Dumazet; +Cc: Linux Network Development list
In-Reply-To: <1265635690.3048.8.camel@edumazet-laptop>
[-- Attachment #1: Type: text/plain, Size: 2784 bytes --]
W dniu 2010-02-08 14:28, Eric Dumazet pisze:
> Le lundi 08 février 2010 à 14:16 +0100, Paweł Staszewski a écrit :
>
>> Hello
>>
>> From some time i have problem with route cache in linux
>> this is an info that i have in dmesg:
>>
>> Route hash chain too long!
>> Adjust your secret_interval!
>> Route hash chain too long!
>> Adjust your secret_interval!
>> Route hash chain too long!
>> Adjust your secret_interval!
>> Route hash chain too long!
>> Adjust your secret_interval!
>> Route hash chain too long!
>> Adjust your secret_interval!
>> Route hash chain too long!
>> Adjust your secret_interval!
>> Route hash chain too long!
>> Adjust your secret_interval!
>> Route hash chain too long!
>> Adjust your secret_interval!
>> vlan0811: 9 rebuilds is over limit, route caching disabled
>> Route hash chain too long!
>> Adjust your secret_interval!
>>
>> The problem is that change of net.ipv4.route.secret_interval is change
>> nothing -- no matter that i set secret_interval from dfault 3600 to 2 or
>> 10000 i have always the same info about route cahce is disabled.
>> Also i change this parameter net.ipv4.rt_cache_rebuild_count from
>> default 4 to 9 and the same info - i try also change this to 12 but also
>> this change nothing.
>>
>> The machine that have this info is:
>> 2x Intel(R) Xeon(R) CPU X5450 @ 3.00GHz
>> 12GB of RAM
>>
>>
> Are you running a 64bit kernel ?
> What is your kernel version ?
>
> Please send :
>
> # grep . /proc/sys/net/ipv4/route/*
> #rtstat -c10 -i1
>
>
Yes this is x86_64 kernel
i kernels 2.6.32.2 / 2.6.32.7 and now 2.6.33-rc6-git5 and on all
kernels the same thing happens.
grep . /proc/sys/net/ipv4/route/*
/proc/sys/net/ipv4/route/error_burst:1250
/proc/sys/net/ipv4/route/error_cost:250
grep: /proc/sys/net/ipv4/route/flush: Permission denied
/proc/sys/net/ipv4/route/gc_elasticity:2
/proc/sys/net/ipv4/route/gc_interval:2
/proc/sys/net/ipv4/route/gc_min_interval:0
/proc/sys/net/ipv4/route/gc_min_interval_ms:500
/proc/sys/net/ipv4/route/gc_thresh:65535
/proc/sys/net/ipv4/route/gc_timeout:300
/proc/sys/net/ipv4/route/max_size:524288
/proc/sys/net/ipv4/route/min_adv_mss:256
/proc/sys/net/ipv4/route/min_pmtu:552
/proc/sys/net/ipv4/route/mtu_expires:600
/proc/sys/net/ipv4/route/redirect_load:5
/proc/sys/net/ipv4/route/redirect_number:9
/proc/sys/net/ipv4/route/redirect_silence:5120
/proc/sys/net/ipv4/route/secret_interval:2
This happens not all the time.
I have this info only when there are "internet rush hours" - thn there
is about 700Mbit/s TX + 700Mbit/s RX forwarded traffic
>
> --
> To unsubscribe from this list: send the line "unsubscribe netdev" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
>
>
>
[-- Attachment #2: rtstat.txt --]
[-- Type: text/plain, Size: 2039 bytes --]
rtstat -c10 -i1
rt_cache|rt_cache|rt_cache|rt_cache|rt_cache|rt_cache|rt_cache|rt_cache|rt_cache|rt_cache|rt_cache|rt_cache|rt_cache|rt_cache|rt_cache|rt_cache|rt_cache|
entries| in_hit|in_slow_|in_slow_|in_no_ro| in_brd|in_marti|in_marti| out_hit|out_slow|out_slow|gc_total|gc_ignor|gc_goal_|gc_dst_o|in_hlist|out_hlis|
| | tot| mc| ute| | an_dst| an_src| | _tot| _mc| | ed| miss| verflow| _search|t_search|
12082|3440217296|6456413873| 0| 623094| 294| 0| 3| 735116| 5701062| 0|261260739|261040365| 0| 0|654617961| 179044|
10037| 0| 152142| 0| 7| 0| 0| 0| 0| 123| 0| 0| 0| 0| 0| 0| 0|
12032| 0| 155770| 0| 4| 0| 0| 0| 0| 122| 0| 0| 0| 0| 0| 0| 0|
10991| 0| 161040| 0| 7| 0| 0| 0| 0| 129| 0| 0| 0| 0| 0| 0| 0|
9898| 0| 155503| 0| 9| 0| 0| 0| 0| 125| 0| 0| 0| 0| 0| 0| 0|
12553| 0| 157455| 0| 6| 0| 0| 0| 0| 129| 0| 0| 0| 0| 0| 0| 0|
10983| 0| 157742| 0| 9| 0| 0| 0| 0| 128| 0| 0| 0| 0| 0| 0| 0|
9375| 0| 158226| 0| 8| 0| 0| 0| 0| 115| 0| 0| 0| 0| 0| 0| 0|
11929| 0| 159342| 0| 10| 0| 0| 0| 0| 130| 0| 0| 0| 0| 0| 0| 0|
11046| 0| 158015| 0| 8| 0| 0| 0| 0| 126| 0| 0| 0| 0| 0| 0| 0|
^ permalink raw reply
* Re: [RFC]: xfrm by mark
From: Patrick McHardy @ 2010-02-08 13:30 UTC (permalink / raw)
To: hadi; +Cc: Herbert Xu, David Miller, Timo Teräs, netdev
In-Reply-To: <1265567522.3688.27.camel@bigi>
jamal wrote:
> Hi,
>
> While catching up with old netdev threads I noticed this one:
> http://marc.info/?l=linux-netdev&m=125621829709171&w=2
> I had a chat with Timo and he is too busy on other things at
> the moment. I have some cycles to burn and would like to pursue
> this unless there is objection to the general idea. I am interested
> because I would like to see most of the net stack capable of
> reference by mark. I am posting this so i can get a general
> pulse before i go off coding.
>
> Approach:
> - introduce attribute XFRMA_MARK
> - both SPD and SAD will have new entries ->mark
> - by default mark is 0, so backward compat for
> control add/get/del as well as fast path lookup continues.
If you simply add the mark to the lookup key, it will break
existing setups already using marks. I'd suggest to also add
a mask which is initialized to 0 when no mark attribute is
present.
^ permalink raw reply
* Re: Problem wit route cache
From: Eric Dumazet @ 2010-02-08 13:28 UTC (permalink / raw)
To: Paweł Staszewski; +Cc: Linux Network Development list
In-Reply-To: <4B700EC2.5090207@itcare.pl>
Le lundi 08 février 2010 à 14:16 +0100, Paweł Staszewski a écrit :
> Hello
>
> From some time i have problem with route cache in linux
> this is an info that i have in dmesg:
>
> Route hash chain too long!
> Adjust your secret_interval!
> Route hash chain too long!
> Adjust your secret_interval!
> Route hash chain too long!
> Adjust your secret_interval!
> Route hash chain too long!
> Adjust your secret_interval!
> Route hash chain too long!
> Adjust your secret_interval!
> Route hash chain too long!
> Adjust your secret_interval!
> Route hash chain too long!
> Adjust your secret_interval!
> Route hash chain too long!
> Adjust your secret_interval!
> vlan0811: 9 rebuilds is over limit, route caching disabled
> Route hash chain too long!
> Adjust your secret_interval!
>
> The problem is that change of net.ipv4.route.secret_interval is change
> nothing -- no matter that i set secret_interval from dfault 3600 to 2 or
> 10000 i have always the same info about route cahce is disabled.
> Also i change this parameter net.ipv4.rt_cache_rebuild_count from
> default 4 to 9 and the same info - i try also change this to 12 but also
> this change nothing.
>
> The machine that have this info is:
> 2x Intel(R) Xeon(R) CPU X5450 @ 3.00GHz
> 12GB of RAM
>
Are you running a 64bit kernel ?
What is your kernel version ?
Please send :
# grep . /proc/sys/net/ipv4/route/*
#rtstat -c10 -i1
^ permalink raw reply
* Re: VLAN egress performance
From: Eric Dumazet @ 2010-02-08 13:24 UTC (permalink / raw)
To: Patrick McHardy; +Cc: Mattias Rönnblom, netdev
In-Reply-To: <4B700DF5.1080406@trash.net>
Le lundi 08 février 2010 à 14:13 +0100, Patrick McHardy a écrit :
> Mattias Rönnblom wrote:
> > Hi.
> >
> > I'm running Linux on PC w/ a Core i7 CPU and two Intel 82598 NICs, and
> > I see some anomalies when it comes to egress VLAN performance. I
> > thought maybe someone on this list was interested in my results.
> >
> > I'm running the stock Ubuntu 2.6.31 kernel, but with a newer ixgbe
> > driver (2.0.44.14).
> >
> > The benchmark is IP forwarding with unidirectional UDP flows @ 64 byte
> > packets, and I get:
> >
> > Ingress VLAN Egress VLAN Packet Rate CPU utilization (all cores)
> > No No 5.0 Mpacket/s ~70%
> > Yes No 5.0 Mpacket/s ~75%
> > No Yes 1.4 Mpacket/s ~26%
> > Yes Yes 1.3 Mpacket/s ~26%
> >
> > "VLAN" here mean I've put a VLAN device on top of the real ixgbe
> > device.
> >
> > As you can see, if the egress i/f is a VLAN i/f, the performance is
> > reduced to less than a third. And in the case of egress VLAN, the
> > systems basically only uses one HW thread (with a softirqd process
> > taking up all the time).
> >
> > Enabling lockdep, it looks like execution is serialized to a large
> > extent by contention around the "vlan_netdev_xmit_lock_key" lock.
> >
> > I call this anomaly because I was surprised to see it (as oppose to
> > other performance degradations/scalability issues in the area of
> > multicore and IP traffic handling performance).
>
> 2.6.32 contains VLAN multiqueue support and should scale better.
Yes, patches were added in September 2009, 2.6.32 should be OK.
commit 2f8bc32b7a08502a79e0ccec8697000f2977f2fd
Author: Eric Dumazet <eric.dumazet@gmail.com>
Date: Thu Sep 3 02:19:58 2009 -0700
vlan: enable multiqueue xmits
vlan_dev_hard_start_xmit() & vlan_dev_hwaccel_hard_start_xmit()
select txqueue number 0, instead of using index provided by
skb_get_queue_mapping().
This is not correct after commit 2e59af3dcbdf11635c03f
[vlan: multiqueue vlan device] because
txq->tx_packets & txq->tx_bytes changes are performed on
a single location, and not the right locking.
Fix is to take the appropriate struct netdev_queue pointer
Signed-off-by: Eric Dumazet <eric.dumazet@gmail.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
commit 2e59af3dcbdf11635c03f22bfc9706744465d589
Author: Eric Dumazet <eric.dumazet@gmail.com>
Date: Wed Sep 2 18:03:00 2009 -0700
vlan: multiqueue vlan device
vlan devices are currently not multi-queue capable.
We can do that with a new rtnl_link_ops method,
get_tx_queues(), called from rtnl_create_link()
This new method gets num_tx_queues/real_num_tx_queues
from real device.
register_vlan_device() is also handled.
Signed-off-by: Eric Dumazet <eric.dumazet@gmail.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
^ permalink raw reply
* Problem wit route cache
From: Paweł Staszewski @ 2010-02-08 13:16 UTC (permalink / raw)
To: Linux Network Development list
Hello
From some time i have problem with route cache in linux
this is an info that i have in dmesg:
Route hash chain too long!
Adjust your secret_interval!
Route hash chain too long!
Adjust your secret_interval!
Route hash chain too long!
Adjust your secret_interval!
Route hash chain too long!
Adjust your secret_interval!
Route hash chain too long!
Adjust your secret_interval!
Route hash chain too long!
Adjust your secret_interval!
Route hash chain too long!
Adjust your secret_interval!
Route hash chain too long!
Adjust your secret_interval!
vlan0811: 9 rebuilds is over limit, route caching disabled
Route hash chain too long!
Adjust your secret_interval!
The problem is that change of net.ipv4.route.secret_interval is change
nothing -- no matter that i set secret_interval from dfault 3600 to 2 or
10000 i have always the same info about route cahce is disabled.
Also i change this parameter net.ipv4.rt_cache_rebuild_count from
default 4 to 9 and the same info - i try also change this to 12 but also
this change nothing.
The machine that have this info is:
2x Intel(R) Xeon(R) CPU X5450 @ 3.00GHz
12GB of RAM
Network controllers are:
04:00.0 Ethernet controller: Intel Corporation 82573E Gigabit Ethernet
Controller (Copper) (rev 03)
05:00.0 Ethernet controller: Intel Corporation 82573L Gigabit Ethernet
Controller
And the weird thing is that i need to set affinity for my nics to "ff"
because when i bind network card to one cpu this cpu have always 100%
Router traffic is about 700Mbit/s RX + 700Mbit/s TX on eth0 (with 2
tagged vlans) and the same traffic on eth1 that is untagged
This traffic is forwarded by this router.
Simple topology:
Clients<- ibgp -> [ vlan0811@eth0 + vlan0508@eth0 - BGP process - eth1 ]<- ebgp -> Internet Provisers
Some info about fibtrie:
cat /proc/net/fib_triestat
Basic info: size of leaf: 40 bytes, size of tnode: 56 bytes.
Main:
Aver depth: 2.58
Max depth: 6
Leaves: 297506
Prefixes: 312472
Internal nodes: 69968
1: 35673 2: 14840 3: 10980 4: 4729 5: 2315 6: 956 7: 364 8: 109 9: 1 16: 1
Pointers: 570018
Null ptrs: 202545
Total size: 36990 kB
Counters:
---------
gets = 2797832581
backtracks = 149015808
semantic match passed = 2789993308
semantic match miss = 766703
null node hit= 860359377
skipped node resize = 0
Local:
Aver depth: 3.66
Max depth: 4
Leaves: 15
Prefixes: 16
Internal nodes: 11
1: 8 2: 3
Pointers: 28
Null ptrs: 3
Total size: 3 kB
Counters:
---------
gets = 2792656726
backtracks = 2185449412
semantic match passed = 5895311
semantic match miss = 0
null node hit= 818902
skipped node resize = 0
And interrupts
cat /proc/interrupts
CPU0 CPU1 CPU2 CPU3 CPU4 CPU5 CPU6 CPU7
0: 80 21 32 27 38 39 38 34 IO-APIC-edge timer
1: 0 1 0 0 0 0 1 0 IO-APIC-edge i8042
9: 0 0 0 0 0 0 0 0 IO-APIC-fasteoi acpi
14: 0 0 0 0 0 0 0 0 IO-APIC-edge ide0
15: 0 0 0 0 0 0 0 0 IO-APIC-edge ide1
20: 0 0 0 0 0 0 0 0 IO-APIC-fasteoi uhci_hcd:usb3
21: 0 0 0 0 0 0 0 0 IO-APIC-fasteoi uhci_hcd:usb6
22: 0 0 0 0 0 0 0 0 IO-APIC-fasteoi ehci_hcd:usb1, uhci_hcd:usb5
23: 0 0 0 0 0 0 0 0 IO-APIC-fasteoi ehci_hcd:usb2, uhci_hcd:usb4
29: 73903 136 135 135 134 138 136 136 PCI-MSI-edge ahci
30: 101136854 101509497 101627297 100701508 101684077 101870217 101154425 100604465 PCI-MSI-edge eth0
31: 91772747 92037311 92317341 92231065 91248484 91062342 91778136 92328099 PCI-MSI-edge eth1
32: 0 0 0 0 0 1 1 0 PCI-MSI-edge
33: 0 1 0 0 0 0 0 1 PCI-MSI-edge
34: 0 0 1 0 0 1 0 0 PCI-MSI-edge
35: 1 0 0 1 0 0 0 0 PCI-MSI-edge
36: 0 0 0 0 1 0 0 1 PCI-MSI-edge
37: 1 0 0 0 0 0 1 0 PCI-MSI-edge
38: 0 0 0 1 1 0 0 0 PCI-MSI-edge
39: 0 1 1 0 0 0 0 0 PCI-MSI-edge
NMI: 0 0 0 0 0 0 0 0 Non-maskable interrupts
LOC: 407733660 395964025 431438426 402307801 434522968 420170984 400450633 390318324 Local timer interrupts
SPU: 0 0 0 0 0 0 0 0 Spurious interrupts
PMI: 0 0 0 0 0 0 0 0 Performance monitoring interrupts
PND: 0 0 0 0 0 0 0 0 Performance pending work
RES: 14378 15781 19276 5691 14761 13579 16846 15629 Rescheduling interrupts
CAL: 378 383 92 86 364 354 92 89 Function call interrupts
TLB: 551 577 433 272 602 543 329 683 TLB shootdowns
ERR: 0
MIS: 0
Regards
Pawel
^ permalink raw reply
* Re: VLAN egress performance
From: Patrick McHardy @ 2010-02-08 13:13 UTC (permalink / raw)
To: Mattias Rönnblom; +Cc: netdev
In-Reply-To: <87wrynrk37.fsf@isengard.friendlyfire.se>
Mattias Rönnblom wrote:
> Hi.
>
> I'm running Linux on PC w/ a Core i7 CPU and two Intel 82598 NICs, and
> I see some anomalies when it comes to egress VLAN performance. I
> thought maybe someone on this list was interested in my results.
>
> I'm running the stock Ubuntu 2.6.31 kernel, but with a newer ixgbe
> driver (2.0.44.14).
>
> The benchmark is IP forwarding with unidirectional UDP flows @ 64 byte
> packets, and I get:
>
> Ingress VLAN Egress VLAN Packet Rate CPU utilization (all cores)
> No No 5.0 Mpacket/s ~70%
> Yes No 5.0 Mpacket/s ~75%
> No Yes 1.4 Mpacket/s ~26%
> Yes Yes 1.3 Mpacket/s ~26%
>
> "VLAN" here mean I've put a VLAN device on top of the real ixgbe
> device.
>
> As you can see, if the egress i/f is a VLAN i/f, the performance is
> reduced to less than a third. And in the case of egress VLAN, the
> systems basically only uses one HW thread (with a softirqd process
> taking up all the time).
>
> Enabling lockdep, it looks like execution is serialized to a large
> extent by contention around the "vlan_netdev_xmit_lock_key" lock.
>
> I call this anomaly because I was surprised to see it (as oppose to
> other performance degradations/scalability issues in the area of
> multicore and IP traffic handling performance).
2.6.32 contains VLAN multiqueue support and should scale better.
^ permalink raw reply
* Re: [RFC 3/4] bond: support more Layer 4 protocols
From: Patrick McHardy @ 2010-02-08 13:11 UTC (permalink / raw)
To: Stephen Hemminger; +Cc: David Miller, Jay Vosburgh, netdev, bonding-devel
In-Reply-To: <20100205084240.0bd895ab@s6510>
Stephen Hemminger wrote:
> On Fri, 05 Feb 2010 11:38:05 +0100
> Patrick McHardy <kaber@trash.net> wrote:
>
>> Stephen Hemminger wrote:
>>> +/* Map of protocols with standard ports available to include in hash */
>>> +static const bool has_layer4[256] = {
>>> + [IPPROTO_TCP] = 1,
>>> + [IPPROTO_UDP] = 1,
>>> + [IPPROTO_UDPLITE] = 1,
>>> + [IPPROTO_SCTP] = 1,
>>> + [IPPROTO_DCCP] = 1,
>>> + [IPPROTO_ESP] = 1,
>>> +};
>>> +
>> How about using a bitmap or u8s to keep this more compact?
>
> I thought about that, but couldn't find a clever way to initialize at
> compile time other than hardcoding constants?
Maybe an __init function with a couple of set_bit() calls?
Otherwise I'd suggest to simply use u8.
^ permalink raw reply
* Re: MAX_SKB_FRAGS and GRO
From: Herbert Xu @ 2010-02-08 12:47 UTC (permalink / raw)
To: Anton Blanchard
Cc: Jeff Kirsher, Jesse Brandeburg, Bruce Allan, Alex Duyck,
PJ Waskiewicz, John Ronciak, divy, netdev
In-Reply-To: <20100208100306.GQ32246@kryten>
On Mon, Feb 08, 2010 at 09:03:07PM +1100, Anton Blanchard wrote:
>
> I was looking through the hardware GRO support in various drivers and I think
> we have a couple of issues with PAGE_SIZE > 4k. For example, if we have a 64kB
> page size then MAX_SKB_FRAGS ends up as 3:
>
> #define MAX_SKB_FRAGS (65536/PAGE_SIZE + 2)
>
> This should be fine for hardware and software GSO, but we encounter issues
> with hardware GRO (not sure about software GRO).
>
> In the ixgbe case we use MAX_SKB_FRAGS to program the max number of GRO
> descriptors, even though we assemble GRO packets using ->frag_list:
>
> #if (MAX_SKB_FRAGS > 16)
> rscctrl |= IXGBE_RSCCTL_MAXDESC_16;
> #elif (MAX_SKB_FRAGS > 8)
> rscctrl |= IXGBE_RSCCTL_MAXDESC_8;
> #elif (MAX_SKB_FRAGS > 4)
> rscctrl |= IXGBE_RSCCTL_MAXDESC_4;
> #else
> rscctrl |= IXGBE_RSCCTL_MAXDESC_1;
> #endif
First of all this isn't GRO, but RSC. With GRO we impose extra
restrictions on what packets can be merged while RSC is more
permissive.
In fact I think the ixgbe code may be broken as it is since it's
not marking RSC packets in any way to prevent them from being
forwarded through another interface.
As to your problem with RSC on a 64K page system, I'm sure one
of the Intel developers would be able to help you out.
> Thinking out aloud, would setting a pessimistic value for MAX_SKB_FRAGS
> be one way to fix this? ie:
>
> #define MAX_SKB_FRAGS (65536/4096 + 2)
While I can't think of any serious issues with this, as this is
an entirely ixgbe-specific problem, the fix should probably stay
there.
Cheers,
--
Visit Openswan at http://www.openswan.org/
Email: Herbert Xu ~{PmV>HI~} <herbert@gondor.apana.org.au>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt
^ permalink raw reply
* Re: [PATCH 0/3][v2] tcp: fix ICMP-RTO war
From: Damian Lukowski @ 2010-02-08 12:34 UTC (permalink / raw)
To: David Miller; +Cc: netdev
In-Reply-To: <20100131.233338.254690877.davem@davemloft.net>
Am 01.02.2010, 08:33 Uhr, schrieb David Miller <davem@davemloft.net>:
> From: Damian Lukowski <damian@tvk.rwth-aachen.de>
> Date: Fri, 29 Jan 2010 23:15:51 +0100
>
>> This patches fix the current RTO calculation routine, when
>> srtt and rttvar are zero, yielding an RTO of zero
>> Under some circumstances, TCPs srtt and rttvar are zero,
>> yielding a calculated RTO of zero.
>> This is particularly unfortunate for ICMP based RTO recalculation
>> as introduced in f1ecd5d9e736660 (Revert Backoff [v3]: Revert RTO
>> on ICMP destination unreachable), as it results in RTO retransmission
>> flooding.
>>
>> Thanks to Ilpo Jarvinen for providing debug patches and to
>> Denys Fedoryshchenko for reporting and testing.
>>
>> Signed-off-by: Damian Lukowski <damian@tvk.rwth-aachen.de>
>
> I still haven't seen a detailed enough analysis of why these
> tiny RTOs can come to exist in the first place.
>
> Please show me a list of events, function by function, the value of
> relevant variables and per-socket TCP state, in the TCP stack, that
> show how this ends up happening.
>
> Thanks for all of your work on this so far.
I might have figured it out, but could not verify it, so maybe you can
comment my thought.
When a listening TCP receives a SYN, it will send a SYN+ACK
and wait for an ACK to complete the handshake.
Look at tcp_rcv_state_process::step 5::case SYN_RECV::acceptable
and the code after the comment "tcp_ack considers this ACK as duplicate
and does not calculate rtt".
If the connecting client has disabled timestamps, the rtt statistics
won't be updated here, while the state is changed above.
I printk'ed at the very end of TCP_SYN_RECV and got the following:
state 1 (ESTABLISHED), srtt 0, rttvar 0.
So my suspicion is: If connectivity breaks right after a listening TCP
has completed the handshake without timestamps, and the listening TCP
sends data after establishing the connection, we will get the observed
behaviour.
Regards
Damian
^ permalink raw reply
* Re: smsc911x suspend/resume
From: Steve.Glendinning @ 2010-02-08 12:27 UTC (permalink / raw)
To: Daniel Mack
Cc: davem, Ian.Saturley, LAKML, Mike Rapoport, netdev, netdev-owner,
Vlad.Lyalikov
In-Reply-To: <20100208115316.GA28972@buzzloop.caiaq.de>
> > I don't have the code in front of me (I'm travelling today) but IIRC
> > the driver doesn't place the LAN9220 in a specific power saving state,
> > so as far as the LAN9220 is concerned it's not actually suspended.
>
> It does, I added that awhile back in b6907b0c7. But as I say, this is
> currently hard-coded to D1, and if support for D2 is desired, we would
> need to make that a platform data option.
You're right, and I should remember as I reviewed it!
Steve
^ permalink raw reply
* VLAN egress performance
From: Mattias Rönnblom @ 2010-02-08 12:24 UTC (permalink / raw)
To: netdev
Hi.
I'm running Linux on PC w/ a Core i7 CPU and two Intel 82598 NICs, and
I see some anomalies when it comes to egress VLAN performance. I
thought maybe someone on this list was interested in my results.
I'm running the stock Ubuntu 2.6.31 kernel, but with a newer ixgbe
driver (2.0.44.14).
The benchmark is IP forwarding with unidirectional UDP flows @ 64 byte
packets, and I get:
Ingress VLAN Egress VLAN Packet Rate CPU utilization (all cores)
No No 5.0 Mpacket/s ~70%
Yes No 5.0 Mpacket/s ~75%
No Yes 1.4 Mpacket/s ~26%
Yes Yes 1.3 Mpacket/s ~26%
"VLAN" here mean I've put a VLAN device on top of the real ixgbe
device.
As you can see, if the egress i/f is a VLAN i/f, the performance is
reduced to less than a third. And in the case of egress VLAN, the
systems basically only uses one HW thread (with a softirqd process
taking up all the time).
Enabling lockdep, it looks like execution is serialized to a large
extent by contention around the "vlan_netdev_xmit_lock_key" lock.
I call this anomaly because I was surprised to see it (as oppose to
other performance degradations/scalability issues in the area of
multicore and IP traffic handling performance).
Does anyone know how difficult it would be to resolve this? There's
not actually any synchronization needed between the different cores in
the case of VLAN. Correct? This is just an artefact of how the
implementation is done?
I looked briefly at the code, and I must admit I had some trouble
understanding where the flow in terms of locking.
Best regards,
Mattias
^ permalink raw reply
* Re: smsc911x suspend/resume
From: Steve.Glendinning @ 2010-02-08 11:46 UTC (permalink / raw)
To: Mike Rapoport
Cc: Daniel Mack, LAKML, netdev, Ian.Saturley, Vlad.Lyalikov, davem
In-Reply-To: <4B6FF5E2.6060304@compulab.co.il>
Hi Mike,
> >> I'm trying to make suspend/resume work on OMAP3-based system and I'm
> >> encountering issues with resume of SMSC 9220 chip.
> >> After resume the interface is unusable unless I do 'ifconfig eth0
down'
> >> and 'ifconfig eth0 up'.
> >> I would expect that 'ping <some host>' would work right after resume
> >> without need to bring the interface down and up, but probably I miss
> >> something.
> >> Any help would be appreciated.
> >
> > What happens to your supply voltages when going to suspend? When I
> > implemented the code for the smsc driver, I could only test scenarios
> > where AVDD remains stable during suspend. So the driver might need
some
> > tweaks if that assumption is not true in your case. The SMSC datasheet
> > is quite comprehensive about this topic IIRC.
>
> By AVDD you mean VDD33A?
> Anyway, I have all the supplies except VDDVARIO shut down...
> And the datasheet is indeed comprehensively describes chip power states
> but I haven't found there anything about supplies required to stay on
> during suspend...
I don't have the code in front of me (I'm travelling today) but IIRC
the driver doesn't place the LAN9220 in a specific power saving state,
so as far as the LAN9220 is concerned it's not actually suspended.
It's hard to do this in a "generic" way: some people expect their link
to stay up during suspend, some want suspend to completely power down
the ethernet chip, and some want to use one of several WoL modes.
David: What's the "done thing" here for similar non-pci drivers?
Obviously "doesn't work after suspend" isn't the intended behaviour, so
there's definitely room for improvement!
> > Also have a look at the Raumfeld device patches which use exactly this
> > chip and which can suspend and resume just fine. You'll need to check
> > out Eric's devel branch for that.
Daniel: I haven't seen this, do you have a location for the git tree?
Thanks,
Steve
^ permalink raw reply
* Re: smsc911x suspend/resume
From: Daniel Mack @ 2010-02-08 11:53 UTC (permalink / raw)
To: Steve.Glendinning
Cc: Mike Rapoport, LAKML, netdev, Ian.Saturley, Vlad.Lyalikov, davem
In-Reply-To: <OFE2B2FE18.A8D1D506-ON802576C4.003FCA39-802576C4.0040B683@smsc.com>
On Mon, Feb 08, 2010 at 11:46:39AM +0000, Steve.Glendinning@smsc.com wrote:
> > Anyway, I have all the supplies except VDDVARIO shut down...
> > And the datasheet is indeed comprehensively describes chip power states
> > but I haven't found there anything about supplies required to stay on
> > during suspend...
>
> I don't have the code in front of me (I'm travelling today) but IIRC
> the driver doesn't place the LAN9220 in a specific power saving state,
> so as far as the LAN9220 is concerned it's not actually suspended.
It does, I added that awhile back in b6907b0c7. But as I say, this is
currently hard-coded to D1, and if support for D2 is desired, we would
need to make that a platform data option.
> It's hard to do this in a "generic" way: some people expect their link
> to stay up during suspend, some want suspend to completely power down
> the ethernet chip, and some want to use one of several WoL modes.
>
> David: What's the "done thing" here for similar non-pci drivers?
> Obviously "doesn't work after suspend" isn't the intended behaviour, so
> there's definitely room for improvement!
>
> > > Also have a look at the Raumfeld device patches which use exactly this
> > > chip and which can suspend and resume just fine. You'll need to check
> > > out Eric's devel branch for that.
>
> Daniel: I haven't seen this, do you have a location for the git tree?
git://git.kernel.org/pub/scm/linux/kernel/git/ycmiao/pxa-linux-2.6.git
See the 'devel' branch. But there's actually nothing exciting in there,
it's just a simple platform support.
Daniel
^ permalink raw reply
* Re: smsc911x suspend/resume
From: Daniel Mack @ 2010-02-08 11:46 UTC (permalink / raw)
To: Mike Rapoport; +Cc: Steve Glendinning, netdev, LAKML
In-Reply-To: <4B6FF5E2.6060304@compulab.co.il>
On Mon, Feb 08, 2010 at 01:30:42PM +0200, Mike Rapoport wrote:
> Daniel Mack wrote:
> > What happens to your supply voltages when going to suspend? When I
> > implemented the code for the smsc driver, I could only test scenarios
> > where AVDD remains stable during suspend. So the driver might need some
> > tweaks if that assumption is not true in your case. The SMSC datasheet
> > is quite comprehensive about this topic IIRC.
>
> By AVDD you mean VDD33A?
Yes.
> Anyway, I have all the supplies except VDDVARIO shut down...
> And the datasheet is indeed comprehensively describes chip power states
> but I haven't found there anything about supplies required to stay on
> during suspend...
In fact the datasheet doesn't really state that, and the chip seems
to expect the power domains to remain switched on during suspend, and
all power switching is done internally. Also the table in "7.4 Power
Consumption (Device and System Components)" describes it like that.
So it all comes down to the question of how low-power you need to be,
and whether you need wake-on-lan or not. What's currently implemented is
D1, but adding support for D2 should be easy.
Daniel
^ permalink raw reply
* Re: smsc911x suspend/resume
From: Mike Rapoport @ 2010-02-08 11:30 UTC (permalink / raw)
To: Daniel Mack; +Cc: Steve Glendinning, netdev, LAKML
In-Reply-To: <20100208101123.GW28972@buzzloop.caiaq.de>
Daniel Mack wrote:
> On Mon, Feb 08, 2010 at 11:42:27AM +0200, Mike Rapoport wrote:
>> I'm trying to make suspend/resume work on OMAP3-based system and I'm
>> encountering issues with resume of SMSC 9220 chip.
>> After resume the interface is unusable unless I do 'ifconfig eth0 down'
>> and 'ifconfig eth0 up'.
>> I would expect that 'ping <some host>' would work right after resume
>> without need to bring the interface down and up, but probably I miss
>> something.
>> Any help would be appreciated.
>
> What happens to your supply voltages when going to suspend? When I
> implemented the code for the smsc driver, I could only test scenarios
> where AVDD remains stable during suspend. So the driver might need some
> tweaks if that assumption is not true in your case. The SMSC datasheet
> is quite comprehensive about this topic IIRC.
By AVDD you mean VDD33A?
Anyway, I have all the supplies except VDDVARIO shut down...
And the datasheet is indeed comprehensively describes chip power states
but I haven't found there anything about supplies required to stay on
during suspend...
> Also have a look at the Raumfeld device patches which use exactly this
> chip and which can suspend and resume just fine. You'll need to check
> out Eric's devel branch for that.
>
> Daniel
>
> --
> To unsubscribe from this list: send the line "unsubscribe netdev" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
--
Sincerely yours,
Mike.
^ permalink raw reply
* Re: [Bug #15196] kmem_cache_create: duplicate cache ccid2_h
From: Rafael J. Wysocki @ 2010-02-08 11:04 UTC (permalink / raw)
To: Heinz Diehl, Xiaotian Feng
Cc: Linux Kernel Mailing List, Kernel Testers List, Maciej Rutecki,
NetDev, David Miller
In-Reply-To: <20100208071515.GA6473-HjJ2MNWy62to6+H+lsi3Gti2O/JbrIOy@public.gmane.org>
On Monday 08 February 2010, Heinz Diehl wrote:
> On 08.02.2010, Rafael J. Wysocki wrote:
>
> > This message has been generated automatically as a part of a summary report
> > of recent regressions.
> >
> > The following bug entry is on the current list of known regressions
> > from 2.6.32. Please verify if it still should be listed and let the tracking team
> > know (either way).
> >
> >
> > Bug-Entry : http://bugzilla.kernel.org/show_bug.cgi?id=15196
> > Subject : kmem_cache_create: duplicate cache ccid2_h
> > Submitter : Heinz Diehl <htd-HjJ2MNWy62to6+H+lsi3Gti2O/JbrIOy@public.gmane.org>
> > Date : 2010-01-30 18:33 (9 days old)
> > References : http://marc.info/?l=linux-kernel&m=126487640324942&w=4
> > Handled-By : Xiaotian Feng <xtfeng-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>
> > Patch : http://patchwork.kernel.org/patch/77601/
>
> Yes, it's still present in -rc7.
Hmm. The bug is in net/dccp/ccid.c, so it looks like something for the
networking guys to merge.
Any chance to send a singed-off version of the patch do David?
Rafael
^ permalink raw reply
* Re: smsc911x suspend/resume
From: Daniel Mack @ 2010-02-08 10:11 UTC (permalink / raw)
To: Mike Rapoport; +Cc: Steve Glendinning, netdev, LAKML
In-Reply-To: <4B6FDC83.9090205@compulab.co.il>
On Mon, Feb 08, 2010 at 11:42:27AM +0200, Mike Rapoport wrote:
> I'm trying to make suspend/resume work on OMAP3-based system and I'm
> encountering issues with resume of SMSC 9220 chip.
> After resume the interface is unusable unless I do 'ifconfig eth0 down'
> and 'ifconfig eth0 up'.
> I would expect that 'ping <some host>' would work right after resume
> without need to bring the interface down and up, but probably I miss
> something.
> Any help would be appreciated.
What happens to your supply voltages when going to suspend? When I
implemented the code for the smsc driver, I could only test scenarios
where AVDD remains stable during suspend. So the driver might need some
tweaks if that assumption is not true in your case. The SMSC datasheet
is quite comprehensive about this topic IIRC.
Also have a look at the Raumfeld device patches which use exactly this
chip and which can suspend and resume just fine. You'll need to check
out Eric's devel branch for that.
Daniel
^ permalink raw reply
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox