* Re: [PATCH net-next] tcp: introduce tcp_try_coalesce
From: Eric Dumazet @ 2012-04-24 2:13 UTC (permalink / raw)
To: Neal Cardwell
Cc: David Miller, netdev, Tom Herbert, Maciej Żenczykowski,
Ilpo Järvinen
In-Reply-To: <CADVnQyngwym7z_zSi8h2U_1e_wVCwGMPZScywWFWM2AYvxyqjw@mail.gmail.com>
On Mon, 2012-04-23 at 21:13 -0400, Neal Cardwell wrote:
> On Mon, Apr 23, 2012 at 1:11 PM, Eric Dumazet <eric.dumazet@gmail.com> wrote:
> > From: Eric Dumazet <edumazet@google.com>
> >
> > commit c8628155ece3 (tcp: reduce out_of_order memory use) took care of
> > coalescing tcp segments provided by legacy devices (linear skbs)
> >
> > We extend this idea to fragged skbs, as their truesize can be heavy.
> >
> > ixgbe for example uses 256+1024+PAGE_SIZE/2 = 3328 bytes per segment.
> >
> > Use this coalescing strategy for receive queue too.
> >
> > This contributes to reduce number of tcp collapses, at minimal cost, and
> > reduces memory overhead and packets drops.
>
> The mechanics look solid, but I'm a little concerned about the
> potential added overhead for the new case where tcp_try_coalesce()
> does a skb_copy_bits() for in-order data that it is coalescing at the
> end of the sk_receive_queue. Do you have any performance numbers for
> this case to help suggest whether this added copy is a concern?
>
> neal
This never happens on connections where performance matters : skb head
can only contains one full mss segment.
This part is only used on wifi devices, where skb head is really fat.
^ permalink raw reply
* linux-next: manual merge of the net-next tree with the infiniband tree
From: Stephen Rothwell @ 2012-04-24 2:06 UTC (permalink / raw)
To: David Miller, netdev-u79uwXL29TY76Z2rM5mHXA
Cc: linux-next-u79uwXL29TY76Z2rM5mHXA,
linux-kernel-u79uwXL29TY76Z2rM5mHXA, Roland Dreier,
linux-rdma-u79uwXL29TY76Z2rM5mHXA, Parav Pandit, Ajit Khaparde
[-- Attachment #1: Type: text/plain, Size: 1974 bytes --]
Hi all,
Today's linux-next merge of the net-next tree got a conflict in
drivers/net/ethernet/emulex/benet/be.h between commit c72adfd767af
("be2net: Add functionality to support RoCE driver") from the infiniband
tree and commit 42f11cf20cc5 ("be2net: fix ethtool get settings") from
the net-next tree.
Just context changes. I fixed it up (see below) and can carry the fix as
necessary.
--
Cheers,
Stephen Rothwell sfr-3FnU+UHB4dNDw9hX6IcOSA@public.gmane.org
diff --cc drivers/net/ethernet/emulex/benet/be.h
index 7bb2e97,ad69cf8..0000000
--- a/drivers/net/ethernet/emulex/benet/be.h
+++ b/drivers/net/ethernet/emulex/benet/be.h
@@@ -379,22 -394,7 +396,18 @@@ struct be_adapter
u32 rx_fc; /* Rx flow control */
u32 tx_fc; /* Tx flow control */
bool stats_cmd_sent;
- int link_speed;
- u8 port_type;
- u8 transceiver;
- u8 autoneg;
u8 generation; /* BladeEngine ASIC generation */
+ u32 if_type;
+ struct {
+ u8 __iomem *base; /* Door Bell */
+ u32 size;
+ u32 total_size;
+ u64 io_addr;
+ } roce_db;
+ u32 num_msix_roce_vec;
+ struct ocrdma_dev *ocrdma_dev;
+ struct list_head entry;
+
u32 flash_status;
struct completion flash_compl;
@@@ -606,17 -597,5 +620,18 @@@ extern void be_link_status_update(struc
extern void be_parse_stats(struct be_adapter *adapter);
extern int be_load_fw(struct be_adapter *adapter, u8 *func);
extern bool be_is_wol_supported(struct be_adapter *adapter);
+ extern bool be_pause_supported(struct be_adapter *adapter);
+
+/*
+ * internal function to initialize-cleanup roce device.
+ */
+extern void be_roce_dev_add(struct be_adapter *);
+extern void be_roce_dev_remove(struct be_adapter *);
+
+/*
+ * internal function to open-close roce device during ifup-ifdown.
+ */
+extern void be_roce_dev_open(struct be_adapter *);
+extern void be_roce_dev_close(struct be_adapter *);
+
#endif /* BE_H */
[-- Attachment #2: Type: application/pgp-signature, Size: 836 bytes --]
^ permalink raw reply
* Re: [PATCH] net ax25: Fix the build when sysctl support is disabled.
From: Randy Dunlap @ 2012-04-24 1:28 UTC (permalink / raw)
To: Eric W. Biederman
Cc: David Miller, Stephen Rothwell, linux-next, LKML, netdev,
linux-hams, Ralf Baechle
In-Reply-To: <m1obqirn5e.fsf_-_@fess.ebiederm.org>
On 04/23/2012 05:25 PM, Eric W. Biederman wrote:
>
> Randy Dunlap <rdunlap@xenotime.net> reported:
>
>> On 04/23/2012 12:07 AM, Stephen Rothwell wrote:
>>
>>> Hi all,
>>>
>>> Changes since 20120420:
>>
>>
>> include/net/ax25.h:447:75: error: expected ';' before '}' token
>>
>> static inline int ax25_register_dev_sysctl(ax25_dev *ax25_dev) { return 0 };
>> static inline void ax25_unregister_dev_sysctl(ax25_dev *ax25_dev) {};
>>
>> First function: move ';' inside braces.
>> Second function: drop the ';'.
>
> Put the semicolons where it makes sense.
>
> Signed-off-by: Eric W. Biederman <ebiederm@xmission.com>
Acked-by: Randy Dunlap <rdunlap@xenotime.net>
Thanks.
> ---
>
> Resent to include David Miller.
>
> include/net/ax25.h | 4 ++--
> 1 files changed, 2 insertions(+), 2 deletions(-)
>
> diff --git a/include/net/ax25.h b/include/net/ax25.h
> index 8a7a122..5d23521 100644
> --- a/include/net/ax25.h
> +++ b/include/net/ax25.h
> @@ -444,8 +444,8 @@ extern void ax25_uid_free(void);
> extern int ax25_register_dev_sysctl(ax25_dev *ax25_dev);
> extern void ax25_unregister_dev_sysctl(ax25_dev *ax25_dev);
> #else
> -static inline int ax25_register_dev_sysctl(ax25_dev *ax25_dev) { return 0 };
> -static inline void ax25_unregister_dev_sysctl(ax25_dev *ax25_dev) {};
> +static inline int ax25_register_dev_sysctl(ax25_dev *ax25_dev) { return 0; }
> +static inline void ax25_unregister_dev_sysctl(ax25_dev *ax25_dev) {}
> #endif /* CONFIG_SYSCTL */
>
> #endif
--
~Randy
^ permalink raw reply
* Re: [PATCH net-next] tcp: introduce tcp_try_coalesce
From: Neal Cardwell @ 2012-04-24 1:13 UTC (permalink / raw)
To: Eric Dumazet
Cc: David Miller, netdev, Tom Herbert, Maciej Żenczykowski,
Ilpo Järvinen
In-Reply-To: <1335201102.5205.28.camel@edumazet-glaptop>
On Mon, Apr 23, 2012 at 1:11 PM, Eric Dumazet <eric.dumazet@gmail.com> wrote:
> From: Eric Dumazet <edumazet@google.com>
>
> commit c8628155ece3 (tcp: reduce out_of_order memory use) took care of
> coalescing tcp segments provided by legacy devices (linear skbs)
>
> We extend this idea to fragged skbs, as their truesize can be heavy.
>
> ixgbe for example uses 256+1024+PAGE_SIZE/2 = 3328 bytes per segment.
>
> Use this coalescing strategy for receive queue too.
>
> This contributes to reduce number of tcp collapses, at minimal cost, and
> reduces memory overhead and packets drops.
The mechanics look solid, but I'm a little concerned about the
potential added overhead for the new case where tcp_try_coalesce()
does a skb_copy_bits() for in-order data that it is coalescing at the
end of the sk_receive_queue. Do you have any performance numbers for
this case to help suggest whether this added copy is a concern?
neal
^ permalink raw reply
* [PATCH] net ax25: Fix the build when sysctl support is disabled.
From: Eric W. Biederman @ 2012-04-24 0:25 UTC (permalink / raw)
To: David Miller
Cc: Stephen Rothwell, linux-next, LKML, netdev, linux-hams,
Ralf Baechle, Randy Dunlap
In-Reply-To: <4F95828A.8050403@xenotime.net>
Randy Dunlap <rdunlap@xenotime.net> reported:
> On 04/23/2012 12:07 AM, Stephen Rothwell wrote:
>
>> Hi all,
>>
>> Changes since 20120420:
>
>
> include/net/ax25.h:447:75: error: expected ';' before '}' token
>
> static inline int ax25_register_dev_sysctl(ax25_dev *ax25_dev) { return 0 };
> static inline void ax25_unregister_dev_sysctl(ax25_dev *ax25_dev) {};
>
> First function: move ';' inside braces.
> Second function: drop the ';'.
Put the semicolons where it makes sense.
Signed-off-by: Eric W. Biederman <ebiederm@xmission.com>
---
Resent to include David Miller.
include/net/ax25.h | 4 ++--
1 files changed, 2 insertions(+), 2 deletions(-)
diff --git a/include/net/ax25.h b/include/net/ax25.h
index 8a7a122..5d23521 100644
--- a/include/net/ax25.h
+++ b/include/net/ax25.h
@@ -444,8 +444,8 @@ extern void ax25_uid_free(void);
extern int ax25_register_dev_sysctl(ax25_dev *ax25_dev);
extern void ax25_unregister_dev_sysctl(ax25_dev *ax25_dev);
#else
-static inline int ax25_register_dev_sysctl(ax25_dev *ax25_dev) { return 0 };
-static inline void ax25_unregister_dev_sysctl(ax25_dev *ax25_dev) {};
+static inline int ax25_register_dev_sysctl(ax25_dev *ax25_dev) { return 0; }
+static inline void ax25_unregister_dev_sysctl(ax25_dev *ax25_dev) {}
#endif /* CONFIG_SYSCTL */
#endif
--
1.7.2.5
^ permalink raw reply related
* [PATCH] net ax25: Fix the build when sysctl support is disabled.
From: Eric W. Biederman @ 2012-04-24 0:23 UTC (permalink / raw)
To: Randy Dunlap
Cc: Stephen Rothwell, linux-next, LKML, netdev, linux-hams,
Ralf Baechle
In-Reply-To: <4F95828A.8050403@xenotime.net>
Randy Dunlap <rdunlap@xenotime.net> reported:
> On 04/23/2012 12:07 AM, Stephen Rothwell wrote:
>
>> Hi all,
>>
>> Changes since 20120420:
>
>
> include/net/ax25.h:447:75: error: expected ';' before '}' token
>
> static inline int ax25_register_dev_sysctl(ax25_dev *ax25_dev) { return 0 };
> static inline void ax25_unregister_dev_sysctl(ax25_dev *ax25_dev) {};
>
> First function: move ';' inside braces.
> Second function: drop the ';'.
Put the semicolons where it makes sense.
Signed-off-by: Eric W. Biederman <ebiederm@xmission.com>
---
include/net/ax25.h | 4 ++--
1 files changed, 2 insertions(+), 2 deletions(-)
diff --git a/include/net/ax25.h b/include/net/ax25.h
index 8a7a122..5d23521 100644
--- a/include/net/ax25.h
+++ b/include/net/ax25.h
@@ -444,8 +444,8 @@ extern void ax25_uid_free(void);
extern int ax25_register_dev_sysctl(ax25_dev *ax25_dev);
extern void ax25_unregister_dev_sysctl(ax25_dev *ax25_dev);
#else
-static inline int ax25_register_dev_sysctl(ax25_dev *ax25_dev) { return 0 };
-static inline void ax25_unregister_dev_sysctl(ax25_dev *ax25_dev) {};
+static inline int ax25_register_dev_sysctl(ax25_dev *ax25_dev) { return 0; }
+static inline void ax25_unregister_dev_sysctl(ax25_dev *ax25_dev) {}
#endif /* CONFIG_SYSCTL */
#endif
--
1.7.2.5
^ permalink raw reply related
* Re: linux-next: Tree for Apr 23 (ax.25 sysctl)
From: Eric W. Biederman @ 2012-04-24 0:16 UTC (permalink / raw)
To: Randy Dunlap
Cc: Stephen Rothwell, linux-next, LKML, netdev, linux-hams,
Ralf Baechle
In-Reply-To: <4F95828A.8050403@xenotime.net>
Randy Dunlap <rdunlap@xenotime.net> writes:
> On 04/23/2012 12:07 AM, Stephen Rothwell wrote:
>
>> Hi all,
>>
>> Changes since 20120420:
>
>
>
> include/net/ax25.h:447:75: error: expected ';' before '}' token
>
>
> static inline int ax25_register_dev_sysctl(ax25_dev *ax25_dev) { return 0 };
> static inline void ax25_unregister_dev_sysctl(ax25_dev *ax25_dev) {};
>
> Is this from Eric B.'s sysctl patches?
Sort of. I made the function per device and return 0.
I missed the inner ';' when I made the change. The outer '};' has
been there for years and has been weird but harmless.
Patch to fix it in a moment.
Eric
> First function: move ';' inside braces.
> Second function: drop the ';'.
^ permalink raw reply
* Re: [PATCH 02/16] mm: sl[au]b: Add knowledge of PFMEMALLOC reserve pages
From: David Rientjes @ 2012-04-23 23:51 UTC (permalink / raw)
To: Mel Gorman
Cc: Andrew Morton, Linux-MM, Linux-Netdev, LKML, David Miller,
Neil Brown, Peter Zijlstra, Mike Christie, Eric B Munson
In-Reply-To: <1334578624-23257-3-git-send-email-mgorman@suse.de>
On Mon, 16 Apr 2012, Mel Gorman wrote:
> diff --git a/mm/page_alloc.c b/mm/page_alloc.c
> index 280eabe..0fa2c72 100644
> --- a/mm/page_alloc.c
> +++ b/mm/page_alloc.c
> @@ -1463,6 +1463,7 @@ failed:
> #define ALLOC_HARDER 0x10 /* try to alloc harder */
> #define ALLOC_HIGH 0x20 /* __GFP_HIGH set */
> #define ALLOC_CPUSET 0x40 /* check for correct cpuset */
> +#define ALLOC_PFMEMALLOC 0x80 /* Caller has PF_MEMALLOC set */
>
> #ifdef CONFIG_FAIL_PAGE_ALLOC
>
> @@ -2208,16 +2209,22 @@ gfp_to_alloc_flags(gfp_t gfp_mask)
> } else if (unlikely(rt_task(current)) && !in_interrupt())
> alloc_flags |= ALLOC_HARDER;
>
> - if (likely(!(gfp_mask & __GFP_NOMEMALLOC))) {
> - if (!in_interrupt() &&
> - ((current->flags & PF_MEMALLOC) ||
> - unlikely(test_thread_flag(TIF_MEMDIE))))
> + if ((current->flags & PF_MEMALLOC) ||
> + unlikely(test_thread_flag(TIF_MEMDIE))) {
> + alloc_flags |= ALLOC_PFMEMALLOC;
> +
> + if (likely(!(gfp_mask & __GFP_NOMEMALLOC)) && !in_interrupt())
> alloc_flags |= ALLOC_NO_WATERMARKS;
> }
>
> return alloc_flags;
> }
>
> +bool gfp_pfmemalloc_allowed(gfp_t gfp_mask)
> +{
> + return !!(gfp_to_alloc_flags(gfp_mask) & ALLOC_PFMEMALLOC);
> +}
> +
> static inline struct page *
> __alloc_pages_slowpath(gfp_t gfp_mask, unsigned int order,
> struct zonelist *zonelist, enum zone_type high_zoneidx,
> @@ -2407,8 +2414,16 @@ nopage:
> got_pg:
> if (kmemcheck_enabled)
> kmemcheck_pagealloc_alloc(page, order, gfp_mask);
> - return page;
>
> + /*
> + * page->pfmemalloc is set when the caller had PFMEMALLOC set or is
> + * been OOM killed. The expectation is that the caller is taking
> + * steps that will free more memory. The caller should avoid the
> + * page being used for !PFMEMALLOC purposes.
> + */
> + page->pfmemalloc = !!(alloc_flags & ALLOC_PFMEMALLOC);
> +
> + return page;
> }
>
> /*
I think this is slightly inconsistent if the page allocation succeeded
without needing ALLOC_NO_WATERMARKS, meaning that page was allocated above
the min watermark. That's possible if the slowpath's first call to
get_page_from_freelist() succeeds without needing
__alloc_pages_high_priority(). So perhaps we need to do something like
got_pg_memalloc:
...
page->pfmemalloc = !!(alloc_flags & ALLOC_PFMEMALLOC);
got_pg:
if (kmemcheck_enabled)
kmemcheck_pagealloc_alloc(page, order, gfp_mask);
return page;
and use got_pg_memalloc everywhere we currently use got_pg other than the
when it succeeds with ALLOC_NO_WATERMARKS.
> @@ -2459,6 +2474,8 @@ retry_cpuset:
> page = __alloc_pages_slowpath(gfp_mask, order,
> zonelist, high_zoneidx, nodemask,
> preferred_zone, migratetype);
> + else
> + page->pfmemalloc = false;
>
> trace_mm_page_alloc(page, order, gfp_mask, migratetype);
>
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Fight unfair telecom internet charges in Canada: sign http://stopthemeter.ca/
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply
* Re: [PATCH 01/16] mm: Serialize access to min_free_kbytes
From: David Rientjes @ 2012-04-23 23:50 UTC (permalink / raw)
To: Mel Gorman
Cc: Andrew Morton, Linux-MM, Linux-Netdev, LKML, David Miller,
Neil Brown, Peter Zijlstra, Mike Christie, Eric B Munson
In-Reply-To: <1334578624-23257-2-git-send-email-mgorman@suse.de>
On Mon, 16 Apr 2012, Mel Gorman wrote:
> There is a race between the min_free_kbytes sysctl, memory hotplug
> and transparent hugepage support enablement. Memory hotplug uses a
> zonelists_mutex to avoid a race when building zonelists. Reuse it to
> serialise watermark updates.
>
> [a.p.zijlstra@chello.nl: Older patch fixed the race with spinlock]
> Signed-off-by: Mel Gorman <mgorman@suse.de>
> Reviewed-by: Rik van Riel <riel@redhat.com>
Acked-by: David Rientjes <rientjes@google.com>
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Fight unfair telecom internet charges in Canada: sign http://stopthemeter.ca/
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply
* Re: [PATCH] net sysctl: Add place holder functions for when sysctl support is compiled out of the kernel.
From: David Miller @ 2012-04-23 23:24 UTC (permalink / raw)
To: rdunlap; +Cc: ebiederm, sfr, linux-next, linux-kernel, netdev
In-Reply-To: <4F95D9F5.3040407@xenotime.net>
From: Randy Dunlap <rdunlap@xenotime.net>
Date: Mon, 23 Apr 2012 15:38:45 -0700
> On 04/23/2012 03:13 PM, Eric W. Biederman wrote:
>
>>
>> Randy Dunlap <rdunlap@xenotime.net> reported:
>>> On 04/23/2012 12:07 AM, Stephen Rothwell wrote:
>>>
>>>> Hi all,
>>>>
>>>> Changes since 20120420:
>>>
>>>
>>>
>>> ERROR: "unregister_net_sysctl_table" [net/phonet/phonet.ko] undefined!
>>> ERROR: "register_net_sysctl" [net/phonet/phonet.ko] undefined!
>>>
>>> when CONFIG_SYSCTL is not enabled.
>>
>> Add static inline stub functions to gracefully handle the case when sysctl
>> support is not present.
>>
>> Signed-off-by: Eric W. Biederman <ebiederm@xmission.com>
>
>
> Yep, that works.
>
> Acked-by: Randy Dunlap <rdunlap@xenotime.net>
Applied, thanks everyone.
^ permalink raw reply
* Re: [PATCH 2/2] e1000e: Remove special case for 82573/82574 ASPM L1 disablement
From: Jeff Kirsher @ 2012-04-23 23:18 UTC (permalink / raw)
To: Chris Boot; +Cc: e1000-devel, netdev, nix, linux-kernel
In-Reply-To: <1335216578-21542-3-git-send-email-bootc@bootc.net>
[-- Attachment #1.1: Type: text/plain, Size: 521 bytes --]
On Mon, 2012-04-23 at 22:29 +0100, Chris Boot wrote:
> For the 82573, ASPM L1 gets disabled wholesale so this special-case
> code
> is not required. For the 82574 the previous patch does the same as for
> the 82573, disabling L1 on the adapter. Thus, this code is no longer
> required and can be removed.
>
> Signed-off-by: Chris Boot <bootc@bootc.net>
> ---
> drivers/net/ethernet/intel/e1000e/netdev.c | 8 --------
> 1 file changed, 8 deletions(-)
I have added the patch to my queue, thanks Chris!
[-- Attachment #1.2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 836 bytes --]
[-- Attachment #2: Type: text/plain, Size: 272 bytes --]
------------------------------------------------------------------------------
For Developers, A Lot Can Happen In A Second.
Boundary is the first to Know...and Tell You.
Monitor Your Applications in Ultra-Fine Resolution. Try it FREE!
http://p.sf.net/sfu/Boundary-d2dvs2
[-- Attachment #3: Type: text/plain, Size: 257 bytes --]
_______________________________________________
E1000-devel mailing list
E1000-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/e1000-devel
To learn more about Intel® Ethernet, visit http://communities.intel.com/community/wired
^ permalink raw reply
* Re: [PATCH 1/2] e1000e: Disable ASPM L1 on 82574
From: Jeff Kirsher @ 2012-04-23 23:18 UTC (permalink / raw)
To: Chris Boot; +Cc: e1000-devel, netdev, nix, linux-kernel
In-Reply-To: <1335216578-21542-2-git-send-email-bootc@bootc.net>
[-- Attachment #1.1: Type: text/plain, Size: 617 bytes --]
On Mon, 2012-04-23 at 22:29 +0100, Chris Boot wrote:
>
> ASPM on the 82574 causes trouble. Currently the driver disables L0s
> for
> this NIC but only disables L1 if the MTU is >1500. This patch simply
> causes L1 to be disabled regardless of the MTU setting.
>
> Signed-off-by: Chris Boot <bootc@bootc.net>
> Cc: "Wyborny, Carolyn" <carolyn.wyborny@intel.com>
> Cc: Nix <nix@esperi.org.uk>
> Link: https://lkml.org/lkml/2012/3/19/362
> ---
> drivers/net/ethernet/intel/e1000e/82571.c | 3 ++-
> 1 file changed, 2 insertions(+), 1 deletion(-)
I have added the patch to my queue, thanks Chris!
[-- Attachment #1.2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 836 bytes --]
[-- Attachment #2: Type: text/plain, Size: 272 bytes --]
------------------------------------------------------------------------------
For Developers, A Lot Can Happen In A Second.
Boundary is the first to Know...and Tell You.
Monitor Your Applications in Ultra-Fine Resolution. Try it FREE!
http://p.sf.net/sfu/Boundary-d2dvs2
[-- Attachment #3: Type: text/plain, Size: 257 bytes --]
_______________________________________________
E1000-devel mailing list
E1000-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/e1000-devel
To learn more about Intel® Ethernet, visit http://communities.intel.com/community/wired
^ permalink raw reply
* Re: [PATCH RFC 0/2] e1000e: 82574 also needs ASPM L1 completely disabled
From: Jesse Brandeburg @ 2012-04-23 23:11 UTC (permalink / raw)
To: Chris Boot; +Cc: e1000-devel, netdev, nix, linux-kernel
In-Reply-To: <1335216578-21542-1-git-send-email-bootc@bootc.net>
[-- Attachment #1.1: Type: text/plain, Size: 763 bytes --]
On Mon, 23 Apr 2012 22:29:36 +0100
Chris Boot <bootc@bootc.net> wrote:
> Please note I haven't as-yet tested this code at all, but I do know that
> disabling ASPM L1 on these NICs (using setpci) fixes the hangs that I
> have been seeing on my Supermicro servers with X9SCL-F boards. I hope to
> get the chance to install an updated kernel on my two afftected servers
> later this week.
>
> Chris Boot (2):
> e1000e: Disable ASPM L1 on 82574
> e1000e: Remove special case for 82573/82574 ASPM L1 disablement
Thanks Chris, we are going to take a look over the patches and Jeff
Kirsher should apply them to our internal testing tree.
Please let us know the results of your testing, we will let you know if
we see any issues as well.
Jesse
[-- Attachment #1.2: signature.asc --]
[-- Type: application/pgp-signature, Size: 834 bytes --]
[-- Attachment #2: Type: text/plain, Size: 272 bytes --]
------------------------------------------------------------------------------
For Developers, A Lot Can Happen In A Second.
Boundary is the first to Know...and Tell You.
Monitor Your Applications in Ultra-Fine Resolution. Try it FREE!
http://p.sf.net/sfu/Boundary-d2dvs2
[-- Attachment #3: Type: text/plain, Size: 257 bytes --]
_______________________________________________
E1000-devel mailing list
E1000-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/e1000-devel
To learn more about Intel® Ethernet, visit http://communities.intel.com/community/wired
^ permalink raw reply
* Re: [RFC v4] Add TCP encap_rcv hook (repost)
From: Jesse Gross @ 2012-04-23 22:59 UTC (permalink / raw)
To: Simon Horman
Cc: dev-yBygre7rU0TnMu66kgdUjQ, eric.dumazet-Re5JQEeQqe8AvxtiuMwx3w,
netdev-u79uwXL29TY76Z2rM5mHXA, jhs-jkUAjuhPggJWk0Htik3J/w,
stephen.hemminger-ZtmgI6mnKB3QT0dZR+AlfA,
shemminger-ZtmgI6mnKB3QT0dZR+AlfA, David Miller
In-Reply-To: <20120423223255.GG580-/R6kz+dDXgpPR4JQBCEnsQ@public.gmane.org>
On Mon, Apr 23, 2012 at 3:32 PM, Simon Horman <horms@verge.net.au> wrote:
> On Mon, Apr 23, 2012 at 02:38:07PM -0700, Jesse Gross wrote:
>> On Mon, Apr 23, 2012 at 2:08 PM, David Miller <davem@davemloft.net> wrote:
>> > From: Jesse Gross <jesse@nicira.com>
>> > Date: Mon, 23 Apr 2012 13:53:42 -0700
>> >
>> >> On Mon, Apr 23, 2012 at 1:13 PM, David Miller <davem@davemloft.net> wrote:
>> >>> From: Jesse Gross <jesse@nicira.com>
>> >>> Date: Mon, 23 Apr 2012 13:08:49 -0700
>> >>>
>> >>>> Assuming that the TCP stack generates large TSO frames on transmit
>> >>>> (which could be the local stack; something sent by a VM; or packets
>> >>>> received, coalesced by GRO and then encapsulated by STT) then you can
>> >>>> just prepend the STT header (possibly slightly adjusting things like
>> >>>> requested MSS, number of segments, etc. slightly). After that it's
>> >>>> possible to just output the resulting frame through the IP stack like
>> >>>> all tunnels do today.
>> >>>
>> >>> Which seems to potentially suggest a stronger intergration of the STT
>> >>> tunnel transmit path into our IP stack rather than the approach Simon
>> >>> is taking
>> >>
>> >> Did you have something in mind?
>> >
>> > A normal bonafide tunnel netdevice driver like GRE instead of the
>> > openvswitch approach Simon is using.
>>
>> Ahh, yes, that I agree with. Independent of this, there's work being
>> done to make it so that OVS can use the normal in-tree tunneling code
>> and not need its own. Once that's done I expect that STT will follow
>> the same model.
>
> Hi Jesse,
>
> I am wondering how firm the plans to on allowing OVS to use in-tree tunnel
> code are. I'm happy to move my efforts over to an in-tree STT implementation
> but ultimately I would like to get STT running in conjunction with OVS.
I would say that it's a firm goal but the implementation probably
still has a ways to go. Kyle Mestery (CC'ed) has volunteered to work
on this in support of adding VXLAN, which needs some additional
flexibility that this approach would also provide. You might want to
talk to him to see if there are ways that you guys can work together
on it if you are interested. Having better integration with upstream
tunneling is definitely a step that OVS needs to make and sooner would
be better than later.
_______________________________________________
dev mailing list
dev@openvswitch.org
http://openvswitch.org/mailman/listinfo/dev
^ permalink raw reply
* Re: [PATCH] net sysctl: Add place holder functions for when sysctl support is compiled out of the kernel.
From: Randy Dunlap @ 2012-04-23 22:38 UTC (permalink / raw)
To: Eric W. Biederman
Cc: David Miller, Stephen Rothwell, linux-next, LKML, netdev
In-Reply-To: <m1lilmt7v5.fsf_-_@fess.ebiederm.org>
On 04/23/2012 03:13 PM, Eric W. Biederman wrote:
>
> Randy Dunlap <rdunlap@xenotime.net> reported:
>> On 04/23/2012 12:07 AM, Stephen Rothwell wrote:
>>
>>> Hi all,
>>>
>>> Changes since 20120420:
>>
>>
>>
>> ERROR: "unregister_net_sysctl_table" [net/phonet/phonet.ko] undefined!
>> ERROR: "register_net_sysctl" [net/phonet/phonet.ko] undefined!
>>
>> when CONFIG_SYSCTL is not enabled.
>
> Add static inline stub functions to gracefully handle the case when sysctl
> support is not present.
>
> Signed-off-by: Eric W. Biederman <ebiederm@xmission.com>
Yep, that works.
Acked-by: Randy Dunlap <rdunlap@xenotime.net>
Thanks.
> ---
> include/net/net_namespace.h | 15 ++++++++++++---
> 1 files changed, 12 insertions(+), 3 deletions(-)
>
> diff --git a/include/net/net_namespace.h b/include/net/net_namespace.h
> index 3ee4a3d..ac9195e 100644
> --- a/include/net/net_namespace.h
> +++ b/include/net/net_namespace.h
> @@ -284,11 +284,20 @@ struct ctl_table_header;
>
> #ifdef CONFIG_SYSCTL
> extern int net_sysctl_init(void);
> -#else
> -static inline int net_sysctl_init(void) { return 0; }
> -#endif
> extern struct ctl_table_header *register_net_sysctl(struct net *net,
> const char *path, struct ctl_table *table);
> extern void unregister_net_sysctl_table(struct ctl_table_header *header);
> +#else
> +static inline int net_sysctl_init(void) { return 0; }
> +static inline struct ctl_table_header *register_net_sysctl(struct net *net,
> + const char *path, struct ctl_table *table)
> +{
> + return NULL;
> +}
> +static inline void unregister_net_sysctl_table(struct ctl_table_header *header)
> +{
> +}
> +#endif
> +
>
> #endif /* __NET_NET_NAMESPACE_H */
--
~Randy
^ permalink raw reply
* Re: [RFC v4] Add TCP encap_rcv hook (repost)
From: Simon Horman @ 2012-04-23 22:32 UTC (permalink / raw)
To: Jesse Gross
Cc: dev-yBygre7rU0TnMu66kgdUjQ, eric.dumazet-Re5JQEeQqe8AvxtiuMwx3w,
netdev-u79uwXL29TY76Z2rM5mHXA, jhs-jkUAjuhPggJWk0Htik3J/w,
stephen.hemminger-ZtmgI6mnKB3QT0dZR+AlfA,
shemminger-ZtmgI6mnKB3QT0dZR+AlfA, David Miller
In-Reply-To: <CAEP_g=-52GOr3LzbUB+97ftNQBZV=7NWXqfWN6GMfq5KmdO25A-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
On Mon, Apr 23, 2012 at 02:38:07PM -0700, Jesse Gross wrote:
> On Mon, Apr 23, 2012 at 2:08 PM, David Miller <davem@davemloft.net> wrote:
> > From: Jesse Gross <jesse@nicira.com>
> > Date: Mon, 23 Apr 2012 13:53:42 -0700
> >
> >> On Mon, Apr 23, 2012 at 1:13 PM, David Miller <davem@davemloft.net> wrote:
> >>> From: Jesse Gross <jesse@nicira.com>
> >>> Date: Mon, 23 Apr 2012 13:08:49 -0700
> >>>
> >>>> Assuming that the TCP stack generates large TSO frames on transmit
> >>>> (which could be the local stack; something sent by a VM; or packets
> >>>> received, coalesced by GRO and then encapsulated by STT) then you can
> >>>> just prepend the STT header (possibly slightly adjusting things like
> >>>> requested MSS, number of segments, etc. slightly). After that it's
> >>>> possible to just output the resulting frame through the IP stack like
> >>>> all tunnels do today.
> >>>
> >>> Which seems to potentially suggest a stronger intergration of the STT
> >>> tunnel transmit path into our IP stack rather than the approach Simon
> >>> is taking
> >>
> >> Did you have something in mind?
> >
> > A normal bonafide tunnel netdevice driver like GRE instead of the
> > openvswitch approach Simon is using.
>
> Ahh, yes, that I agree with. Independent of this, there's work being
> done to make it so that OVS can use the normal in-tree tunneling code
> and not need its own. Once that's done I expect that STT will follow
> the same model.
Hi Jesse,
I am wondering how firm the plans to on allowing OVS to use in-tree tunnel
code are. I'm happy to move my efforts over to an in-tree STT implementation
but ultimately I would like to get STT running in conjunction with OVS.
_______________________________________________
dev mailing list
dev@openvswitch.org
http://openvswitch.org/mailman/listinfo/dev
^ permalink raw reply
* Re: [PATCH 2/2 net-next] tcp: sk_add_backlog() is too agressive for TCP
From: Rick Jones @ 2012-04-23 22:16 UTC (permalink / raw)
To: Eric Dumazet
Cc: David Miller, netdev, therbert, ncardwell, maze, ycheng,
ilpo.jarvinen
In-Reply-To: <1335218707.5205.87.camel@edumazet-glaptop>
On 04/23/2012 03:05 PM, Eric Dumazet wrote:
> On Mon, 2012-04-23 at 14:51 -0700, Rick Jones wrote:
>> On 04/23/2012 02:30 PM, Eric Dumazet wrote:
>
>>> Yet, in the small time it takes to perform this operation, softirq can
>>> queue up to 300 packets coming from the other side.
>>
>> There is more to it than just queue-up 16 KB right?
>
> At full rate, we send 825.000 packets per second, and should receive
> 412.000 ACKS per second if receiver is standard TCP.
>
> The ACK are not smooth, because receiver also have a huge backlog issue
> and can send train of ACKS. (I have seen backlogs on receiver using more
> than 500 us to be processed)
>
> If the copyin(16KB) from user to kernel takes some us (preempt,
> irqs...), its pretty easy to catch an ACK train in this window.
Is it at all possible to have the copies happen without the connection
being locked? If indeed it is possible to be held-off with the
connection locked for the better part of 3/4 of a millisecond, just what
will that do to 40 or 100 GbE? If you've been seeing queues of 300 ACKs
at 10 GbE that would be 3000 at 100 GbE, and assuming those are all in a
2048 byte buffer thats 6MB just of ACKs. I suppose 100GbE does mean
non-trivial quantities of buffering anyway but that does still seem
rather high.
rick
thank goodness for GRO's ACK stretching as an ACK avoidance heuristic I
guess...
^ permalink raw reply
* [PATCH] net sysctl: Add place holder functions for when sysctl support is compiled out of the kernel.
From: Eric W. Biederman @ 2012-04-23 22:13 UTC (permalink / raw)
To: David Miller; +Cc: Stephen Rothwell, linux-next, LKML, netdev, Randy Dunlap
In-Reply-To: <4F958598.6080305@xenotime.net>
Randy Dunlap <rdunlap@xenotime.net> reported:
> On 04/23/2012 12:07 AM, Stephen Rothwell wrote:
>
>> Hi all,
>>
>> Changes since 20120420:
>
>
>
> ERROR: "unregister_net_sysctl_table" [net/phonet/phonet.ko] undefined!
> ERROR: "register_net_sysctl" [net/phonet/phonet.ko] undefined!
>
> when CONFIG_SYSCTL is not enabled.
Add static inline stub functions to gracefully handle the case when sysctl
support is not present.
Signed-off-by: Eric W. Biederman <ebiederm@xmission.com>
---
include/net/net_namespace.h | 15 ++++++++++++---
1 files changed, 12 insertions(+), 3 deletions(-)
diff --git a/include/net/net_namespace.h b/include/net/net_namespace.h
index 3ee4a3d..ac9195e 100644
--- a/include/net/net_namespace.h
+++ b/include/net/net_namespace.h
@@ -284,11 +284,20 @@ struct ctl_table_header;
#ifdef CONFIG_SYSCTL
extern int net_sysctl_init(void);
-#else
-static inline int net_sysctl_init(void) { return 0; }
-#endif
extern struct ctl_table_header *register_net_sysctl(struct net *net,
const char *path, struct ctl_table *table);
extern void unregister_net_sysctl_table(struct ctl_table_header *header);
+#else
+static inline int net_sysctl_init(void) { return 0; }
+static inline struct ctl_table_header *register_net_sysctl(struct net *net,
+ const char *path, struct ctl_table *table)
+{
+ return NULL;
+}
+static inline void unregister_net_sysctl_table(struct ctl_table_header *header)
+{
+}
+#endif
+
#endif /* __NET_NET_NAMESPACE_H */
--
1.7.2.5
^ permalink raw reply related
* [PATCH net] net/davinci_emac: fix failing PHY connect attempts
From: Anatolij Gustschin @ 2012-04-23 22:06 UTC (permalink / raw)
To: netdev; +Cc: David S. Miller
PHY connect attempts fail if no PHY id is specified in the emac platform
data and another mdio bus has been registered before 'davinci_mdio' bus. In
this case when configuring the interface, there will be an attempt to
connect to already attached PHY on the previously registered mdio bus:
net eth1: PHY already attached
net eth1: could not connect to phy smsc911x-0:01
IP-Config: Failed to open eth1
IP-Config: Device `eth1' not found
Fix this by modifying match_first_device() to match first PHY device
on 'davinci_mdio' bus.
Signed-off-by: Anatolij Gustschin <agust@denx.de>
---
drivers/net/ethernet/ti/davinci_emac.c | 2 +-
1 files changed, 1 insertions(+), 1 deletions(-)
diff --git a/drivers/net/ethernet/ti/davinci_emac.c b/drivers/net/ethernet/ti/davinci_emac.c
index 174a334..08aff1a 100644
--- a/drivers/net/ethernet/ti/davinci_emac.c
+++ b/drivers/net/ethernet/ti/davinci_emac.c
@@ -1511,7 +1511,7 @@ static int emac_devioctl(struct net_device *ndev, struct ifreq *ifrq, int cmd)
static int match_first_device(struct device *dev, void *data)
{
- return 1;
+ return !strncmp(dev_name(dev), "davinci_mdio", 12);
}
/**
--
1.7.1
^ permalink raw reply related
* Re: [PATCH 2/2 net-next] tcp: sk_add_backlog() is too agressive for TCP
From: Eric Dumazet @ 2012-04-23 22:05 UTC (permalink / raw)
To: Rick Jones
Cc: David Miller, netdev, therbert, ncardwell, maze, ycheng,
ilpo.jarvinen
In-Reply-To: <4F95CECF.6030901@hp.com>
On Mon, 2012-04-23 at 14:51 -0700, Rick Jones wrote:
> On 04/23/2012 02:30 PM, Eric Dumazet wrote:
> > Yet, in the small time it takes to perform this operation, softirq can
> > queue up to 300 packets coming from the other side.
>
> There is more to it than just queue-up 16 KB right?
At full rate, we send 825.000 packets per second, and should receive
412.000 ACKS per second if receiver is standard TCP.
The ACK are not smooth, because receiver also have a huge backlog issue
and can send train of ACKS. (I have seen backlogs on receiver using more
than 500 us to be processed)
If the copyin(16KB) from user to kernel takes some us (preempt,
irqs...), its pretty easy to catch an ACK train in this window.
^ permalink raw reply
* Re: [PATCH 2/2 net-next] tcp: sk_add_backlog() is too agressive for TCP
From: Rick Jones @ 2012-04-23 21:56 UTC (permalink / raw)
To: Eric Dumazet
Cc: David Miller, netdev, therbert, ncardwell, maze, ycheng,
ilpo.jarvinen
In-Reply-To: <4F95CECF.6030901@hp.com>
On 04/23/2012 02:51 PM, Rick Jones wrote:
> On 04/23/2012 02:30 PM, Eric Dumazet wrote:
>> On Mon, 2012-04-23 at 13:57 -0700, Rick Jones wrote:
>>> Probably better to call that something other than 16K buffers - the send
>>> size was probably 16K, which reflected SO_SNDBUF at the time the data
>>> socket was created, but clearly SO_SNDBUF grew in that timeframe.
>>>
>>
>>
>> Maybe I was not clear : Application does sendmsg() of 16KB buffers.
>
> I'd probably call that a 16K send test. The root of the issue being
> there being "send buffers" and "send socket buffers" (and their receive
> versions).
>
> My "canonical" test - at least one that appears in most of my
> contemporary scripts uses a 64K send size for the bulk transfer tests. I
> switch back-and-forth between tests which allow the socket buffer size
> to be determined automagically, and those where I set both sides' socket
> buffers to 1M via the test-specific -s and -S options. In "netperf
> speak" those would probably be "x64K" and "1Mx64k" respectively. More
> generally "<socket buffer size>x<send size>" (I rarely set/specify the
> receive size in those tests, leaving it at whatever SO_RCVBUF is at the
> start.
>
>> Yet, in the small time it takes to perform this operation, softirq can
>> queue up to 300 packets coming from the other side.
>
> There is more to it than just queue-up 16 KB right?
I should have added that 300 ACKs seems huge as a backlog. At
ack-every-other that is 300 * 1448 * 2 or 868800 bytes worth of ACKs.
That sounds like a great deal more than just one 16KB send's worth of
being held-off. I mean at 10Gbe speeds (using your 54 usec for 64KB)
that represents data which took something like three quarters of a
millisecond to transmit on the wire.
rick
^ permalink raw reply
* Re: [PATCH 2/2 net-next] tcp: sk_add_backlog() is too agressive for TCP
From: Rick Jones @ 2012-04-23 21:51 UTC (permalink / raw)
To: Eric Dumazet
Cc: David Miller, netdev, therbert, ncardwell, maze, ycheng,
ilpo.jarvinen
In-Reply-To: <1335216631.5205.71.camel@edumazet-glaptop>
On 04/23/2012 02:30 PM, Eric Dumazet wrote:
> On Mon, 2012-04-23 at 13:57 -0700, Rick Jones wrote:
>> Probably better to call that something other than 16K buffers - the send
>> size was probably 16K, which reflected SO_SNDBUF at the time the data
>> socket was created, but clearly SO_SNDBUF grew in that timeframe.
>>
>
>
> Maybe I was not clear : Application does sendmsg() of 16KB buffers.
I'd probably call that a 16K send test. The root of the issue being
there being "send buffers" and "send socket buffers" (and their receive
versions).
My "canonical" test - at least one that appears in most of my
contemporary scripts uses a 64K send size for the bulk transfer tests.
I switch back-and-forth between tests which allow the socket buffer size
to be determined automagically, and those where I set both sides' socket
buffers to 1M via the test-specific -s and -S options. In "netperf
speak" those would probably be "x64K" and "1Mx64k" respectively. More
generally "<socket buffer size>x<send size>" (I rarely set/specify the
receive size in those tests, leaving it at whatever SO_RCVBUF is at the
start.
> Yet, in the small time it takes to perform this operation, softirq can
> queue up to 300 packets coming from the other side.
There is more to it than just queue-up 16 KB right?
rick
^ permalink raw reply
* Re: [RFC v4] Add TCP encap_rcv hook (repost)
From: Jesse Gross @ 2012-04-23 21:38 UTC (permalink / raw)
To: David Miller
Cc: shemminger, horms, jhs, stephen.hemminger, netdev, dev,
eric.dumazet
In-Reply-To: <20120423.170817.1103719420692884446.davem@davemloft.net>
On Mon, Apr 23, 2012 at 2:08 PM, David Miller <davem@davemloft.net> wrote:
> From: Jesse Gross <jesse@nicira.com>
> Date: Mon, 23 Apr 2012 13:53:42 -0700
>
>> On Mon, Apr 23, 2012 at 1:13 PM, David Miller <davem@davemloft.net> wrote:
>>> From: Jesse Gross <jesse@nicira.com>
>>> Date: Mon, 23 Apr 2012 13:08:49 -0700
>>>
>>>> Assuming that the TCP stack generates large TSO frames on transmit
>>>> (which could be the local stack; something sent by a VM; or packets
>>>> received, coalesced by GRO and then encapsulated by STT) then you can
>>>> just prepend the STT header (possibly slightly adjusting things like
>>>> requested MSS, number of segments, etc. slightly). After that it's
>>>> possible to just output the resulting frame through the IP stack like
>>>> all tunnels do today.
>>>
>>> Which seems to potentially suggest a stronger intergration of the STT
>>> tunnel transmit path into our IP stack rather than the approach Simon
>>> is taking
>>
>> Did you have something in mind?
>
> A normal bonafide tunnel netdevice driver like GRE instead of the
> openvswitch approach Simon is using.
Ahh, yes, that I agree with. Independent of this, there's work being
done to make it so that OVS can use the normal in-tree tunneling code
and not need its own. Once that's done I expect that STT will follow
the same model.
^ permalink raw reply
* Re: [PATCH 2/2 net-next] tcp: sk_add_backlog() is too agressive for TCP
From: Eric Dumazet @ 2012-04-23 21:38 UTC (permalink / raw)
To: David Miller
Cc: rick.jones2, netdev, therbert, ncardwell, maze, ycheng,
ilpo.jarvinen
In-Reply-To: <20120423.170101.1369764871919045849.davem@davemloft.net>
On Mon, 2012-04-23 at 17:01 -0400, David Miller wrote:
> From: Eric Dumazet <eric.dumazet@gmail.com>
> Date: Mon, 23 Apr 2012 22:37:26 +0200
>
> > We could try to coalesce ACKs before backlogging them. I'll work on
> > this.
>
> Great idea, although I wonder about the effect this could have on RTT
> measurements. Instead of having N RTT measurements, we'd have just
> one.
>
> Granted, what happens right now wrt. RTT measurements with such huge
> ACK backlogs isn't all that nice either.
>
> Ideally, perhaps, we'd do a timestamp diff at the time we insert the
> packet into the backlog. That way we wouldn't gain the RTT inaccuracy
> introduced by such queueing delays and ACK backlogs.
>
> Another way to look at it is that the coalesced scheme would actually
> improve RTT measurements, since the most accurate (and least
> "delayed") of the timestamps would be the only one processed :-)
The big part of the work is not doing the coalesce, but also counting
the number of ACKS that are going to be carried into TCP stack if we
want cwnd being updated correctly.
Basically I'll have to add a new skb field (in cb[]) to properly count
number of ACKS 'included' in a single packet.
About the RTT, some congestion modules need TCP_CONG_RTT_STAMP but time
is taken when backlog processing is done, that is after backlog in/out
^ permalink raw reply
* Re: [PATCH 2/2 net-next] tcp: sk_add_backlog() is too agressive for TCP
From: Eric Dumazet @ 2012-04-23 21:30 UTC (permalink / raw)
To: Rick Jones
Cc: David Miller, netdev, therbert, ncardwell, maze, ycheng,
ilpo.jarvinen
In-Reply-To: <4F95C22D.3010908@hp.com>
On Mon, 2012-04-23 at 13:57 -0700, Rick Jones wrote:
> On 04/23/2012 01:37 PM, Eric Dumazet wrote:
> > In my 10Gbit tests (standard netperf using 16K buffers), I've seen
> > backlogs of 300 ACK packets...
>
> Probably better to call that something other than 16K buffers - the send
> size was probably 16K, which reflected SO_SNDBUF at the time the data
> socket was created, but clearly SO_SNDBUF grew in that timeframe.
>
Maybe I was not clear : Application does sendmsg() of 16KB buffers.
Yet, in the small time it takes to perform this operation, softirq can
queue up to 300 packets coming from the other side.
^ permalink raw reply
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox