* Re: [PATCH][PPPOL2TP]: Fix SMP oops in pppol2tp driver
From: James Chapman @ 2008-02-20 16:02 UTC (permalink / raw)
To: Jarek Poplawski; +Cc: David Miller, Paul Mackerras, netdev
In-Reply-To: <20080219230640.GA2755@ami.dom.local>
Jarek Poplawski wrote:
> On Mon, Feb 18, 2008 at 10:09:24PM +0000, James Chapman wrote:
> ...
>> Unfortunately the ISP's syslog stops. But I've been able to borrow
>> two Quad Xeon boxes and have reproduced the problem.
>>
>> Here's a new version of the patch. The patch avoids disabling irqs
>> and fixes the sk_dst_get() usage that DaveM mentioned. But even with
>> this patch, lockdep still complains if hundreds of ppp sessions are
>> inserted into a tunnel as rapidly as possible (lockdep trace is below).
>> I can stop these errors by wrapping the call to ppp_input() in
>> pppol2tp_recv_dequeue_skb() with local_irq_save/restore. What is a
>> better fix?
>
> I send here my proposal: it's intended for testing and to check one of
> possible solutions here. IMHO your lockdep reports show there is no
> use to change anything around sk_dst_lock: it would need the global
> change of this lock to fix this problem. So the fix should be done
> around pch->upl lock and this means changing ppp_generic.
Hmm, I need to study the lockdep report again. It seems I'm misreading
the lockdep output. :(
> In the patch below I've used trylock in places which seem to allow
> for skipping some things (while config is changed only) or simply
> don't need this lock because there is no ppp struct. This could be
> modified to add some waiting loop if necessary. Another option is to
> change the write side of this lock: it looks like more vulnerable if
> something missed because there are more locks involved, but probably
> should be enough to solve this problem too.
>
> I think pppol2tp need to be first checked only with hlist_lock bh
> patch, unless there were some lockdep reports on these other locks
> too. (BTW, I added ppp maintainer to CC - I hope we get Paul's opinion
> on this.)
I tried your ppp_generic patch with only the hlist_lock bh patch in
pppol2tp and it seems to fix the ppp create/delete issue. However, when
I added much more traffic into the test (flood pings over ppp interfaces
while repeatedly creating/deleting the L2TP (PPP) sessions) I get a soft
lockup detected in pppol2tp_xmit() after anything between 1 minute and
an hour. :( I'm investigating that now.
Thanks for your help!
> (testing patch #1)
> ---
>
> drivers/net/ppp_generic.c | 33 +++++++++++++++++++++++----------
> 1 files changed, 23 insertions(+), 10 deletions(-)
>
> diff --git a/drivers/net/ppp_generic.c b/drivers/net/ppp_generic.c
> index 4dc5b4b..5cbc534 100644
> --- a/drivers/net/ppp_generic.c
> +++ b/drivers/net/ppp_generic.c
> @@ -1473,7 +1473,7 @@ void
> ppp_input(struct ppp_channel *chan, struct sk_buff *skb)
> {
> struct channel *pch = chan->ppp;
> - int proto;
> + int proto, locked;
>
> if (!pch || skb->len == 0) {
> kfree_skb(skb);
> @@ -1481,8 +1481,13 @@ ppp_input(struct ppp_channel *chan, struct sk_buff *skb)
> }
>
> proto = PPP_PROTO(skb);
> - read_lock_bh(&pch->upl);
> - if (!pch->ppp || proto >= 0xc000 || proto == PPP_CCPFRAG) {
> + /*
> + * We use trylock to avoid dependency between soft-irq-safe upl lock
> + * and soft-irq-unsafe sk_dst_lock.
> + */
> + local_bh_disable();
> + locked = read_trylock(&pch->upl);
> + if (!locked || !pch->ppp || proto >= 0xc000 || proto == PPP_CCPFRAG) {
> /* put it on the channel queue */
> skb_queue_tail(&pch->file.rq, skb);
> /* drop old frames if queue too long */
> @@ -1493,7 +1498,10 @@ ppp_input(struct ppp_channel *chan, struct sk_buff *skb)
> } else {
> ppp_do_recv(pch->ppp, skb, pch);
> }
> - read_unlock_bh(&pch->upl);
> +
> + if (locked)
> + read_unlock(&pch->upl);
> + local_bh_enable();
> }
>
> /* Put a 0-length skb in the receive queue as an error indication */
> @@ -1506,16 +1514,18 @@ ppp_input_error(struct ppp_channel *chan, int code)
> if (!pch)
> return;
>
> - read_lock_bh(&pch->upl);
> - if (pch->ppp) {
> + /* a trylock comment in ppp_input() */
> + local_bh_disable();
> + if (read_trylock(&pch->upl) && pch->ppp) {
> skb = alloc_skb(0, GFP_ATOMIC);
> if (skb) {
> skb->len = 0; /* probably unnecessary */
> skb->cb[0] = code;
> ppp_do_recv(pch->ppp, skb, pch);
> }
> + read_unlock(&pch->upl);
> }
> - read_unlock_bh(&pch->upl);
> + local_bh_enable();
> }
>
> /*
> @@ -2044,10 +2054,13 @@ int ppp_unit_number(struct ppp_channel *chan)
> int unit = -1;
>
> if (pch) {
> - read_lock_bh(&pch->upl);
> - if (pch->ppp)
> + /* a trylock comment in ppp_input() */
> + local_bh_disable();
> + if (read_trylock(&pch->upl) && pch->ppp) {
> unit = pch->ppp->file.index;
> - read_unlock_bh(&pch->upl);
> + read_unlock(&pch->upl);
> + }
> + local_bh_enable();
> }
> return unit;
> }
> --
--
James Chapman
Katalix Systems Ltd
http://www.katalix.com
Catalysts for your Embedded Linux software development
^ permalink raw reply
* Re: [PATCH 04/04] smc91x: add insw/outsw to default config
From: Nicolas Pitre @ 2008-02-20 16:00 UTC (permalink / raw)
To: Magnus Damm; +Cc: netdev, lethal, akpm
In-Reply-To: <20080220095821.22645.60604.sendpatchset@clockwork.opensource.se>
On Wed, 20 Feb 2008, Magnus Damm wrote:
> This patch makes sure SMC_insw()/SMC_outsw() are defined for the
> default configuration. Without this change BUG()s will be triggered
> when using 16-bit only platform data and the default configuration.
>
> Signed-off-by: Magnus Damm <damm@igel.co.jp>
You should have introduced this patch as 3/4 instead of 4/4 so to make
sure the series won't create a non functional kernel between 3/4 and
4/4.
Nicolas
^ permalink raw reply
* Re: [PATCH 02/04] smc91x: introduce platform data flags
From: Nicolas Pitre @ 2008-02-20 15:58 UTC (permalink / raw)
To: Magnus Damm; +Cc: netdev, lethal, akpm
In-Reply-To: <20080220095802.22645.44691.sendpatchset@clockwork.opensource.se>
On Wed, 20 Feb 2008, Magnus Damm wrote:
> This patch introduces struct smc91x_platdata and modifies the driver so
> bus width is checked during run time using SMC_nBIT() instead of
> SMC_CAN_USE_nBIT.
>
> Signed-off-by: Magnus Damm <damm@igel.co.jp>
> ---
NAK.
The SMC91C111 (for example) is often used on devices which have a CPU
clock barely higher than the network throughput, hence it is crutial for
those devices to have the most efficient access possible to the chip or
performance will suffer. This is the main reason behind the heavily
macroized register access so things are always optimized for the data
bus capabilities at compile time.
This patch introduces a runtimes test on lp->cfg.flags for every
register access even on those platforms not using the platform data
based bus configuration at all.
I think you should add a SMC_DYNAMIC_BUS_CONFIG and redefine SMC_nBITS()
so they dereference cfg.flags only when it is defined.
Nicolas
^ permalink raw reply
* Re: [PATCH] SUNRPC: Mark buffer used for debug printks with __maybe_unused
From: Joe Perches @ 2008-02-20 15:56 UTC (permalink / raw)
To: Pavel Emelyanov; +Cc: J. Bruce Fields, Linux Netdev List
In-Reply-To: <47BC4908.7010104@openvz.org>
> Sigh... Why is that better than a strait declaration with attribute?
If at some point there's a gcc'ism to remove a maybe_unused
variable from the stack declaration, you only have change
the macro.
cheers, Joe
^ permalink raw reply
* Re: [PATCH 01/04] smc91x: pass along private data
From: Nicolas Pitre @ 2008-02-20 15:42 UTC (permalink / raw)
To: Magnus Damm; +Cc: netdev, lethal, akpm
In-Reply-To: <20080220095751.22645.15949.sendpatchset@clockwork.opensource.se>
On Wed, 20 Feb 2008, Magnus Damm wrote:
> Pass a private data pointer to macros and functions. This makes it easy
> to later on make run time decisions. This patch does not change any logic.
> These changes should be optimized away during compilation.
>
> Signed-off-by: Magnus Damm <damm@igel.co.jp>
> ---
> --- 0001/drivers/net/smc91x.c
> +++ work/drivers/net/smc91x.c 2008-02-20 16:52:48.000000000 +0900
> @@ -220,23 +220,23 @@ static void PRINT_PKT(u_char *buf, int l
>
>
> /* this enables an interrupt in the interrupt mask register */
> -#define SMC_ENABLE_INT(x) do { \
> +#define SMC_ENABLE_INT(priv, x) do { \
> unsigned char mask; \
> - spin_lock_irq(&lp->lock); \
> - mask = SMC_GET_INT_MASK(); \
> + spin_lock_irq(&priv->lock); \
> + mask = SMC_GET_INT_MASK(priv); \
Since "lp" is already used all over the place, could you simply use "lp"
for the macro argument name as well instead of "priv"? This will make
the code more uniform and reduce the patch size.
Nicolas
^ permalink raw reply
* [PATCH 06/28] mm: serialize access to min_free_kbytes
From: Peter Zijlstra @ 2008-02-20 14:46 UTC (permalink / raw)
To: Linus Torvalds, Andrew Morton, linux-kernel, linux-mm, netdev,
trond.myklebust
Cc: Peter Zijlstra
In-Reply-To: <20080220144610.548202000@chello.nl>
[-- Attachment #1: mm-setup_per_zone_pages_min.patch --]
[-- Type: text/plain, Size: 1836 bytes --]
There is a small race between the procfs caller and the memory hotplug caller
of setup_per_zone_pages_min(). Not a big deal, but the next patch will add yet
another caller. Time to close the gap.
Signed-off-by: Peter Zijlstra <a.p.zijlstra@chello.nl>
---
mm/page_alloc.c | 16 +++++++++++++---
1 file changed, 13 insertions(+), 3 deletions(-)
Index: linux-2.6/mm/page_alloc.c
===================================================================
--- linux-2.6.orig/mm/page_alloc.c
+++ linux-2.6/mm/page_alloc.c
@@ -116,6 +116,7 @@ static char * const zone_names[MAX_NR_ZO
"Movable",
};
+static DEFINE_SPINLOCK(min_free_lock);
int min_free_kbytes = 1024;
unsigned long __meminitdata nr_kernel_pages;
@@ -4087,12 +4088,12 @@ static void setup_per_zone_lowmem_reserv
}
/**
- * setup_per_zone_pages_min - called when min_free_kbytes changes.
+ * __setup_per_zone_pages_min - called when min_free_kbytes changes.
*
* Ensures that the pages_{min,low,high} values for each zone are set correctly
* with respect to min_free_kbytes.
*/
-void setup_per_zone_pages_min(void)
+static void __setup_per_zone_pages_min(void)
{
unsigned long pages_min = min_free_kbytes >> (PAGE_SHIFT - 10);
unsigned long lowmem_pages = 0;
@@ -4147,6 +4148,15 @@ void setup_per_zone_pages_min(void)
calculate_totalreserve_pages();
}
+void setup_per_zone_pages_min(void)
+{
+ unsigned long flags;
+
+ spin_lock_irqsave(&min_free_lock, flags);
+ __setup_per_zone_pages_min();
+ spin_unlock_irqrestore(&min_free_lock, flags);
+}
+
/*
* Initialise min_free_kbytes.
*
@@ -4182,7 +4192,7 @@ static int __init init_per_zone_pages_mi
min_free_kbytes = 128;
if (min_free_kbytes > 65536)
min_free_kbytes = 65536;
- setup_per_zone_pages_min();
+ __setup_per_zone_pages_min();
setup_per_zone_lowmem_reserve();
return 0;
}
--
^ permalink raw reply
* [PATCH 24/28] nfs: remove mempools
From: Peter Zijlstra @ 2008-02-20 14:46 UTC (permalink / raw)
To: Linus Torvalds, Andrew Morton, linux-kernel, linux-mm, netdev,
trond.myklebust
Cc: Peter Zijlstra
In-Reply-To: <20080220144610.548202000@chello.nl>
[-- Attachment #1: nfs-no-mempool.patch --]
[-- Type: text/plain, Size: 4919 bytes --]
With the introduction of the shared dirty page accounting in .19, NFS should
not be able to surpise the VM with all dirty pages. Thus it should always be
able to free some memory. Hence no more need for mempools.
Signed-off-by: Peter Zijlstra <a.p.zijlstra@chello.nl>
---
fs/nfs/read.c | 15 +++------------
fs/nfs/write.c | 27 +++++----------------------
2 files changed, 8 insertions(+), 34 deletions(-)
Index: linux-2.6/fs/nfs/read.c
===================================================================
--- linux-2.6.orig/fs/nfs/read.c
+++ linux-2.6/fs/nfs/read.c
@@ -33,13 +33,10 @@ static const struct rpc_call_ops nfs_rea
static const struct rpc_call_ops nfs_read_full_ops;
static struct kmem_cache *nfs_rdata_cachep;
-static mempool_t *nfs_rdata_mempool;
-
-#define MIN_POOL_READ (32)
struct nfs_read_data *nfs_readdata_alloc(unsigned int pagecount)
{
- struct nfs_read_data *p = mempool_alloc(nfs_rdata_mempool, GFP_NOFS);
+ struct nfs_read_data *p = kmem_cache_alloc(nfs_rdata_cachep, GFP_NOFS);
if (p) {
memset(p, 0, sizeof(*p));
@@ -50,7 +47,7 @@ struct nfs_read_data *nfs_readdata_alloc
else {
p->pagevec = kcalloc(pagecount, sizeof(struct page *), GFP_NOFS);
if (!p->pagevec) {
- mempool_free(p, nfs_rdata_mempool);
+ kmem_cache_free(nfs_rdata_cachep, p);
p = NULL;
}
}
@@ -63,7 +60,7 @@ static void nfs_readdata_rcu_free(struct
struct nfs_read_data *p = container_of(head, struct nfs_read_data, task.u.tk_rcu);
if (p && (p->pagevec != &p->page_array[0]))
kfree(p->pagevec);
- mempool_free(p, nfs_rdata_mempool);
+ kmem_cache_free(nfs_rdata_cachep, p);
}
static void nfs_readdata_free(struct nfs_read_data *rdata)
@@ -595,16 +592,10 @@ int __init nfs_init_readpagecache(void)
if (nfs_rdata_cachep == NULL)
return -ENOMEM;
- nfs_rdata_mempool = mempool_create_slab_pool(MIN_POOL_READ,
- nfs_rdata_cachep);
- if (nfs_rdata_mempool == NULL)
- return -ENOMEM;
-
return 0;
}
void nfs_destroy_readpagecache(void)
{
- mempool_destroy(nfs_rdata_mempool);
kmem_cache_destroy(nfs_rdata_cachep);
}
Index: linux-2.6/fs/nfs/write.c
===================================================================
--- linux-2.6.orig/fs/nfs/write.c
+++ linux-2.6/fs/nfs/write.c
@@ -28,9 +28,6 @@
#define NFSDBG_FACILITY NFSDBG_PAGECACHE
-#define MIN_POOL_WRITE (32)
-#define MIN_POOL_COMMIT (4)
-
/*
* Local function declarations
*/
@@ -44,12 +41,10 @@ static const struct rpc_call_ops nfs_wri
static const struct rpc_call_ops nfs_commit_ops;
static struct kmem_cache *nfs_wdata_cachep;
-static mempool_t *nfs_wdata_mempool;
-static mempool_t *nfs_commit_mempool;
struct nfs_write_data *nfs_commit_alloc(void)
{
- struct nfs_write_data *p = mempool_alloc(nfs_commit_mempool, GFP_NOFS);
+ struct nfs_write_data *p = kmem_cache_alloc(nfs_wdata_cachep, GFP_NOFS);
if (p) {
memset(p, 0, sizeof(*p));
@@ -63,7 +58,7 @@ static void nfs_commit_rcu_free(struct r
struct nfs_write_data *p = container_of(head, struct nfs_write_data, task.u.tk_rcu);
if (p && (p->pagevec != &p->page_array[0]))
kfree(p->pagevec);
- mempool_free(p, nfs_commit_mempool);
+ kmem_cache_free(nfs_wdata_cachep, p);
}
void nfs_commit_free(struct nfs_write_data *wdata)
@@ -73,7 +68,7 @@ void nfs_commit_free(struct nfs_write_da
struct nfs_write_data *nfs_writedata_alloc(unsigned int pagecount)
{
- struct nfs_write_data *p = mempool_alloc(nfs_wdata_mempool, GFP_NOFS);
+ struct nfs_write_data *p = kmem_cache_alloc(nfs_wdata_cachep, GFP_NOFS);
if (p) {
memset(p, 0, sizeof(*p));
@@ -84,7 +79,7 @@ struct nfs_write_data *nfs_writedata_all
else {
p->pagevec = kcalloc(pagecount, sizeof(struct page *), GFP_NOFS);
if (!p->pagevec) {
- mempool_free(p, nfs_wdata_mempool);
+ kmem_cache_free(nfs_wdata_cachep, p);
p = NULL;
}
}
@@ -97,7 +92,7 @@ static void nfs_writedata_rcu_free(struc
struct nfs_write_data *p = container_of(head, struct nfs_write_data, task.u.tk_rcu);
if (p && (p->pagevec != &p->page_array[0]))
kfree(p->pagevec);
- mempool_free(p, nfs_wdata_mempool);
+ kmem_cache_free(nfs_wdata_cachep, p);
}
static void nfs_writedata_free(struct nfs_write_data *wdata)
@@ -1514,16 +1509,6 @@ int __init nfs_init_writepagecache(void)
if (nfs_wdata_cachep == NULL)
return -ENOMEM;
- nfs_wdata_mempool = mempool_create_slab_pool(MIN_POOL_WRITE,
- nfs_wdata_cachep);
- if (nfs_wdata_mempool == NULL)
- return -ENOMEM;
-
- nfs_commit_mempool = mempool_create_slab_pool(MIN_POOL_COMMIT,
- nfs_wdata_cachep);
- if (nfs_commit_mempool == NULL)
- return -ENOMEM;
-
/*
* NFS congestion size, scale with available memory.
*
@@ -1549,8 +1534,6 @@ int __init nfs_init_writepagecache(void)
void nfs_destroy_writepagecache(void)
{
- mempool_destroy(nfs_commit_mempool);
- mempool_destroy(nfs_wdata_mempool);
kmem_cache_destroy(nfs_wdata_cachep);
}
--
^ permalink raw reply
* [PATCH 05/28] mm: allow PF_MEMALLOC from softirq context
From: Peter Zijlstra @ 2008-02-20 14:46 UTC (permalink / raw)
To: Linus Torvalds, Andrew Morton, linux-kernel, linux-mm, netdev,
trond.myklebust
Cc: Peter Zijlstra
In-Reply-To: <20080220144610.548202000@chello.nl>
[-- Attachment #1: mm-PF_MEMALLOC-softirq.patch --]
[-- Type: text/plain, Size: 2287 bytes --]
Allow PF_MEMALLOC to be set in softirq context. When running softirqs from
a borrowed context save current->flags, ksoftirqd will have its own
task_struct.
This is needed to allow network softirq packet processing to make use of
PF_MEMALLOC.
Signed-off-by: Peter Zijlstra <a.p.zijlstra@chello.nl>
---
include/linux/sched.h | 4 ++++
kernel/softirq.c | 3 +++
mm/page_alloc.c | 7 ++++---
3 files changed, 11 insertions(+), 3 deletions(-)
Index: linux-2.6/mm/page_alloc.c
===================================================================
--- linux-2.6.orig/mm/page_alloc.c
+++ linux-2.6/mm/page_alloc.c
@@ -1471,9 +1471,10 @@ int gfp_to_alloc_flags(gfp_t gfp_mask)
alloc_flags |= ALLOC_HARDER;
if (likely(!(gfp_mask & __GFP_NOMEMALLOC))) {
- if (!in_interrupt() &&
- ((p->flags & PF_MEMALLOC) ||
- unlikely(test_thread_flag(TIF_MEMDIE))))
+ if (!in_irq() && (p->flags & PF_MEMALLOC))
+ alloc_flags |= ALLOC_NO_WATERMARKS;
+ else if (!in_interrupt() &&
+ unlikely(test_thread_flag(TIF_MEMDIE)))
alloc_flags |= ALLOC_NO_WATERMARKS;
}
Index: linux-2.6/kernel/softirq.c
===================================================================
--- linux-2.6.orig/kernel/softirq.c
+++ linux-2.6/kernel/softirq.c
@@ -213,6 +213,8 @@ asmlinkage void __do_softirq(void)
__u32 pending;
int max_restart = MAX_SOFTIRQ_RESTART;
int cpu;
+ unsigned long pflags = current->flags;
+ current->flags &= ~PF_MEMALLOC;
pending = local_softirq_pending();
account_system_vtime(current);
@@ -251,6 +253,7 @@ restart:
account_system_vtime(current);
_local_bh_enable();
+ tsk_restore_flags(current, pflags, PF_MEMALLOC);
}
#ifndef __ARCH_HAS_DO_SOFTIRQ
Index: linux-2.6/include/linux/sched.h
===================================================================
--- linux-2.6.orig/include/linux/sched.h
+++ linux-2.6/include/linux/sched.h
@@ -1497,6 +1497,10 @@ static inline void put_task_struct(struc
#define tsk_used_math(p) ((p)->flags & PF_USED_MATH)
#define used_math() tsk_used_math(current)
+#define tsk_restore_flags(p, pflags, mask) \
+ do { (p)->flags &= ~(mask); \
+ (p)->flags |= ((pflags) & (mask)); } while (0)
+
#ifdef CONFIG_SMP
extern int set_cpus_allowed(struct task_struct *p, cpumask_t new_mask);
#else
--
^ permalink raw reply
* Re: [PATCH 00/04] smc91x: request bus width using platform data
From: Nicolas Pitre @ 2008-02-20 15:36 UTC (permalink / raw)
To: Magnus Damm; +Cc: netdev, lethal, akpm
In-Reply-To: <20080220095740.22645.36783.sendpatchset@clockwork.opensource.se>
On Wed, 20 Feb 2008, Magnus Damm wrote:
> These patches make it possible to request bus width in the platform data.
>
> Instead of keep on updating smc91x.h with board specific configuration,
> use platform data to pass along bus width and irq flags to the driver.
> This change is designed to be backwards-compatible, so all boards configured
> in the header file should just work as usual.
Thank you for doing this work. I really meant to do it, and commit
09779c6df2dbe95483269d194b327d41fe2cc57e was the first step towards that
goal, but as you can see we're nearly two years later and I didn't do
it.
I have a few comments though. I will reply to those messages
separately.
Nicolas
^ permalink raw reply
* Re: [PATCH] SUNRPC: Mark buffer used for debug printks with __maybe_unused
From: Pavel Emelyanov @ 2008-02-20 15:36 UTC (permalink / raw)
To: Joe Perches; +Cc: J. Bruce Fields, Linux Netdev List
In-Reply-To: <1203521357.7181.163.camel@localhost>
Joe Perches wrote:
> On Wed, 2008-02-20 at 17:02 +0300, Pavel Emelyanov wrote:
>> There are tree places, which declare the char buf[...] on the stack
>> to push it later into dprintk(). Since the dprintk sometimes (if the
>> CONFIG_SYSCTL=n) becomes an empty do { } while (0) stub, these buffers
>> cause gcc to produce appropriate warnings.
>
> What about the uses in fs?
>
> fs/lockd/svc.c: char buf[RPC_MAX_ADDRBUFLEN];
> fs/lockd/svc4proc.c: char buf[RPC_MAX_ADDRBUFLEN];
> fs/lockd/svcproc.c: char buf[RPC_MAX_ADDRBUFLEN];
> fs/nfs/callback.c: char buf[RPC_MAX_ADDRBUFLEN];
> fs/nfsd/nfsfh.c: char buf[RPC_MAX_ADDRBUFLEN];
> fs/nfsd/nfsproc.c: char buf[RPC_MAX_ADDRBUFLEN];
>
> Perhaps there should be a DECLARE_RPC_BUF(buf) macro?
>
> #define DECLARE_RPC_BUF(var) char var[MAC_BUF_SIZE] __maybe_unused
Sigh... Why is that better than a strait declaration with attribute?
>
>
^ permalink raw reply
* Re: [PATCH] SUNRPC: Mark buffer used for debug printks with __maybe_unused
From: Patrick McHardy @ 2008-02-20 15:35 UTC (permalink / raw)
To: Joe Perches; +Cc: Pavel Emelyanov, J. Bruce Fields, Linux Netdev List
In-Reply-To: <1203521514.7181.167.camel@localhost>
Joe Perches wrote:
> On Wed, 2008-02-20 at 07:29 -0800, Joe Perches wrote:
>
>> fs/nfsd/nfsproc.c: char buf[RPC_MAX_ADDRBUFLEN];
>> Perhaps there should be a DECLARE_RPC_BUF(buf) macro?
>> #define DECLARE_RPC_BUF(var) char var[MAC_BUF_SIZE] __maybe_unused
>>
>
> Make that:
>
> #define DECLARE_RPC_BUF(var) char var[RPC_MAX_ADDRBUFLEN] __maybe_unuse
Alternatively change the dprintk macro to behave similar like
pr_debug() and mark things like svc_print_addr() __pure, which
has the advantage that is still performs format checking even
if debugging is disabled.
^ permalink raw reply
* Re: [PATCH] SUNRPC: Mark buffer used for debug printks with __maybe_unused
From: Joe Perches @ 2008-02-20 15:31 UTC (permalink / raw)
To: Pavel Emelyanov; +Cc: J. Bruce Fields, Linux Netdev List
In-Reply-To: <1203521357.7181.163.camel@localhost>
On Wed, 2008-02-20 at 07:29 -0800, Joe Perches wrote:
> fs/nfsd/nfsproc.c: char buf[RPC_MAX_ADDRBUFLEN];
> Perhaps there should be a DECLARE_RPC_BUF(buf) macro?
> #define DECLARE_RPC_BUF(var) char var[MAC_BUF_SIZE] __maybe_unused
Make that:
#define DECLARE_RPC_BUF(var) char var[RPC_MAX_ADDRBUFLEN] __maybe_unused
^ permalink raw reply
* Re: [PATCH] SUNRPC: Mark buffer used for debug printks with __maybe_unused
From: Joe Perches @ 2008-02-20 15:29 UTC (permalink / raw)
To: Pavel Emelyanov; +Cc: J. Bruce Fields, Linux Netdev List
In-Reply-To: <47BC3303.2040101@openvz.org>
On Wed, 2008-02-20 at 17:02 +0300, Pavel Emelyanov wrote:
> There are tree places, which declare the char buf[...] on the stack
> to push it later into dprintk(). Since the dprintk sometimes (if the
> CONFIG_SYSCTL=n) becomes an empty do { } while (0) stub, these buffers
> cause gcc to produce appropriate warnings.
What about the uses in fs?
fs/lockd/svc.c: char buf[RPC_MAX_ADDRBUFLEN];
fs/lockd/svc4proc.c: char buf[RPC_MAX_ADDRBUFLEN];
fs/lockd/svcproc.c: char buf[RPC_MAX_ADDRBUFLEN];
fs/nfs/callback.c: char buf[RPC_MAX_ADDRBUFLEN];
fs/nfsd/nfsfh.c: char buf[RPC_MAX_ADDRBUFLEN];
fs/nfsd/nfsproc.c: char buf[RPC_MAX_ADDRBUFLEN];
Perhaps there should be a DECLARE_RPC_BUF(buf) macro?
#define DECLARE_RPC_BUF(var) char var[MAC_BUF_SIZE] __maybe_unused
^ permalink raw reply
* [PATCH 22/28] mm: add support for non block device backed swap files
From: Peter Zijlstra @ 2008-02-20 14:46 UTC (permalink / raw)
To: Linus Torvalds, Andrew Morton, linux-kernel, linux-mm, netdev,
trond.myklebust
Cc: Peter Zijlstra
In-Reply-To: <20080220144610.548202000@chello.nl>
[-- Attachment #1: mm-swapfile.patch --]
[-- Type: text/plain, Size: 11996 bytes --]
New addres_space_operations methods are added:
int swapfile(struct address_space *, int);
int swap_out(struct file *, struct page *, struct writeback_control *);
int swap_in(struct file *, struct page *);
When during sys_swapon() the swapfile() method is found and returns no error
the swapper_space.a_ops will proxy to sis->swap_file->f_mapping->a_ops, and
make use of swap_{out,in}() to write/read swapcache pages.
The swapfile method will be used to communicate to the address_space that the
VM relies on it, and the address_space should take adequate measures (like
reserving memory for mempools or the like).
This new interface can be used to obviate the need for ->bmap in the swapfile
code. A filesystem would need to load (and maybe even allocate) the full block
map for a file into memory and pin it there on ->swapfile(,1) so that
->swap_{out,in}() have instant access to it. It can be released on
->swapfile(,0).
The reason to provide ->swap_{out,in}() over using {write,read}page() is to
1) make a distinction between swapcache and pagecache pages, and
2) to provide a struct file * for credential context (normally not needed
in the context of writepage, as the page content is normally dirtied
using either of the following interfaces:
write_{begin,end}()
{prepare,commit}_write()
page_mkwrite()
which do have the file context.
Signed-off-by: Peter Zijlstra <a.p.zijlstra@chello.nl>
---
Documentation/filesystems/Locking | 19 +++++++++++++
Documentation/filesystems/vfs.txt | 17 ++++++++++++
include/linux/buffer_head.h | 2 -
include/linux/fs.h | 8 +++++
include/linux/swap.h | 4 ++
mm/page_io.c | 52 ++++++++++++++++++++++++++++++++++++++
mm/swap_state.c | 4 +-
mm/swapfile.c | 26 ++++++++++++++++++-
8 files changed, 128 insertions(+), 4 deletions(-)
Index: linux-2.6/include/linux/swap.h
===================================================================
--- linux-2.6.orig/include/linux/swap.h
+++ linux-2.6/include/linux/swap.h
@@ -120,6 +120,7 @@ enum {
SWP_USED = (1 << 0), /* is slot in swap_info[] used? */
SWP_WRITEOK = (1 << 1), /* ok to write to this swap? */
SWP_ACTIVE = (SWP_USED | SWP_WRITEOK),
+ SWP_FILE = (1 << 2), /* file swap area */
/* add others here before... */
SWP_SCANNING = (1 << 8), /* refcount in scan_swap_map */
};
@@ -217,6 +218,8 @@ extern void swap_unplug_io_fn(struct bac
/* linux/mm/page_io.c */
extern int swap_readpage(struct file *, struct page *);
extern int swap_writepage(struct page *page, struct writeback_control *wbc);
+extern void swap_sync_page(struct page *page);
+extern int swap_set_page_dirty(struct page *page);
extern void end_swap_bio_read(struct bio *bio, int err);
/* linux/mm/swap_state.c */
@@ -250,6 +253,7 @@ extern unsigned int count_swap_pages(int
extern sector_t map_swap_page(struct swap_info_struct *, pgoff_t);
extern sector_t swapdev_block(int, pgoff_t);
extern struct swap_info_struct *get_swap_info_struct(unsigned);
+extern struct swap_info_struct *page_swap_info(struct page *);
extern int can_share_swap_page(struct page *);
extern int remove_exclusive_swap_page(struct page *);
struct backing_dev_info;
Index: linux-2.6/mm/page_io.c
===================================================================
--- linux-2.6.orig/mm/page_io.c
+++ linux-2.6/mm/page_io.c
@@ -17,6 +17,7 @@
#include <linux/bio.h>
#include <linux/swapops.h>
#include <linux/writeback.h>
+#include <linux/buffer_head.h>
#include <asm/pgtable.h>
static struct bio *get_swap_bio(gfp_t gfp_flags, pgoff_t index,
@@ -97,11 +98,21 @@ int swap_writepage(struct page *page, st
{
struct bio *bio;
int ret = 0, rw = WRITE;
+ struct swap_info_struct *sis = page_swap_info(page);
if (remove_exclusive_swap_page(page)) {
unlock_page(page);
goto out;
}
+
+ if (sis->flags & SWP_FILE) {
+ ret = sis->swap_file->f_mapping->
+ a_ops->swap_out(sis->swap_file, page, wbc);
+ if (!ret)
+ count_vm_event(PSWPOUT);
+ return ret;
+ }
+
bio = get_swap_bio(GFP_NOIO, page_private(page), page,
end_swap_bio_write);
if (bio == NULL) {
@@ -120,13 +131,54 @@ out:
return ret;
}
+void swap_sync_page(struct page *page)
+{
+ struct swap_info_struct *sis = page_swap_info(page);
+
+ if (sis->flags & SWP_FILE) {
+ const struct address_space_operations *a_ops =
+ sis->swap_file->f_mapping->a_ops;
+ if (a_ops->sync_page)
+ a_ops->sync_page(page);
+ } else
+ block_sync_page(page);
+}
+
+int swap_set_page_dirty(struct page *page)
+{
+ struct swap_info_struct *sis = page_swap_info(page);
+
+ if (sis->flags & SWP_FILE) {
+ const struct address_space_operations *a_ops =
+ sis->swap_file->f_mapping->a_ops;
+ int (*spd)(struct page *) = a_ops->set_page_dirty;
+#ifdef CONFIG_BLOCK
+ if (!spd)
+ spd = __set_page_dirty_buffers;
+#endif
+ return (*spd)(page);
+ }
+
+ return __set_page_dirty_nobuffers(page);
+}
+
int swap_readpage(struct file *file, struct page *page)
{
struct bio *bio;
int ret = 0;
+ struct swap_info_struct *sis = page_swap_info(page);
BUG_ON(!PageLocked(page));
BUG_ON(PageUptodate(page));
+
+ if (sis->flags & SWP_FILE) {
+ ret = sis->swap_file->f_mapping->
+ a_ops->swap_in(sis->swap_file, page);
+ if (!ret)
+ count_vm_event(PSWPIN);
+ return ret;
+ }
+
bio = get_swap_bio(GFP_KERNEL, page_private(page), page,
end_swap_bio_read);
if (bio == NULL) {
Index: linux-2.6/mm/swap_state.c
===================================================================
--- linux-2.6.orig/mm/swap_state.c
+++ linux-2.6/mm/swap_state.c
@@ -27,8 +27,8 @@
*/
static const struct address_space_operations swap_aops = {
.writepage = swap_writepage,
- .sync_page = block_sync_page,
- .set_page_dirty = __set_page_dirty_nobuffers,
+ .sync_page = swap_sync_page,
+ .set_page_dirty = swap_set_page_dirty,
.migratepage = migrate_page,
};
Index: linux-2.6/mm/swapfile.c
===================================================================
--- linux-2.6.orig/mm/swapfile.c
+++ linux-2.6/mm/swapfile.c
@@ -1012,6 +1012,12 @@ static void destroy_swap_extents(struct
list_del(&se->list);
kfree(se);
}
+
+ if (sis->flags & SWP_FILE) {
+ sis->flags &= ~SWP_FILE;
+ sis->swap_file->f_mapping->a_ops->
+ swapfile(sis->swap_file->f_mapping, 0);
+ }
}
/*
@@ -1104,6 +1110,17 @@ static int setup_swap_extents(struct swa
goto done;
}
+ if (sis->swap_file->f_mapping->a_ops->swapfile) {
+ ret = sis->swap_file->f_mapping->a_ops->
+ swapfile(sis->swap_file->f_mapping, 1);
+ if (!ret) {
+ sis->flags |= SWP_FILE;
+ ret = add_swap_extent(sis, 0, sis->max, 0);
+ *span = sis->pages;
+ }
+ goto done;
+ }
+
blkbits = inode->i_blkbits;
blocks_per_page = PAGE_SIZE >> blkbits;
@@ -1668,7 +1685,7 @@ asmlinkage long sys_swapon(const char __
mutex_lock(&swapon_mutex);
spin_lock(&swap_lock);
- p->flags = SWP_ACTIVE;
+ p->flags |= SWP_WRITEOK;
nr_swap_pages += nr_good_pages;
total_swap_pages += nr_good_pages;
@@ -1793,6 +1810,13 @@ get_swap_info_struct(unsigned type)
return &swap_info[type];
}
+struct swap_info_struct *page_swap_info(struct page *page)
+{
+ swp_entry_t swap = { .val = page_private(page) };
+ BUG_ON(!PageSwapCache(page));
+ return &swap_info[swp_type(swap)];
+}
+
/*
* swap_lock prevents swap_map being freed. Don't grab an extra
* reference on the swaphandle, it doesn't matter if it becomes unused.
Index: linux-2.6/include/linux/fs.h
===================================================================
--- linux-2.6.orig/include/linux/fs.h
+++ linux-2.6/include/linux/fs.h
@@ -481,6 +481,14 @@ struct address_space_operations {
int (*migratepage) (struct address_space *,
struct page *, struct page *);
int (*launder_page) (struct page *);
+
+ /*
+ * swapfile support
+ */
+ int (*swapfile)(struct address_space *, int);
+ int (*swap_out)(struct file *file, struct page *page,
+ struct writeback_control *wbc);
+ int (*swap_in)(struct file *file, struct page *page);
};
/*
Index: linux-2.6/Documentation/filesystems/Locking
===================================================================
--- linux-2.6.orig/Documentation/filesystems/Locking
+++ linux-2.6/Documentation/filesystems/Locking
@@ -171,6 +171,9 @@ prototypes:
int (*direct_IO)(int, struct kiocb *, const struct iovec *iov,
loff_t offset, unsigned long nr_segs);
int (*launder_page) (struct page *);
+ int (*swapfile) (struct address_space *, int);
+ int (*swap_out) (struct file *, struct page *, struct writeback_control *);
+ int (*swap_in) (struct file *, struct page *);
locking rules:
All except set_page_dirty may block
@@ -192,6 +195,9 @@ invalidatepage: no yes
releasepage: no yes
direct_IO: no
launder_page: no yes
+swapfile no
+swap_out no yes, unlocks
+swap_in no yes, unlocks
->prepare_write(), ->commit_write(), ->sync_page() and ->readpage()
may be called from the request handler (/dev/loop).
@@ -291,6 +297,19 @@ cleaned, or an error value if not. Note
getting mapped back in and redirtied, it needs to be kept locked
across the entire operation.
+ ->swapfile() will be called with a non zero argument on address spaces
+backing non block device backed swapfiles. A return value of zero indicates
+success. In which case this address space can be used for backing swapspace.
+The swapspace operations will be proxied to the address space operations.
+Swapoff will call this method with a zero argument to release the address
+space.
+
+ ->swap_out() when swapfile() returned success, this method is used to
+write the swap page.
+
+ ->swap_in() when swapfile() returned success, this method is used to
+read the swap page.
+
Note: currently almost all instances of address_space methods are
using BKL for internal serialization and that's one of the worst sources
of contention. Normally they are calling library functions (in fs/buffer.c)
Index: linux-2.6/include/linux/buffer_head.h
===================================================================
--- linux-2.6.orig/include/linux/buffer_head.h
+++ linux-2.6/include/linux/buffer_head.h
@@ -339,7 +339,7 @@ static inline void invalidate_inode_buff
static inline int remove_inode_buffers(struct inode *inode) { return 1; }
static inline int sync_mapping_buffers(struct address_space *mapping) { return 0; }
static inline void invalidate_bdev(struct block_device *bdev) {}
-
+static inline void block_sync_page(struct page *) { }
#endif /* CONFIG_BLOCK */
#endif /* _LINUX_BUFFER_HEAD_H */
Index: linux-2.6/Documentation/filesystems/vfs.txt
===================================================================
--- linux-2.6.orig/Documentation/filesystems/vfs.txt
+++ linux-2.6/Documentation/filesystems/vfs.txt
@@ -543,6 +543,10 @@ struct address_space_operations {
/* migrate the contents of a page to the specified target */
int (*migratepage) (struct page *, struct page *);
int (*launder_page) (struct page *);
+ int (*swapfile)(struct address_space *, int);
+ int (*swap_out)(struct file *file, struct page *page,
+ struct writeback_control *wbc);
+ int (*swap_in)(struct file *file, struct page *page);
};
writepage: called by the VM to write a dirty page to backing store.
@@ -728,6 +732,19 @@ struct address_space_operations {
prevent redirtying the page, it is kept locked during the whole
operation.
+ swapfile: Called with a non-zero argument when swapon is used on a file. A
+ return value of zero indicates success. In which case this
+ address_space can be used to back swapspace. The swapspace operations
+ will be proxied to this address space's ->swap_{out,in} methods.
+ Swapoff will call this method with a zero argument to release the
+ address space.
+
+ swap_out: Called to write a swapcache page to a backing store, similar to
+ writepage.
+
+ swap_in: Called to read a swapcache page from a backing store, similar to
+ readpage.
+
The File Object
===============
--
^ permalink raw reply
* [PATCH 28/28] nfs: fix various memory recursions possible with swap over NFS.
From: Peter Zijlstra @ 2008-02-20 14:46 UTC (permalink / raw)
To: Linus Torvalds, Andrew Morton, linux-kernel, linux-mm, netdev,
trond.myklebust
Cc: Peter Zijlstra
In-Reply-To: <20080220144610.548202000@chello.nl>
[-- Attachment #1: nfs-alloc-recursions.patch --]
[-- Type: text/plain, Size: 1861 bytes --]
GFP_NOFS is not enough, since swap traffic is IO, hence fall back to GFP_NOIO.
Signed-off-by: Peter Zijlstra <a.p.zijlstra@chello.nl>
---
fs/nfs/pagelist.c | 2 +-
fs/nfs/write.c | 6 +++---
2 files changed, 4 insertions(+), 4 deletions(-)
Index: linux-2.6/fs/nfs/write.c
===================================================================
--- linux-2.6.orig/fs/nfs/write.c
+++ linux-2.6/fs/nfs/write.c
@@ -44,7 +44,7 @@ static struct kmem_cache *nfs_wdata_cach
struct nfs_write_data *nfs_commit_alloc(void)
{
- struct nfs_write_data *p = kmem_cache_alloc(nfs_wdata_cachep, GFP_NOFS);
+ struct nfs_write_data *p = kmem_cache_alloc(nfs_wdata_cachep, GFP_NOIO);
if (p) {
memset(p, 0, sizeof(*p));
@@ -68,7 +68,7 @@ void nfs_commit_free(struct nfs_write_da
struct nfs_write_data *nfs_writedata_alloc(unsigned int pagecount)
{
- struct nfs_write_data *p = kmem_cache_alloc(nfs_wdata_cachep, GFP_NOFS);
+ struct nfs_write_data *p = kmem_cache_alloc(nfs_wdata_cachep, GFP_NOIO);
if (p) {
memset(p, 0, sizeof(*p));
@@ -77,7 +77,7 @@ struct nfs_write_data *nfs_writedata_all
if (pagecount <= ARRAY_SIZE(p->page_array))
p->pagevec = p->page_array;
else {
- p->pagevec = kcalloc(pagecount, sizeof(struct page *), GFP_NOFS);
+ p->pagevec = kcalloc(pagecount, sizeof(struct page *), GFP_NOIO);
if (!p->pagevec) {
kmem_cache_free(nfs_wdata_cachep, p);
p = NULL;
Index: linux-2.6/fs/nfs/pagelist.c
===================================================================
--- linux-2.6.orig/fs/nfs/pagelist.c
+++ linux-2.6/fs/nfs/pagelist.c
@@ -27,7 +27,7 @@ static inline struct nfs_page *
nfs_page_alloc(void)
{
struct nfs_page *p;
- p = kmem_cache_alloc(nfs_page_cachep, GFP_KERNEL);
+ p = kmem_cache_alloc(nfs_page_cachep, GFP_NOIO);
if (p) {
memset(p, 0, sizeof(*p));
INIT_LIST_HEAD(&p->wb_list);
--
^ permalink raw reply
* [PATCH 09/28] mm: __GFP_MEMALLOC
From: Peter Zijlstra @ 2008-02-20 14:46 UTC (permalink / raw)
To: Linus Torvalds, Andrew Morton, linux-kernel, linux-mm, netdev,
trond.myklebust
Cc: Peter Zijlstra
In-Reply-To: <20080220144610.548202000@chello.nl>
[-- Attachment #1: mm-page_alloc-GFP_EMERGENCY.patch --]
[-- Type: text/plain, Size: 2048 bytes --]
__GFP_MEMALLOC will allow the allocation to disregard the watermarks,
much like PF_MEMALLOC.
Signed-off-by: Peter Zijlstra <a.p.zijlstra@chello.nl>
---
include/linux/gfp.h | 3 ++-
mm/page_alloc.c | 4 +++-
2 files changed, 5 insertions(+), 2 deletions(-)
Index: linux-2.6/include/linux/gfp.h
===================================================================
--- linux-2.6.orig/include/linux/gfp.h
+++ linux-2.6/include/linux/gfp.h
@@ -43,6 +43,7 @@ struct vm_area_struct;
#define __GFP_REPEAT ((__force gfp_t)0x400u) /* Retry the allocation. Might fail */
#define __GFP_NOFAIL ((__force gfp_t)0x800u) /* Retry for ever. Cannot fail */
#define __GFP_NORETRY ((__force gfp_t)0x1000u)/* Do not retry. Might fail */
+#define __GFP_MEMALLOC ((__force gfp_t)0x2000u)/* Use emergency reserves */
#define __GFP_COMP ((__force gfp_t)0x4000u)/* Add compound page metadata */
#define __GFP_ZERO ((__force gfp_t)0x8000u)/* Return zeroed page on success */
#define __GFP_NOMEMALLOC ((__force gfp_t)0x10000u) /* Don't use emergency reserves */
@@ -88,7 +89,7 @@ struct vm_area_struct;
/* Control page allocator reclaim behavior */
#define GFP_RECLAIM_MASK (__GFP_WAIT|__GFP_HIGH|__GFP_IO|__GFP_FS|\
__GFP_NOWARN|__GFP_REPEAT|__GFP_NOFAIL|\
- __GFP_NORETRY|__GFP_NOMEMALLOC)
+ __GFP_NORETRY|__GFP_MEMALLOC|__GFP_NOMEMALLOC)
/* Control allocation constraints */
#define GFP_CONSTRAINT_MASK (__GFP_HARDWALL|__GFP_THISNODE)
Index: linux-2.6/mm/page_alloc.c
===================================================================
--- linux-2.6.orig/mm/page_alloc.c
+++ linux-2.6/mm/page_alloc.c
@@ -1474,7 +1474,9 @@ int gfp_to_alloc_flags(gfp_t gfp_mask)
alloc_flags |= ALLOC_HARDER;
if (likely(!(gfp_mask & __GFP_NOMEMALLOC))) {
- if (!in_irq() && (p->flags & PF_MEMALLOC))
+ if (gfp_mask & __GFP_MEMALLOC)
+ alloc_flags |= ALLOC_NO_WATERMARKS;
+ else if (!in_irq() && (p->flags & PF_MEMALLOC))
alloc_flags |= ALLOC_NO_WATERMARKS;
else if (!in_interrupt() &&
unlikely(test_thread_flag(TIF_MEMDIE)))
--
^ permalink raw reply
* [PATCH 18/28] netvm: filter emergency skbs.
From: Peter Zijlstra @ 2008-02-20 14:46 UTC (permalink / raw)
To: Linus Torvalds, Andrew Morton, linux-kernel, linux-mm, netdev,
trond.myklebust
Cc: Peter Zijlstra
In-Reply-To: <20080220144610.548202000@chello.nl>
[-- Attachment #1: netvm-sk_filter.patch --]
[-- Type: text/plain, Size: 765 bytes --]
Toss all emergency packets not for a SOCK_MEMALLOC socket. This ensures our
precious memory reserve doesn't get stuck waiting for user-space.
The correctness of this approach relies on the fact that networks must be
assumed lossy.
Signed-off-by: Peter Zijlstra <a.p.zijlstra@chello.nl>
---
include/net/sock.h | 3 +++
1 file changed, 3 insertions(+)
Index: linux-2.6/include/net/sock.h
===================================================================
--- linux-2.6.orig/include/net/sock.h
+++ linux-2.6/include/net/sock.h
@@ -998,6 +998,9 @@ static inline int sk_filter(struct sock
{
int err;
struct sk_filter *filter;
+
+ if (skb_emergency(skb) && !sk_has_memalloc(sk))
+ return -ENOMEM;
err = security_sock_rcv_skb(sk, skb);
if (err)
--
^ permalink raw reply
* [PATCH 16/28] netvm: INET reserves.
From: Peter Zijlstra @ 2008-02-20 14:46 UTC (permalink / raw)
To: Linus Torvalds, Andrew Morton, linux-kernel, linux-mm, netdev,
trond.myklebust
Cc: Peter Zijlstra
In-Reply-To: <20080220144610.548202000@chello.nl>
[-- Attachment #1: netvm-reserve-inet.patch --]
[-- Type: text/plain, Size: 12009 bytes --]
Add reserves for INET.
The two big users seem to be the route cache and ip-fragment cache.
Reserve the route cache under generic RX reserve, its usage is bounded by
the high reclaim watermark, and thus does not need further accounting.
Reserve the ip-fragement caches under SKB data reserve, these add to the
SKB RX limit. By ensuring we can at least receive as much data as fits in
the reassmbly line we avoid fragment attack deadlocks.
Use proc conv() routines to update these limits and return -ENOMEM to user
space.
Adds to the reserve tree:
total network reserve
network TX reserve
protocol TX pages
network RX reserve
+ IPv6 route cache
+ IPv4 route cache
SKB data reserve
+ IPv6 fragment cache
+ IPv4 fragment cache
Signed-off-by: Peter Zijlstra <a.p.zijlstra@chello.nl>
---
net/ipv4/ip_fragment.c | 65 +++++++++++++++++++++++++++++++++++++++++++++++--
net/ipv4/route.c | 65 +++++++++++++++++++++++++++++++++++++++++++++++--
net/ipv6/reassembly.c | 65 +++++++++++++++++++++++++++++++++++++++++++++++--
net/ipv6/route.c | 65 +++++++++++++++++++++++++++++++++++++++++++++++--
4 files changed, 252 insertions(+), 8 deletions(-)
Index: linux-2.6/net/ipv4/ip_fragment.c
===================================================================
--- linux-2.6.orig/net/ipv4/ip_fragment.c
+++ linux-2.6/net/ipv4/ip_fragment.c
@@ -44,6 +44,7 @@
#include <linux/udp.h>
#include <linux/inet.h>
#include <linux/netfilter_ipv4.h>
+#include <linux/reserve.h>
/* NOTE. Logic of IP defragmentation is parallel to corresponding IPv6
* code now. If you change something here, _PLEASE_ update ipv6/reassembly.c
@@ -591,17 +592,72 @@ int ip_defrag(struct sk_buff *skb, u32 u
return -ENOMEM;
}
+static struct mem_reserve ipv4_frag_reserve;
+
#ifdef CONFIG_SYSCTL
+static int ipv4_frag_bytes;
+
+static int proc_dointvec_fragment(struct ctl_table *table, int write,
+ struct file *filp, void __user *buffer, size_t *lenp,
+ loff_t *ppos)
+{
+ int old_bytes, ret;
+
+ if (!write)
+ ipv4_frag_bytes = init_net.ipv4.frags.high_thresh;
+ old_bytes = ipv4_frag_bytes;
+
+ ret = proc_dointvec(table, write, filp, buffer, lenp, ppos);
+
+ if (!ret && write) {
+ ret = mem_reserve_kmalloc_set(&ipv4_frag_reserve,
+ ipv4_frag_bytes);
+ if (!ret)
+ init_net.ipv4.frags.high_thresh = ipv4_frag_bytes;
+ else
+ ipv4_frag_bytes = old_bytes;
+ }
+
+ return ret;
+}
+
+static int sysctl_intvec_fragment(struct ctl_table *table,
+ int __user *name, int nlen,
+ void __user *oldval, size_t __user *oldlenp,
+ void __user *newval, size_t newlen)
+{
+ int old_bytes, ret;
+ int write = (newval && newlen);
+
+ if (!write)
+ ipv4_frag_bytes = init_net.ipv4.frags.high_thresh;
+ old_bytes = ipv4_frag_bytes;
+
+ ret = sysctl_intvec(table, name, nlen, oldval, oldlenp, newval, newlen);
+
+ if (!ret && write) {
+ ret = mem_reserve_kmalloc_set(&ipv4_frag_reserve,
+ ipv4_frag_bytes);
+ if (!ret)
+ init_net.ipv4.frags.high_thresh = ipv4_frag_bytes;
+ else
+ ipv4_frag_bytes = old_bytes;
+ }
+
+ return ret;
+}
+
static int zero;
static struct ctl_table ip4_frags_ctl_table[] = {
{
.ctl_name = NET_IPV4_IPFRAG_HIGH_THRESH,
.procname = "ipfrag_high_thresh",
- .data = &init_net.ipv4.frags.high_thresh,
+ .data = &ipv4_frag_bytes,
.maxlen = sizeof(int),
.mode = 0644,
- .proc_handler = &proc_dointvec
+ .proc_handler = &proc_dointvec_fragment,
+ .strategy = &sysctl_intvec_fragment,
},
{
.ctl_name = NET_IPV4_IPFRAG_LOW_THRESH,
@@ -736,6 +792,11 @@ void __init ipfrag_init(void)
ip4_frags.frag_expire = ip_expire;
ip4_frags.secret_interval = 10 * 60 * HZ;
inet_frags_init(&ip4_frags);
+
+ mem_reserve_init(&ipv4_frag_reserve, "IPv4 fragment cache",
+ &net_skb_reserve);
+ mem_reserve_kmalloc_set(&ipv4_frag_reserve,
+ init_net.ipv4.frags.high_thresh);
}
EXPORT_SYMBOL(ip_defrag);
Index: linux-2.6/net/ipv6/reassembly.c
===================================================================
--- linux-2.6.orig/net/ipv6/reassembly.c
+++ linux-2.6/net/ipv6/reassembly.c
@@ -43,6 +43,7 @@
#include <linux/random.h>
#include <linux/jhash.h>
#include <linux/skbuff.h>
+#include <linux/reserve.h>
#include <net/sock.h>
#include <net/snmp.h>
@@ -628,15 +629,70 @@ static struct inet6_protocol frag_protoc
.flags = INET6_PROTO_NOPOLICY,
};
+static struct mem_reserve ipv6_frag_reserve;
+
#ifdef CONFIG_SYSCTL
+static int ipv6_frag_bytes;
+
+static int proc_dointvec_fragment(struct ctl_table *table, int write,
+ struct file *filp, void __user *buffer, size_t *lenp,
+ loff_t *ppos)
+{
+ int old_bytes, ret;
+
+ if (!write)
+ ipv6_frag_bytes = init_net.ipv6.frags.high_thresh;
+ old_bytes = ipv6_frag_bytes;
+
+ ret = proc_dointvec(table, write, filp, buffer, lenp, ppos);
+
+ if (!ret && write) {
+ ret = mem_reserve_kmalloc_set(&ipv6_frag_reserve,
+ ipv6_frag_bytes);
+ if (!ret)
+ init_net.ipv6.frags.high_thresh = ipv6_frag_bytes;
+ else
+ ipv6_frag_bytes = old_bytes;
+ }
+
+ return ret;
+}
+
+static int sysctl_intvec_fragment(struct ctl_table *table,
+ int __user *name, int nlen,
+ void __user *oldval, size_t __user *oldlenp,
+ void __user *newval, size_t newlen)
+{
+ int old_bytes, ret;
+ int write = (newval && newlen);
+
+ if (!write)
+ ipv6_frag_bytes = init_net.ipv6.frags.high_thresh;
+ old_bytes = ipv6_frag_bytes;
+
+ ret = sysctl_intvec(table, name, nlen, oldval, oldlenp, newval, newlen);
+
+ if (!ret && write) {
+ ret = mem_reserve_kmalloc_set(&ipv6_frag_reserve,
+ ipv6_frag_bytes);
+ if (!ret)
+ init_net.ipv6.frags.high_thresh = ipv6_frag_bytes;
+ else
+ ipv6_frag_bytes = old_bytes;
+ }
+
+ return ret;
+}
+
static struct ctl_table ip6_frags_ctl_table[] = {
{
.ctl_name = NET_IPV6_IP6FRAG_HIGH_THRESH,
.procname = "ip6frag_high_thresh",
- .data = &init_net.ipv6.frags.high_thresh,
+ .data = &ipv6_frag_bytes,
.maxlen = sizeof(int),
.mode = 0644,
- .proc_handler = &proc_dointvec
+ .proc_handler = &proc_dointvec_fragment,
+ .strategy = &sysctl_intvec_fragment,
},
{
.ctl_name = NET_IPV6_IP6FRAG_LOW_THRESH,
@@ -758,6 +814,11 @@ int __init ipv6_frag_init(void)
ip6_frags.frag_expire = ip6_frag_expire;
ip6_frags.secret_interval = 10 * 60 * HZ;
inet_frags_init(&ip6_frags);
+
+ mem_reserve_init(&ipv6_frag_reserve, "IPv6 fragment cache",
+ &net_skb_reserve);
+ mem_reserve_kmalloc_set(&ipv6_frag_reserve,
+ init_net.ipv6.frags.high_thresh);
out:
return ret;
}
Index: linux-2.6/net/ipv4/route.c
===================================================================
--- linux-2.6.orig/net/ipv4/route.c
+++ linux-2.6/net/ipv4/route.c
@@ -109,6 +109,7 @@
#ifdef CONFIG_SYSCTL
#include <linux/sysctl.h>
#endif
+#include <linux/reserve.h>
#define RT_FL_TOS(oldflp) \
((u32)(oldflp->fl4_tos & (IPTOS_RT_MASK | RTO_ONLINK)))
@@ -2794,6 +2795,8 @@ void ip_rt_multicast_event(struct in_dev
rt_cache_flush(0);
}
+static struct mem_reserve ipv4_route_reserve;
+
#ifdef CONFIG_SYSCTL
static int flush_delay;
@@ -2827,6 +2830,58 @@ static int ipv4_sysctl_rtcache_flush_str
return 0;
}
+static int ipv4_route_size;
+
+static int proc_dointvec_route(struct ctl_table *table, int write,
+ struct file *filp, void __user *buffer, size_t *lenp,
+ loff_t *ppos)
+{
+ int old_size, ret;
+
+ if (!write)
+ ipv4_route_size = ip_rt_max_size;
+ old_size = ipv4_route_size;
+
+ ret = proc_dointvec(table, write, filp, buffer, lenp, ppos);
+
+ if (!ret && write) {
+ ret = mem_reserve_kmem_cache_set(&ipv4_route_reserve,
+ ipv4_dst_ops.kmem_cachep, ipv4_route_size);
+ if (!ret)
+ ip_rt_max_size = ipv4_route_size;
+ else
+ ipv4_route_size = old_size;
+ }
+
+ return ret;
+}
+
+static int sysctl_intvec_route(struct ctl_table *table,
+ int __user *name, int nlen,
+ void __user *oldval, size_t __user *oldlenp,
+ void __user *newval, size_t newlen)
+{
+ int old_size, ret;
+ int write = (newval && newlen);
+
+ if (!write)
+ ipv4_route_size = ip_rt_max_size;
+ old_size = ipv4_route_size;
+
+ ret = sysctl_intvec(table, name, nlen, oldval, oldlenp, newval, newlen);
+
+ if (!ret && write) {
+ ret = mem_reserve_kmem_cache_set(&ipv4_route_reserve,
+ ipv4_dst_ops.kmem_cachep, ipv4_route_size);
+ if (!ret)
+ ip_rt_max_size = ipv4_route_size;
+ else
+ ipv4_route_size = old_size;
+ }
+
+ return ret;
+}
+
ctl_table ipv4_route_table[] = {
{
.ctl_name = NET_IPV4_ROUTE_FLUSH,
@@ -2848,10 +2903,11 @@ ctl_table ipv4_route_table[] = {
{
.ctl_name = NET_IPV4_ROUTE_MAX_SIZE,
.procname = "max_size",
- .data = &ip_rt_max_size,
+ .data = &ipv4_route_size,
.maxlen = sizeof(int),
.mode = 0644,
- .proc_handler = &proc_dointvec,
+ .proc_handler = &proc_dointvec_route,
+ .strategy = &sysctl_intvec_route,
},
{
/* Deprecated. Use gc_min_interval_ms */
@@ -3026,6 +3082,11 @@ int __init ip_rt_init(void)
ipv4_dst_ops.gc_thresh = (rt_hash_mask + 1);
ip_rt_max_size = (rt_hash_mask + 1) * 16;
+ mem_reserve_init(&ipv4_route_reserve, "IPv4 route cache",
+ &net_rx_reserve);
+ mem_reserve_kmem_cache_set(&ipv4_route_reserve,
+ ipv4_dst_ops.kmem_cachep, ip_rt_max_size);
+
devinet_init();
ip_fib_init();
Index: linux-2.6/net/ipv6/route.c
===================================================================
--- linux-2.6.orig/net/ipv6/route.c
+++ linux-2.6/net/ipv6/route.c
@@ -38,6 +38,7 @@
#include <linux/in6.h>
#include <linux/init.h>
#include <linux/if_arp.h>
+#include <linux/reserve.h>
#include <linux/proc_fs.h>
#include <linux/seq_file.h>
#include <net/net_namespace.h>
@@ -2391,6 +2392,8 @@ static inline void ipv6_route_proc_fini(
}
#endif /* CONFIG_PROC_FS */
+static struct mem_reserve ipv6_route_reserve;
+
#ifdef CONFIG_SYSCTL
static
@@ -2406,6 +2409,58 @@ int ipv6_sysctl_rtcache_flush(ctl_table
return -EINVAL;
}
+static int ipv6_route_size;
+
+static int proc_dointvec_route(struct ctl_table *table, int write,
+ struct file *filp, void __user *buffer, size_t *lenp,
+ loff_t *ppos)
+{
+ int old_size, ret;
+
+ if (!write)
+ ipv6_route_size = ip6_rt_max_size;
+ old_size = ipv6_route_size;
+
+ ret = proc_dointvec(table, write, filp, buffer, lenp, ppos);
+
+ if (!ret && write) {
+ ret = mem_reserve_kmem_cache_set(&ipv6_route_reserve,
+ ip6_dst_ops.kmem_cachep, ipv6_route_size);
+ if (!ret)
+ ip6_rt_max_size = ipv6_route_size;
+ else
+ ipv6_route_size = old_size;
+ }
+
+ return ret;
+}
+
+static int sysctl_intvec_route(struct ctl_table *table,
+ int __user *name, int nlen,
+ void __user *oldval, size_t __user *oldlenp,
+ void __user *newval, size_t newlen)
+{
+ int old_size, ret;
+ int write = (newval && newlen);
+
+ if (!write)
+ ipv6_route_size = ip6_rt_max_size;
+ old_size = ipv6_route_size;
+
+ ret = sysctl_intvec(table, name, nlen, oldval, oldlenp, newval, newlen);
+
+ if (!ret && write) {
+ ret = mem_reserve_kmem_cache_set(&ipv6_route_reserve,
+ ip6_dst_ops.kmem_cachep, ipv6_route_size);
+ if (!ret)
+ ip6_rt_max_size = ipv6_route_size;
+ else
+ ipv6_route_size = old_size;
+ }
+
+ return ret;
+}
+
ctl_table ipv6_route_table_template[] = {
{
.procname = "flush",
@@ -2425,10 +2480,11 @@ ctl_table ipv6_route_table_template[] =
{
.ctl_name = NET_IPV6_ROUTE_MAX_SIZE,
.procname = "max_size",
- .data = &init_net.ipv6.sysctl.ip6_rt_max_size,
+ .data = &ipv6_route_size,
.maxlen = sizeof(int),
.mode = 0644,
- .proc_handler = &proc_dointvec,
+ .proc_handler = &proc_dointvec_route,
+ .strategy = &sysctl_intvec_route,
},
{
.ctl_name = NET_IPV6_ROUTE_GC_MIN_INTERVAL,
@@ -2519,6 +2575,11 @@ int __init ip6_route_init(void)
ip6_dst_blackhole_ops.kmem_cachep = ip6_dst_ops.kmem_cachep;
+ mem_reserve_init(&ipv6_route_reserve, "IPv6 route cache",
+ &net_rx_reserve);
+ mem_reserve_kmem_cache_set(&ipv6_route_reserve,
+ ip6_dst_ops.kmem_cachep, ip6_rt_max_size);
+
ret = fib6_init();
if (ret)
goto out_kmem_cache;
--
^ permalink raw reply
* [PATCH 26/28] nfs: disable data cache revalidation for swapfiles
From: Peter Zijlstra @ 2008-02-20 14:46 UTC (permalink / raw)
To: Linus Torvalds, Andrew Morton, linux-kernel, linux-mm, netdev,
trond.myklebust
Cc: Peter Zijlstra
In-Reply-To: <20080220144610.548202000@chello.nl>
[-- Attachment #1: nfs-swapper.patch --]
[-- Type: text/plain, Size: 5292 bytes --]
Do as Trond suggested:
http://lkml.org/lkml/2006/8/25/348
Disable NFS data cache revalidation on swap files since it doesn't really
make sense to have other clients change the file while you are using it.
Thereby we can stop setting PG_private on swap pages, since there ought to
be no further races with invalidate_inode_pages2() to deal with.
And since we cannot set PG_private we cannot use page->private (which is
already used by PG_swapcache pages anyway) to store the nfs_page. Thus
augment the new nfs_page_find_request logic.
Signed-off-by: Peter Zijlstra <a.p.zijlstra@chello.nl>
---
fs/nfs/inode.c | 6 ++++
fs/nfs/write.c | 73 ++++++++++++++++++++++++++++++++++++++++++++++-----------
2 files changed, 65 insertions(+), 14 deletions(-)
Index: linux-2.6/fs/nfs/inode.c
===================================================================
--- linux-2.6.orig/fs/nfs/inode.c
+++ linux-2.6/fs/nfs/inode.c
@@ -763,6 +763,12 @@ int nfs_revalidate_mapping_nolock(struct
struct nfs_inode *nfsi = NFS_I(inode);
int ret = 0;
+ /*
+ * swapfiles are not supposed to be shared.
+ */
+ if (IS_SWAPFILE(inode))
+ goto out;
+
if ((nfsi->cache_validity & NFS_INO_REVAL_PAGECACHE)
|| nfs_attribute_timeout(inode) || NFS_STALE(inode)) {
ret = __nfs_revalidate_inode(NFS_SERVER(inode), inode);
Index: linux-2.6/fs/nfs/write.c
===================================================================
--- linux-2.6.orig/fs/nfs/write.c
+++ linux-2.6/fs/nfs/write.c
@@ -112,25 +112,62 @@ static void nfs_context_set_write_error(
set_bit(NFS_CONTEXT_ERROR_WRITE, &ctx->flags);
}
-static struct nfs_page *nfs_page_find_request_locked(struct page *page)
+static struct nfs_page *
+__nfs_page_find_request_locked(struct nfs_inode *nfsi, struct page *page, int get)
{
struct nfs_page *req = NULL;
- if (PagePrivate(page)) {
+ if (PagePrivate(page))
req = (struct nfs_page *)page_private(page);
- if (req != NULL)
- kref_get(&req->wb_kref);
- }
+ else if (unlikely(PageSwapCache(page)))
+ req = radix_tree_lookup(&nfsi->nfs_page_tree, page_file_index(page));
+
+ if (get && req)
+ kref_get(&req->wb_kref);
+
return req;
}
+static inline struct nfs_page *
+nfs_page_find_request_locked(struct nfs_inode *nfsi, struct page *page)
+{
+ return __nfs_page_find_request_locked(nfsi, page, 1);
+}
+
+static int __nfs_page_has_request(struct page *page)
+{
+ struct inode *inode = page_file_mapping(page)->host;
+ struct nfs_page *req = NULL;
+
+ spin_lock(&inode->i_lock);
+ req = __nfs_page_find_request_locked(NFS_I(inode), page, 0);
+ spin_unlock(&inode->i_lock);
+
+ /*
+ * hole here plugged by the caller holding onto PG_locked
+ */
+
+ return req != NULL;
+}
+
+static inline int nfs_page_has_request(struct page *page)
+{
+ if (PagePrivate(page))
+ return 1;
+
+ if (unlikely(PageSwapCache(page)))
+ return __nfs_page_has_request(page);
+
+ return 0;
+}
+
static struct nfs_page *nfs_page_find_request(struct page *page)
{
struct inode *inode = page_file_mapping(page)->host;
struct nfs_page *req = NULL;
spin_lock(&inode->i_lock);
- req = nfs_page_find_request_locked(page);
+ req = nfs_page_find_request_locked(NFS_I(inode), page);
spin_unlock(&inode->i_lock);
return req;
}
@@ -252,7 +289,7 @@ static int nfs_page_async_flush(struct n
spin_lock(&inode->i_lock);
for(;;) {
- req = nfs_page_find_request_locked(page);
+ req = nfs_page_find_request_locked(NFS_I(inode), page);
if (req == NULL) {
spin_unlock(&inode->i_lock);
return 0;
@@ -367,8 +404,14 @@ static void nfs_inode_add_request(struct
if (nfs_have_delegation(inode, FMODE_WRITE))
nfsi->change_attr++;
}
- SetPagePrivate(req->wb_page);
- set_page_private(req->wb_page, (unsigned long)req);
+ /*
+ * Swap-space should not get truncated. Hence no need to plug the race
+ * with invalidate/truncate.
+ */
+ if (likely(!PageSwapCache(req->wb_page))) {
+ SetPagePrivate(req->wb_page);
+ set_page_private(req->wb_page, (unsigned long)req);
+ }
nfsi->npages++;
kref_get(&req->wb_kref);
radix_tree_tag_set(&nfsi->nfs_page_tree, req->wb_index,
@@ -386,8 +429,10 @@ static void nfs_inode_remove_request(str
BUG_ON (!NFS_WBACK_BUSY(req));
spin_lock(&inode->i_lock);
- set_page_private(req->wb_page, 0);
- ClearPagePrivate(req->wb_page);
+ if (likely(!PageSwapCache(req->wb_page))) {
+ set_page_private(req->wb_page, 0);
+ ClearPagePrivate(req->wb_page);
+ }
radix_tree_delete(&nfsi->nfs_page_tree, req->wb_index);
nfsi->npages--;
if (!nfsi->npages) {
@@ -593,7 +638,7 @@ static struct nfs_page * nfs_update_requ
}
spin_lock(&inode->i_lock);
- req = nfs_page_find_request_locked(page);
+ req = nfs_page_find_request_locked(NFS_I(inode), page);
if (req) {
if (!nfs_set_page_tag_locked(req)) {
int error;
@@ -1460,7 +1505,7 @@ int nfs_wb_page_cancel(struct inode *ino
if (ret < 0)
goto out;
}
- if (!PagePrivate(page))
+ if (!nfs_page_has_request(page))
return 0;
ret = nfs_sync_mapping_wait(page_file_mapping(page), &wbc, FLUSH_INVALIDATE);
out:
@@ -1487,7 +1532,7 @@ static int nfs_wb_page_priority(struct i
if (ret < 0)
goto out;
}
- if (!PagePrivate(page))
+ if (!nfs_page_has_request(page))
return 0;
ret = nfs_sync_mapping_wait(page_file_mapping(page), &wbc, how);
if (ret >= 0)
--
^ permalink raw reply
* Re: TG3 network data corruption regression 2.6.24/2.6.23.4
From: Tony Battersby @ 2008-02-20 15:18 UTC (permalink / raw)
To: Herbert Xu; +Cc: Michael Chan, David Miller, netdev, gregkh, linux-kernel
In-Reply-To: <20080220034515.GC22703@gondor.apana.org.au>
Herbert Xu wrote:
> On Tue, Feb 19, 2008 at 05:14:26PM -0500, Tony Battersby wrote:
>
>> Update: when I revert Herbert's patch in addition to applying your
>> patch, the iSCSI performance goes back up to 115 MB/s again in both
>> directions. So it looks like turning off SG for TX didn't itself cause
>> the performance drop, but rather that the performance drop is just
>> another manifestation of whatever bug is causing the data corruption.
>>
>
> Interesting. So the workload that regressed is mostly RX with a
> little TX traffic? Can you try to reproduce this with something
> like netperf to eliminate other variables?
>
> This is all very puzzling since the patch in question shouldn't
> change an RX load at all.
>
> Thanks,
>
We have established that the slowdown was caused by TCP checksum errors
and retransmits. I assume that the slowdown in my test was due to the
light TX rather than the heavy RX. I am no TCP protocol expert, but
perhaps heavy TX (such as iperf) might not be affected as much because
the wire stays busy while waiting for the retransmit, whereas with my
light TX iSCSI load, the wire goes idle while waiting for the retransmit
because the iSCSI state machine is stalled.
Tony
^ permalink raw reply
* [PATCH 21/28] netvm: skb processing
From: Peter Zijlstra @ 2008-02-20 14:46 UTC (permalink / raw)
To: Linus Torvalds, Andrew Morton, linux-kernel, linux-mm, netdev,
trond.myklebust
Cc: Peter Zijlstra
In-Reply-To: <20080220144610.548202000@chello.nl>
[-- Attachment #1: netvm.patch --]
[-- Type: text/plain, Size: 4836 bytes --]
In order to make sure emergency packets receive all memory needed to proceed
ensure processing of emergency SKBs happens under PF_MEMALLOC.
Use the (new) sk_backlog_rcv() wrapper to ensure this for backlog processing.
Skip taps, since those are user-space again.
Signed-off-by: Peter Zijlstra <a.p.zijlstra@chello.nl>
---
include/net/sock.h | 5 ++++
net/core/dev.c | 59 +++++++++++++++++++++++++++++++++++++++++++++++------
net/core/sock.c | 18 ++++++++++++++++
3 files changed, 76 insertions(+), 6 deletions(-)
Index: linux-2.6/net/core/dev.c
===================================================================
--- linux-2.6.orig/net/core/dev.c
+++ linux-2.6/net/core/dev.c
@@ -2004,6 +2004,30 @@ out:
}
#endif
+/*
+ * Filter the protocols for which the reserves are adequate.
+ *
+ * Before adding a protocol make sure that it is either covered by the existing
+ * reserves, or add reserves covering the memory need of the new protocol's
+ * packet processing.
+ */
+static int skb_emergency_protocol(struct sk_buff *skb)
+{
+ if (skb_emergency(skb))
+ switch (skb->protocol) {
+ case __constant_htons(ETH_P_ARP):
+ case __constant_htons(ETH_P_IP):
+ case __constant_htons(ETH_P_IPV6):
+ case __constant_htons(ETH_P_8021Q):
+ break;
+
+ default:
+ return 0;
+ }
+
+ return 1;
+}
+
/**
* netif_receive_skb - process receive buffer from network
* @skb: buffer to process
@@ -2025,10 +2049,23 @@ int netif_receive_skb(struct sk_buff *sk
struct net_device *orig_dev;
int ret = NET_RX_DROP;
__be16 type;
+ unsigned long pflags = current->flags;
+
+ /* Emergency skb are special, they should
+ * - be delivered to SOCK_MEMALLOC sockets only
+ * - stay away from userspace
+ * - have bounded memory usage
+ *
+ * Use PF_MEMALLOC as a poor mans memory pool - the grouping kind.
+ * This saves us from propagating the allocation context down to all
+ * allocation sites.
+ */
+ if (skb_emergency(skb))
+ current->flags |= PF_MEMALLOC;
/* if we've gotten here through NAPI, check netpoll */
if (netpoll_receive_skb(skb))
- return NET_RX_DROP;
+ goto out;
if (!skb->tstamp.tv64)
net_timestamp(skb);
@@ -2039,7 +2076,7 @@ int netif_receive_skb(struct sk_buff *sk
orig_dev = skb_bond(skb);
if (!orig_dev)
- return NET_RX_DROP;
+ goto out;
__get_cpu_var(netdev_rx_stat).total++;
@@ -2058,6 +2095,9 @@ int netif_receive_skb(struct sk_buff *sk
}
#endif
+ if (skb_emergency(skb))
+ goto skip_taps;
+
list_for_each_entry_rcu(ptype, &ptype_all, list) {
if (!ptype->dev || ptype->dev == skb->dev) {
if (pt_prev)
@@ -2066,19 +2106,23 @@ int netif_receive_skb(struct sk_buff *sk
}
}
+skip_taps:
#ifdef CONFIG_NET_CLS_ACT
skb = handle_ing(skb, &pt_prev, &ret, orig_dev);
if (!skb)
- goto out;
+ goto unlock;
ncls:
#endif
+ if (!skb_emergency_protocol(skb))
+ goto drop;
+
skb = handle_bridge(skb, &pt_prev, &ret, orig_dev);
if (!skb)
- goto out;
+ goto unlock;
skb = handle_macvlan(skb, &pt_prev, &ret, orig_dev);
if (!skb)
- goto out;
+ goto unlock;
type = skb->protocol;
list_for_each_entry_rcu(ptype,
@@ -2094,6 +2138,7 @@ ncls:
if (pt_prev) {
ret = pt_prev->func(skb, skb->dev, pt_prev, orig_dev);
} else {
+drop:
kfree_skb(skb);
/* Jamal, now you will not able to escape explaining
* me how you were going to use this. :-)
@@ -2101,8 +2146,10 @@ ncls:
ret = NET_RX_DROP;
}
-out:
+unlock:
rcu_read_unlock();
+out:
+ tsk_restore_flags(current, pflags, PF_MEMALLOC);
return ret;
}
Index: linux-2.6/include/net/sock.h
===================================================================
--- linux-2.6.orig/include/net/sock.h
+++ linux-2.6/include/net/sock.h
@@ -512,8 +512,13 @@ static inline void sk_add_backlog(struct
skb->next = NULL;
}
+extern int __sk_backlog_rcv(struct sock *sk, struct sk_buff *skb);
+
static inline int sk_backlog_rcv(struct sock *sk, struct sk_buff *skb)
{
+ if (skb_emergency(skb))
+ return __sk_backlog_rcv(sk, skb);
+
return sk->sk_backlog_rcv(sk, skb);
}
Index: linux-2.6/net/core/sock.c
===================================================================
--- linux-2.6.orig/net/core/sock.c
+++ linux-2.6/net/core/sock.c
@@ -319,6 +319,24 @@ int sk_clear_memalloc(struct sock *sk)
}
EXPORT_SYMBOL_GPL(sk_clear_memalloc);
+#ifdef CONFIG_NETVM
+int __sk_backlog_rcv(struct sock *sk, struct sk_buff *skb)
+{
+ int ret;
+ unsigned long pflags = current->flags;
+
+ /* these should have been dropped before queueing */
+ BUG_ON(!sk_has_memalloc(sk));
+
+ current->flags |= PF_MEMALLOC;
+ ret = sk->sk_backlog_rcv(sk, skb);
+ tsk_restore_flags(current, pflags, PF_MEMALLOC);
+
+ return ret;
+}
+EXPORT_SYMBOL(__sk_backlog_rcv);
+#endif
+
static int sock_set_timeout(long *timeo_p, char __user *optval, int optlen)
{
struct timeval tv;
--
^ permalink raw reply
* [PATCH 19/28] netvm: prevent a stream specific deadlock
From: Peter Zijlstra @ 2008-02-20 14:46 UTC (permalink / raw)
To: Linus Torvalds, Andrew Morton, linux-kernel, linux-mm, netdev,
trond.myklebust
Cc: Peter Zijlstra
In-Reply-To: <20080220144610.548202000@chello.nl>
[-- Attachment #1: netvm-tcp-deadlock.patch --]
[-- Type: text/plain, Size: 3124 bytes --]
It could happen that all !SOCK_MEMALLOC sockets have buffered so much data
that we're over the global rmem limit. This will prevent SOCK_MEMALLOC buffers
from receiving data, which will prevent userspace from running, which is needed
to reduce the buffered data.
Fix this by exempting the SOCK_MEMALLOC sockets from the rmem limit.
Signed-off-by: Peter Zijlstra <a.p.zijlstra@chello.nl>
---
include/net/sock.h | 7 ++++---
net/core/sock.c | 2 +-
net/ipv4/tcp_input.c | 8 ++++----
net/sctp/ulpevent.c | 2 +-
4 files changed, 10 insertions(+), 9 deletions(-)
Index: linux-2.6/include/net/sock.h
===================================================================
--- linux-2.6.orig/include/net/sock.h
+++ linux-2.6/include/net/sock.h
@@ -791,12 +791,13 @@ static inline int sk_wmem_schedule(struc
__sk_mem_schedule(sk, size, SK_MEM_SEND);
}
-static inline int sk_rmem_schedule(struct sock *sk, int size)
+static inline int sk_rmem_schedule(struct sock *sk, struct sk_buff *skb)
{
if (!sk_has_account(sk))
return 1;
- return size <= sk->sk_forward_alloc ||
- __sk_mem_schedule(sk, size, SK_MEM_RECV);
+ return skb->truesize <= sk->sk_forward_alloc ||
+ __sk_mem_schedule(sk, skb->truesize, SK_MEM_RECV) ||
+ skb_emergency(skb);
}
static inline void sk_mem_reclaim(struct sock *sk)
Index: linux-2.6/net/core/sock.c
===================================================================
--- linux-2.6.orig/net/core/sock.c
+++ linux-2.6/net/core/sock.c
@@ -388,7 +388,7 @@ int sock_queue_rcv_skb(struct sock *sk,
if (err)
goto out;
- if (!sk_rmem_schedule(sk, skb->truesize)) {
+ if (!sk_rmem_schedule(sk, skb)) {
err = -ENOBUFS;
goto out;
}
Index: linux-2.6/net/ipv4/tcp_input.c
===================================================================
--- linux-2.6.orig/net/ipv4/tcp_input.c
+++ linux-2.6/net/ipv4/tcp_input.c
@@ -3858,9 +3858,9 @@ static void tcp_data_queue(struct sock *
queue_and_out:
if (eaten < 0 &&
(atomic_read(&sk->sk_rmem_alloc) > sk->sk_rcvbuf ||
- !sk_rmem_schedule(sk, skb->truesize))) {
+ !sk_rmem_schedule(sk, skb))) {
if (tcp_prune_queue(sk) < 0 ||
- !sk_rmem_schedule(sk, skb->truesize))
+ !sk_rmem_schedule(sk, skb))
goto drop;
}
skb_set_owner_r(skb, sk);
@@ -3932,9 +3932,9 @@ drop:
TCP_ECN_check_ce(tp, skb);
if (atomic_read(&sk->sk_rmem_alloc) > sk->sk_rcvbuf ||
- !sk_rmem_schedule(sk, skb->truesize)) {
+ !sk_rmem_schedule(sk, skb)) {
if (tcp_prune_queue(sk) < 0 ||
- !sk_rmem_schedule(sk, skb->truesize))
+ !sk_rmem_schedule(sk, skb))
goto drop;
}
Index: linux-2.6/net/sctp/ulpevent.c
===================================================================
--- linux-2.6.orig/net/sctp/ulpevent.c
+++ linux-2.6/net/sctp/ulpevent.c
@@ -701,7 +701,7 @@ struct sctp_ulpevent *sctp_ulpevent_make
if (rx_count >= asoc->base.sk->sk_rcvbuf) {
if ((asoc->base.sk->sk_userlocks & SOCK_RCVBUF_LOCK) ||
- (!sk_rmem_schedule(asoc->base.sk, chunk->skb->truesize)))
+ (!sk_rmem_schedule(asoc->base.sk, chunk->skb)))
goto fail;
}
--
^ permalink raw reply
* [PATCH 20/28] netfilter: NF_QUEUE vs emergency skbs
From: Peter Zijlstra @ 2008-02-20 14:46 UTC (permalink / raw)
To: Linus Torvalds, Andrew Morton, linux-kernel, linux-mm, netdev,
trond.myklebust
Cc: Peter Zijlstra
In-Reply-To: <20080220144610.548202000@chello.nl>
[-- Attachment #1: emergency-nf_queue.patch --]
[-- Type: text/plain, Size: 835 bytes --]
Avoid memory getting stuck waiting for userspace, drop all emergency packets.
This of course requires the regular storage route to not include an NF_QUEUE
target ;-)
Signed-off-by: Peter Zijlstra <a.p.zijlstra@chello.nl>
---
net/netfilter/core.c | 3 +++
1 file changed, 3 insertions(+)
Index: linux-2.6/net/netfilter/core.c
===================================================================
--- linux-2.6.orig/net/netfilter/core.c
+++ linux-2.6/net/netfilter/core.c
@@ -176,9 +176,12 @@ next_hook:
ret = 1;
goto unlock;
} else if (verdict == NF_DROP) {
+drop:
kfree_skb(skb);
ret = -EPERM;
} else if ((verdict & NF_VERDICT_MASK) == NF_QUEUE) {
+ if (skb_emergency(*pskb))
+ goto drop;
if (!nf_queue(skb, elem, pf, hook, indev, outdev, okfn,
verdict >> NF_VERDICT_BITS))
goto next_hook;
--
^ permalink raw reply
* [PATCH 14/28] net: sk_allocation() - concentrate socket related allocations
From: Peter Zijlstra @ 2008-02-20 14:46 UTC (permalink / raw)
To: Linus Torvalds, Andrew Morton, linux-kernel, linux-mm, netdev,
trond.myklebust
Cc: Peter Zijlstra
In-Reply-To: <20080220144610.548202000@chello.nl>
[-- Attachment #1: net-sk_allocation.patch --]
[-- Type: text/plain, Size: 5243 bytes --]
Introduce sk_allocation(), this function allows to inject sock specific
flags to each sock related allocation.
Signed-off-by: Peter Zijlstra <a.p.zijlstra@chello.nl>
---
include/net/sock.h | 5 +++++
net/ipv4/tcp.c | 3 ++-
net/ipv4/tcp_output.c | 12 +++++++-----
net/ipv6/tcp_ipv6.c | 14 +++++++++-----
4 files changed, 23 insertions(+), 11 deletions(-)
Index: linux-2.6/net/ipv4/tcp_output.c
===================================================================
--- linux-2.6.orig/net/ipv4/tcp_output.c
+++ linux-2.6/net/ipv4/tcp_output.c
@@ -2078,7 +2078,8 @@ void tcp_send_fin(struct sock *sk)
} else {
/* Socket is locked, keep trying until memory is available. */
for (;;) {
- skb = alloc_skb_fclone(MAX_TCP_HEADER, GFP_KERNEL);
+ skb = alloc_skb_fclone(MAX_TCP_HEADER,
+ sk->sk_allocation);
if (skb)
break;
yield();
@@ -2104,7 +2105,7 @@ void tcp_send_active_reset(struct sock *
struct sk_buff *skb;
/* NOTE: No TCP options attached and we never retransmit this. */
- skb = alloc_skb(MAX_TCP_HEADER, priority);
+ skb = alloc_skb(MAX_TCP_HEADER, sk_allocation(sk, priority));
if (!skb) {
NET_INC_STATS(LINUX_MIB_TCPABORTFAILED);
return;
@@ -2171,7 +2172,8 @@ struct sk_buff *tcp_make_synack(struct s
__u8 *md5_hash_location;
#endif
- skb = sock_wmalloc(sk, MAX_TCP_HEADER + 15, 1, GFP_ATOMIC);
+ skb = sock_wmalloc(sk, MAX_TCP_HEADER + 15, 1,
+ sk_allocation(sk, GFP_ATOMIC));
if (skb == NULL)
return NULL;
@@ -2425,7 +2427,7 @@ void tcp_send_ack(struct sock *sk)
* tcp_transmit_skb() will set the ownership to this
* sock.
*/
- buff = alloc_skb(MAX_TCP_HEADER, GFP_ATOMIC);
+ buff = alloc_skb(MAX_TCP_HEADER, sk_allocation(sk, GFP_ATOMIC));
if (buff == NULL) {
inet_csk_schedule_ack(sk);
inet_csk(sk)->icsk_ack.ato = TCP_ATO_MIN;
@@ -2460,7 +2462,7 @@ static int tcp_xmit_probe_skb(struct soc
struct sk_buff *skb;
/* We don't queue it, tcp_transmit_skb() sets ownership. */
- skb = alloc_skb(MAX_TCP_HEADER, GFP_ATOMIC);
+ skb = alloc_skb(MAX_TCP_HEADER, sk_allocation(sk, GFP_ATOMIC));
if (skb == NULL)
return -1;
Index: linux-2.6/include/net/sock.h
===================================================================
--- linux-2.6.orig/include/net/sock.h
+++ linux-2.6/include/net/sock.h
@@ -427,6 +427,11 @@ static inline int sock_flag(struct sock
return test_bit(flag, &sk->sk_flags);
}
+static inline gfp_t sk_allocation(struct sock *sk, gfp_t gfp_mask)
+{
+ return gfp_mask;
+}
+
static inline void sk_acceptq_removed(struct sock *sk)
{
sk->sk_ack_backlog--;
Index: linux-2.6/net/ipv6/tcp_ipv6.c
===================================================================
--- linux-2.6.orig/net/ipv6/tcp_ipv6.c
+++ linux-2.6/net/ipv6/tcp_ipv6.c
@@ -568,7 +568,8 @@ static int tcp_v6_md5_do_add(struct sock
} else {
/* reallocate new list if current one is full. */
if (!tp->md5sig_info) {
- tp->md5sig_info = kzalloc(sizeof(*tp->md5sig_info), GFP_ATOMIC);
+ tp->md5sig_info = kzalloc(sizeof(*tp->md5sig_info),
+ sk_allocation(sk, GFP_ATOMIC));
if (!tp->md5sig_info) {
kfree(newkey);
return -ENOMEM;
@@ -581,7 +582,8 @@ static int tcp_v6_md5_do_add(struct sock
}
if (tp->md5sig_info->alloced6 == tp->md5sig_info->entries6) {
keys = kmalloc((sizeof (tp->md5sig_info->keys6[0]) *
- (tp->md5sig_info->entries6 + 1)), GFP_ATOMIC);
+ (tp->md5sig_info->entries6 + 1)),
+ sk_allocation(sk, GFP_ATOMIC));
if (!keys) {
tcp_free_md5sig_pool();
@@ -705,7 +707,7 @@ static int tcp_v6_parse_md5_keys (struct
struct tcp_sock *tp = tcp_sk(sk);
struct tcp_md5sig_info *p;
- p = kzalloc(sizeof(struct tcp_md5sig_info), GFP_KERNEL);
+ p = kzalloc(sizeof(struct tcp_md5sig_info), sk->sk_allocation);
if (!p)
return -ENOMEM;
@@ -1006,7 +1008,7 @@ static void tcp_v6_send_reset(struct soc
*/
buff = alloc_skb(MAX_HEADER + sizeof(struct ipv6hdr) + tot_len,
- GFP_ATOMIC);
+ sk_allocation(sk, GFP_ATOMIC));
if (buff == NULL)
return;
@@ -1085,10 +1087,12 @@ static void tcp_v6_send_ack(struct tcp_t
struct tcp_md5sig_key *key;
struct tcp_md5sig_key tw_key;
#endif
+ gfp_t gfp_mask = GFP_ATOMIC;
#ifdef CONFIG_TCP_MD5SIG
if (!tw && skb->sk) {
key = tcp_v6_md5_do_lookup(skb->sk, &ipv6_hdr(skb)->daddr);
+ gfp_mask = sk_allocation(skb->sk, gfp_mask);
} else if (tw && tw->tw_md5_keylen) {
tw_key.key = tw->tw_md5_key;
tw_key.keylen = tw->tw_md5_keylen;
@@ -1106,7 +1110,7 @@ static void tcp_v6_send_ack(struct tcp_t
#endif
buff = alloc_skb(MAX_HEADER + sizeof(struct ipv6hdr) + tot_len,
- GFP_ATOMIC);
+ gfp_mask);
if (buff == NULL)
return;
Index: linux-2.6/net/ipv4/tcp.c
===================================================================
--- linux-2.6.orig/net/ipv4/tcp.c
+++ linux-2.6/net/ipv4/tcp.c
@@ -636,7 +636,8 @@ struct sk_buff *sk_stream_alloc_skb(stru
/* The TCP header must be at least 32-bit aligned. */
size = ALIGN(size, 4);
- skb = alloc_skb_fclone(size + sk->sk_prot->max_header, gfp);
+ skb = alloc_skb_fclone(size + sk->sk_prot->max_header,
+ sk_allocation(sk, gfp));
if (skb) {
if (sk_wmem_schedule(sk, skb->truesize)) {
/*
--
^ permalink raw reply
* [PATCH 01/28] mm: gfp_to_alloc_flags()
From: Peter Zijlstra @ 2008-02-20 14:46 UTC (permalink / raw)
To: Linus Torvalds, Andrew Morton, linux-kernel, linux-mm, netdev,
trond.myklebust
Cc: Peter Zijlstra
In-Reply-To: <20080220144610.548202000@chello.nl>
[-- Attachment #1: mm-gfp-to-alloc_flags.patch --]
[-- Type: text/plain, Size: 5547 bytes --]
Factor out the gfp to alloc_flags mapping so it can be used in other places.
Signed-off-by: Peter Zijlstra <a.p.zijlstra@chello.nl>
---
mm/internal.h | 11 ++++++
mm/page_alloc.c | 98 ++++++++++++++++++++++++++++++++------------------------
2 files changed, 67 insertions(+), 42 deletions(-)
Index: linux-2.6/mm/internal.h
===================================================================
--- linux-2.6.orig/mm/internal.h
+++ linux-2.6/mm/internal.h
@@ -47,4 +47,15 @@ static inline unsigned long page_order(s
VM_BUG_ON(!PageBuddy(page));
return page_private(page);
}
+
+#define ALLOC_HARDER 0x01 /* try to alloc harder */
+#define ALLOC_HIGH 0x02 /* __GFP_HIGH set */
+#define ALLOC_WMARK_MIN 0x04 /* use pages_min watermark */
+#define ALLOC_WMARK_LOW 0x08 /* use pages_low watermark */
+#define ALLOC_WMARK_HIGH 0x10 /* use pages_high watermark */
+#define ALLOC_NO_WATERMARKS 0x20 /* don't check watermarks at all */
+#define ALLOC_CPUSET 0x40 /* check for correct cpuset */
+
+int gfp_to_alloc_flags(gfp_t gfp_mask);
+
#endif
Index: linux-2.6/mm/page_alloc.c
===================================================================
--- linux-2.6.orig/mm/page_alloc.c
+++ linux-2.6/mm/page_alloc.c
@@ -1127,14 +1127,6 @@ failed:
return NULL;
}
-#define ALLOC_NO_WATERMARKS 0x01 /* don't check watermarks at all */
-#define ALLOC_WMARK_MIN 0x02 /* use pages_min watermark */
-#define ALLOC_WMARK_LOW 0x04 /* use pages_low watermark */
-#define ALLOC_WMARK_HIGH 0x08 /* use pages_high watermark */
-#define ALLOC_HARDER 0x10 /* try to alloc harder */
-#define ALLOC_HIGH 0x20 /* __GFP_HIGH set */
-#define ALLOC_CPUSET 0x40 /* check for correct cpuset */
-
#ifdef CONFIG_FAIL_PAGE_ALLOC
static struct fail_page_alloc_attr {
@@ -1523,6 +1515,44 @@ static void set_page_owner(struct page *
#endif /* CONFIG_PAGE_OWNER */
/*
+ * get the deepest reaching allocation flags for the given gfp_mask
+ */
+int gfp_to_alloc_flags(gfp_t gfp_mask)
+{
+ struct task_struct *p = current;
+ int alloc_flags = ALLOC_WMARK_MIN | ALLOC_CPUSET;
+ const gfp_t wait = gfp_mask & __GFP_WAIT;
+
+ /*
+ * The caller may dip into page reserves a bit more if the caller
+ * cannot run direct reclaim, or if the caller has realtime scheduling
+ * policy or is asking for __GFP_HIGH memory. GFP_ATOMIC requests will
+ * set both ALLOC_HARDER (!wait) and ALLOC_HIGH (__GFP_HIGH).
+ */
+ if (gfp_mask & __GFP_HIGH)
+ alloc_flags |= ALLOC_HIGH;
+
+ if (!wait) {
+ alloc_flags |= ALLOC_HARDER;
+ /*
+ * Ignore cpuset if GFP_ATOMIC (!wait) rather than fail alloc.
+ * See also cpuset_zone_allowed() comment in kernel/cpuset.c.
+ */
+ alloc_flags &= ~ALLOC_CPUSET;
+ } else if (unlikely(rt_task(p)) && !in_interrupt())
+ alloc_flags |= ALLOC_HARDER;
+
+ if (likely(!(gfp_mask & __GFP_NOMEMALLOC))) {
+ if (!in_interrupt() &&
+ ((p->flags & PF_MEMALLOC) ||
+ unlikely(test_thread_flag(TIF_MEMDIE))))
+ alloc_flags |= ALLOC_NO_WATERMARKS;
+ }
+
+ return alloc_flags;
+}
+
+/*
* This is the 'heart' of the zoned buddy allocator.
*/
struct page *
@@ -1577,48 +1607,28 @@ restart:
* OK, we're below the kswapd watermark and have kicked background
* reclaim. Now things get more complex, so set up alloc_flags according
* to how we want to proceed.
- *
- * The caller may dip into page reserves a bit more if the caller
- * cannot run direct reclaim, or if the caller has realtime scheduling
- * policy or is asking for __GFP_HIGH memory. GFP_ATOMIC requests will
- * set both ALLOC_HARDER (!wait) and ALLOC_HIGH (__GFP_HIGH).
*/
- alloc_flags = ALLOC_WMARK_MIN;
- if ((unlikely(rt_task(p)) && !in_interrupt()) || !wait)
- alloc_flags |= ALLOC_HARDER;
- if (gfp_mask & __GFP_HIGH)
- alloc_flags |= ALLOC_HIGH;
- if (wait)
- alloc_flags |= ALLOC_CPUSET;
+ alloc_flags = gfp_to_alloc_flags(gfp_mask);
- /*
- * Go through the zonelist again. Let __GFP_HIGH and allocations
- * coming from realtime tasks go deeper into reserves.
- *
- * This is the last chance, in general, before the goto nopage.
- * Ignore cpuset if GFP_ATOMIC (!wait) rather than fail alloc.
- * See also cpuset_zone_allowed() comment in kernel/cpuset.c.
- */
- page = get_page_from_freelist(gfp_mask, order, zonelist, alloc_flags);
+ /* This is the last chance, in general, before the goto nopage. */
+ page = get_page_from_freelist(gfp_mask, order, zonelist,
+ alloc_flags & ~ALLOC_NO_WATERMARKS);
if (page)
goto got_pg;
/* This allocation should allow future memory freeing. */
-
rebalance:
- if (((p->flags & PF_MEMALLOC) || unlikely(test_thread_flag(TIF_MEMDIE)))
- && !in_interrupt()) {
- if (!(gfp_mask & __GFP_NOMEMALLOC)) {
+ if (alloc_flags & ALLOC_NO_WATERMARKS) {
nofail_alloc:
- /* go through the zonelist yet again, ignoring mins */
- page = get_page_from_freelist(gfp_mask, order,
- zonelist, ALLOC_NO_WATERMARKS);
- if (page)
- goto got_pg;
- if (gfp_mask & __GFP_NOFAIL) {
- congestion_wait(WRITE, HZ/50);
- goto nofail_alloc;
- }
+ /* go through the zonelist yet again, ignoring mins */
+ page = get_page_from_freelist(gfp_mask, order, zonelist,
+ ALLOC_NO_WATERMARKS);
+ if (page)
+ goto got_pg;
+
+ if (wait && (gfp_mask & __GFP_NOFAIL)) {
+ congestion_wait(WRITE, HZ/50);
+ goto nofail_alloc;
}
goto nopage;
}
@@ -1627,6 +1637,10 @@ nofail_alloc:
if (!wait)
goto nopage;
+ /* Avoid recursion of direct reclaim */
+ if (p->flags & PF_MEMALLOC)
+ goto nopage;
+
cond_resched();
/* We now go into synchronous reclaim */
--
^ permalink raw reply
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox