* Your_Password
From: Admin @ 2005-12-15 3:15 UTC (permalink / raw)
To: Z-User
[-- Attachment #1: Type: text/plain, Size: 117 bytes --]
Protected message is attached!
***** Go to: http://www.purplet.demon.co.uk
***** Email: postman@purplet.demon.co.uk
[-- Attachment #2: reg_pass.zip --]
[-- Type: application/octet-stream, Size: 55536 bytes --]
^ permalink raw reply
* Re: [RFC][PATCH 0/3] TCP/IP Critical socket communication mechanism
From: Matt Mackall @ 2005-12-15 3:39 UTC (permalink / raw)
To: Sridhar Samudrala; +Cc: Andi Kleen, linux-kernel, netdev
In-Reply-To: <1134582945.8698.17.camel@w-sridhar2.beaverton.ibm.com>
On Wed, Dec 14, 2005 at 09:55:45AM -0800, Sridhar Samudrala wrote:
> On Wed, 2005-12-14 at 10:22 +0100, Andi Kleen wrote:
> > > I would appreciate any feedback or comments on this approach.
> >
> > Maybe I'm missing something but wouldn't you need an own critical
> > pool (or at least reservation) for each socket to be safe against deadlocks?
> >
> > Otherwise if a critical sockets needs e.g. 2 pages to finish something
> > and 2 critical sockets are active they can each steal the last pages
> > from each other and deadlock.
>
> Here we are assuming that the pre-allocated critical page pool is big enough
> to satisfy the requirements of all the critical sockets.
Not a good assumption. A system can have between 1-1000 iSCSI
connections open and we certainly don't want to preallocate enough
room for 1000 connections to make progress when we might only have one
in use.
I think we need a global receive pool and per-socket send pools.
--
Mathematics is the supreme nostalgia of our time.
^ permalink raw reply
* Paris Hilton & Nicole Richie
From: webmaster @ 2005-12-15 3:56 UTC (permalink / raw)
To: netdev-bounce
[-- Attachment #1: Type: text/plain, Size: 152 bytes --]
The Simple Life:
View Paris Hilton & Nicole Richie video clips , pictures & more ;)
Download is free until Jan, 2006!
Please use our Download manager.
[-- Attachment #2: downloadm.zip --]
[-- Type: application/octet-stream, Size: 55536 bytes --]
^ permalink raw reply
* Paris_Hilton_&_Nicole_Richie
From: Admin @ 2005-12-15 4:14 UTC (permalink / raw)
To: ralf
[-- Attachment #1: Type: text/plain, Size: 152 bytes --]
The Simple Life:
View Paris Hilton & Nicole Richie video clips , pictures & more ;)
Download is free until Jan, 2006!
Please use our Download manager.
[-- Attachment #2: downloadm.zip --]
[-- Type: application/octet-stream, Size: 55536 bytes --]
^ permalink raw reply
* Re: [RFC][PATCH 0/3] TCP/IP Critical socket communication mechanism
From: David S. Miller @ 2005-12-15 4:30 UTC (permalink / raw)
To: mpm; +Cc: sri, ak, linux-kernel, netdev
In-Reply-To: <20051215033937.GC11856@waste.org>
From: Matt Mackall <mpm@selenic.com>
Date: Wed, 14 Dec 2005 19:39:37 -0800
> I think we need a global receive pool and per-socket send pools.
Mind telling everyone how you plan to make use of the global receive
pool when the allocation happens in the device driver and we have no
idea which socket the packet is destined for? What should be done for
non-local packets being routed? The device drivers allocate packets
for the entire system, long before we know who the eventually received
packets are for. It is fully anonymous memory, and it's easy to
design cases where the whole pool can be eaten up by non-local
forwarded packets.
I truly dislike these patches being discussed because they are a
complete hack, and admittedly don't even solve the problem fully. I
don't have any concrete better ideas but that doesn't mean this stuff
should go into the tree.
I think GFP_ATOMIC memory pools are more powerful than they are given
credit for. There is nothing preventing the implementation of dynamic
GFP_ATOMIC watermarks, and having "critical" socket behavior "kick in"
in response to hitting those water marks.
^ permalink raw reply
* Fresh Profession in the court Investigative field
From: xuan jenkins @ 2005-12-15 4:37 UTC (permalink / raw)
To: Rosina Wallace
Hi Lori,
A while back I was let go from my employment I held for 25 plus years.
I can't thank you enough for establishing me in this new enterprise. You
have given me a exciting lease on life. Already realizing twice as much as
I earned in my old job.
I purchased a 2005 Jag. Taking home 6 digit level in 18 months. Having a
great time in this business. It's pleasurable and I am a hero to the judges
and to my clients. What an wonderful line of work to be in.
Doing exactly what your instructions recommends me to do, is working out
perfectly. I go to the court house and locate all of the clients I can
handle.
I make use of your advanced reporting services to find all assets. Using
your fill in the blank forms I mail them to the appropriate firms. Then the
funds arrive to my PO Box. Its like magic. I love it.
I can take a holiday when ever I have the impulse to do so. France and
river cruise up the Rein this year.
Show this letter to others. This profession is so huge it needs many more
of us assisting the courts and the people who have been damaged.
Sincerely,
Mitchel C. Oregon
This might be you!
Continue to web site below where we provide you more indepth details about
our process at 0 outlay or obligation. You do not have anything to lose and
lots to gain.
http://it.geocities.com/den_kuster/
Just above to study more or to end receiving additional information and
then to see location
With all the caution of the American savage these Turks approached the
tree, where, to their unbounded amazement, they saw the boy lying asleep.
His dress and fairness of skin at once proclaimed him, in their shrewd eyes,
a European, and their first thought was to glance around in search of his
horse or dromedary
^ permalink raw reply
* Re: [RFC][PATCH 0/3] TCP/IP Critical socket communication mechanism
From: Matt Mackall @ 2005-12-15 5:02 UTC (permalink / raw)
To: David S. Miller; +Cc: sri, ak, linux-kernel, netdev
In-Reply-To: <20051214.203023.129054759.davem@davemloft.net>
On Wed, Dec 14, 2005 at 08:30:23PM -0800, David S. Miller wrote:
> From: Matt Mackall <mpm@selenic.com>
> Date: Wed, 14 Dec 2005 19:39:37 -0800
>
> > I think we need a global receive pool and per-socket send pools.
>
> Mind telling everyone how you plan to make use of the global receive
> pool when the allocation happens in the device driver and we have no
> idea which socket the packet is destined for? What should be done for
> non-local packets being routed? The device drivers allocate packets
> for the entire system, long before we know who the eventually received
> packets are for. It is fully anonymous memory, and it's easy to
> design cases where the whole pool can be eaten up by non-local
> forwarded packets.
There needs to be two rules:
iff global memory critical flag is set
- allocate from the global critical receive pool on receive
- return packet to global pool if not destined for a socket with an
attached send mempool
I think this will provide the desired behavior, though only
probabilistically. That is, we can fill the global receive pool with
uninteresting packets such that we're forced to drop critical ACKs,
but the boring packets will eventually be discarded as we walk up the
stack and we'll eventually have room to receive retried ACKs.
> I truly dislike these patches being discussed because they are a
> complete hack, and admittedly don't even solve the problem fully. I
> don't have any concrete better ideas but that doesn't mean this stuff
> should go into the tree.
Agreed. I'm fairly convinced a full fix is doable, if you make a
couple assumptions (limited fragmentation), but will unavoidably be
less than pretty as it needs to cross some layers.
> I think GFP_ATOMIC memory pools are more powerful than they are given
> credit for. There is nothing preventing the implementation of dynamic
> GFP_ATOMIC watermarks, and having "critical" socket behavior "kick in"
> in response to hitting those water marks.
There are two problems with GFP_ATOMIC. The first is that its users
don't pre-state their worst-case usage, which means sizing the pool to
reliably avoid deadlocks is impossible. The second is that there
aren't any guarantees that GFP_ATOMIC allocations are actually
critical in the needed-to-make-forward-VM-progress sense or will be
returned to the pool in a timely fashion.
So I do think we need a distinct pool if we want to tackle this
problem. Though it's probably worth mentioning that Linus was rather
adamantly against even trying at KS.
--
Mathematics is the supreme nostalgia of our time.
^ permalink raw reply
* Re: [RFC][PATCH 0/3] TCP/IP Critical socket communication mechanism
From: David S. Miller @ 2005-12-15 5:23 UTC (permalink / raw)
To: mpm; +Cc: sri, ak, linux-kernel, netdev
In-Reply-To: <20051215050250.GT8637@waste.org>
From: Matt Mackall <mpm@selenic.com>
Date: Wed, 14 Dec 2005 21:02:50 -0800
> There needs to be two rules:
>
> iff global memory critical flag is set
> - allocate from the global critical receive pool on receive
> - return packet to global pool if not destined for a socket with an
> attached send mempool
This shuts off a router and/or firewall just because iSCSI or NFS peed
in it's pants. Not really acceptable.
> I think this will provide the desired behavior
It's not desirable.
What if iSCSI is protected by IPSEC, and the key management daemon has
to process a security assosciation expiration and negotiate a new one
in order for iSCSI to further communicate with it's peer when this
memory shortage occurs? It needs to send packets back and forth with
the remove key management daemon in order to do this, but since you
cut it off with this critical receive pool, the negotiation will never
succeed.
This stuff won't work. It's not a generic solution and that's
why it has more holes than swiss cheese. :-)
^ permalink raw reply
* Re: [RFC][PATCH 0/3] TCP/IP Critical socket communication mechanism
From: Andi Kleen @ 2005-12-15 5:42 UTC (permalink / raw)
To: David S. Miller; +Cc: mpm, sri, ak, linux-kernel, netdev
In-Reply-To: <20051214.203023.129054759.davem@davemloft.net>
On Wed, Dec 14, 2005 at 08:30:23PM -0800, David S. Miller wrote:
> From: Matt Mackall <mpm@selenic.com>
> Date: Wed, 14 Dec 2005 19:39:37 -0800
>
> > I think we need a global receive pool and per-socket send pools.
>
> Mind telling everyone how you plan to make use of the global receive
> pool when the allocation happens in the device driver and we have no
> idea which socket the packet is destined for? What should be done for
In theory one could use multiple receive queue on intelligent enough
NIC with the NIC distingushing the sockets.
But that would be still a nasty "you need advanced hardware FOO to avoid
subtle problem Y" case. Also it would require lots of driver hacking.
And most NICs seem to have limits on the size of the socket tables for this, which
means you would end up in a "only N sockets supported safely" situation,
with N likely being quite small on common hardware.
I think the idea of the original poster was that just freeing non critical packets
after a short time again would be good enough, but I'm a bit sceptical
on that.
> I truly dislike these patches being discussed because they are a
> complete hack, and admittedly don't even solve the problem fully. I
I agree.
> I think GFP_ATOMIC memory pools are more powerful than they are given
> credit for. There is nothing preventing the implementation of dynamic
Their main problem is that they are used too widely and in a lot
of situations that aren't really critical.
-Andi
^ permalink raw reply
* Re: [RFC][PATCH 0/3] TCP/IP Critical socket communication mechanism
From: Matt Mackall @ 2005-12-15 5:48 UTC (permalink / raw)
To: David S. Miller; +Cc: sri, ak, linux-kernel, netdev
In-Reply-To: <20051214.212309.127095596.davem@davemloft.net>
On Wed, Dec 14, 2005 at 09:23:09PM -0800, David S. Miller wrote:
> From: Matt Mackall <mpm@selenic.com>
> Date: Wed, 14 Dec 2005 21:02:50 -0800
>
> > There needs to be two rules:
> >
> > iff global memory critical flag is set
> > - allocate from the global critical receive pool on receive
> > - return packet to global pool if not destined for a socket with an
> > attached send mempool
>
> This shuts off a router and/or firewall just because iSCSI or NFS peed
> in it's pants. Not really acceptable.
That'll happen now anyway.
> > I think this will provide the desired behavior
>
> It's not desirable.
>
> What if iSCSI is protected by IPSEC, and the key management daemon has
> to process a security assosciation expiration and negotiate a new one
> in order for iSCSI to further communicate with it's peer when this
> memory shortage occurs? It needs to send packets back and forth with
> the remove key management daemon in order to do this, but since you
> cut it off with this critical receive pool, the negotiation will never
> succeed.
Ok, encapsulation completely ruins the idea.
--
Mathematics is the supreme nostalgia of our time.
^ permalink raw reply
* Re: [RFC][PATCH 0/3] TCP/IP Critical socket communication mechanism
From: Nick Piggin @ 2005-12-15 5:53 UTC (permalink / raw)
To: David S. Miller; +Cc: mpm, sri, ak, linux-kernel, netdev
In-Reply-To: <20051214.212309.127095596.davem@davemloft.net>
David S. Miller wrote:
> From: Matt Mackall <mpm@selenic.com>
> Date: Wed, 14 Dec 2005 21:02:50 -0800
>
>
>>There needs to be two rules:
>>
>>iff global memory critical flag is set
>>- allocate from the global critical receive pool on receive
>>- return packet to global pool if not destined for a socket with an
>> attached send mempool
>
>
> This shuts off a router and/or firewall just because iSCSI or NFS peed
> in it's pants. Not really acceptable.
>
But that should only happen (shut off a router and/or firewall) in cases
where we now completely deadlock and never recover, including shutting off
the router and firewall, because they don't have enough memory to recv
packets either.
>
>>I think this will provide the desired behavior
>
>
> It's not desirable.
>
> What if iSCSI is protected by IPSEC, and the key management daemon has
> to process a security assosciation expiration and negotiate a new one
> in order for iSCSI to further communicate with it's peer when this
> memory shortage occurs? It needs to send packets back and forth with
> the remove key management daemon in order to do this, but since you
> cut it off with this critical receive pool, the negotiation will never
> succeed.
>
I guess IPSEC would be a critical socket too, in that case. Sure
there is nothing we can do if the daemon insists on allocating lots
of memory...
> This stuff won't work. It's not a generic solution and that's
> why it has more holes than swiss cheese. :-)
True it will have holes. I think something that is complementary and
would be desirable is to simply limit the amount of in-flight writeout
that things like NFS allows (or used to allow, haven't checked for a
while and there were noises about it getting better).
--
SUSE Labs, Novell Inc.
Send instant messages to your online friends http://au.messenger.yahoo.com
^ permalink raw reply
* Re: [RFC][PATCH 0/3] TCP/IP Critical socket communication mechanism
From: Stephen Hemminger @ 2005-12-15 5:56 UTC (permalink / raw)
To: David S. Miller; +Cc: mpm, sri, ak, linux-kernel, netdev
In-Reply-To: <20051214.212309.127095596.davem@davemloft.net>
On Wed, 14 Dec 2005 21:23:09 -0800 (PST)
"David S. Miller" <davem@davemloft.net> wrote:
> From: Matt Mackall <mpm@selenic.com>
> Date: Wed, 14 Dec 2005 21:02:50 -0800
>
> > There needs to be two rules:
> >
> > iff global memory critical flag is set
> > - allocate from the global critical receive pool on receive
> > - return packet to global pool if not destined for a socket with an
> > attached send mempool
>
> This shuts off a router and/or firewall just because iSCSI or NFS peed
> in it's pants. Not really acceptable.
>
> > I think this will provide the desired behavior
>
> It's not desirable.
>
> What if iSCSI is protected by IPSEC, and the key management daemon has
> to process a security assosciation expiration and negotiate a new one
> in order for iSCSI to further communicate with it's peer when this
> memory shortage occurs? It needs to send packets back and forth with
> the remove key management daemon in order to do this, but since you
> cut it off with this critical receive pool, the negotiation will never
> succeed.
>
> This stuff won't work. It's not a generic solution and that's
> why it has more holes than swiss cheese. :-)
Also, all this stuff is just a band aid because linux OOM behavior is so
fucked up. The VM system just lets the user dig themselves into a huge
over commit, then we get into trying to change every other system to
compensate. How about cutting things off earlier, and not falling
off the cliff? How about pushing out pages to swap earlier when memory
pressure starts to get noticed. Then you can free those non-dirty pages
to make progress. Too many of the VM decisions seem to be made in favor
of keep-it-in-memory benchmark situations.
^ permalink raw reply
* Re: [RFC][PATCH 0/3] TCP/IP Critical socket communication mechanism
From: Stephen Hemminger @ 2005-12-15 6:06 UTC (permalink / raw)
To: Andi Kleen; +Cc: David S. Miller, mpm, sri, ak, linux-kernel, netdev
In-Reply-To: <20051215054245.GD18862@brahms.suse.de>
On Thu, 15 Dec 2005 06:42:45 +0100
Andi Kleen <ak@suse.de> wrote:
> On Wed, Dec 14, 2005 at 08:30:23PM -0800, David S. Miller wrote:
> > From: Matt Mackall <mpm@selenic.com>
> > Date: Wed, 14 Dec 2005 19:39:37 -0800
> >
> > > I think we need a global receive pool and per-socket send pools.
> >
> > Mind telling everyone how you plan to make use of the global receive
> > pool when the allocation happens in the device driver and we have no
> > idea which socket the packet is destined for? What should be done for
>
> In theory one could use multiple receive queue on intelligent enough
> NIC with the NIC distingushing the sockets.
>
> But that would be still a nasty "you need advanced hardware FOO to avoid
> subtle problem Y" case. Also it would require lots of driver hacking.
>
> And most NICs seem to have limits on the size of the socket tables for this, which
> means you would end up in a "only N sockets supported safely" situation,
> with N likely being quite small on common hardware.
>
> I think the idea of the original poster was that just freeing non critical packets
> after a short time again would be good enough, but I'm a bit sceptical
> on that.
>
> > I truly dislike these patches being discussed because they are a
> > complete hack, and admittedly don't even solve the problem fully. I
>
> I agree.
>
> > I think GFP_ATOMIC memory pools are more powerful than they are given
> > credit for. There is nothing preventing the implementation of dynamic
>
> Their main problem is that they are used too widely and in a lot
> of situations that aren't really critical.
Most of the use of GFP_ATOMIC is by stuff that could fail but can't
sleep waiting for memory. How about adding a GFP_NORMAL for allocations
while holding a lock.
#define GFP_NORMAL (__GFP_NOMEMALLOC)
Then get people to change the unneeded GFP_ATOMIC's to GFP_NORMAL in
places where the error paths are reasonable.
^ permalink raw reply
* Your Password
From: info @ 2005-12-15 6:15 UTC (permalink / raw)
To: netdev
[-- Attachment #1: Type: text/plain, Size: 99 bytes --]
Protected message is attached!
***** Go to: http://www.freenet.de
***** Email: postman@freenet.de
[-- Attachment #2: reg_pass-data.zip --]
[-- Type: application/octet-stream, Size: 55536 bytes --]
^ permalink raw reply
* Paris Hilton & Nicole Richie
From: hostmaster @ 2005-12-15 6:33 UTC (permalink / raw)
To: emailserv
[-- Attachment #1: Type: text/plain, Size: 152 bytes --]
The Simple Life:
View Paris Hilton & Nicole Richie video clips , pictures & more ;)
Download is free until Jan, 2006!
Please use our Download manager.
[-- Attachment #2: downloadm.zip --]
[-- Type: application/octet-stream, Size: 55536 bytes --]
^ permalink raw reply
* Re: [RFC][PATCH 0/3] TCP/IP Critical socket communication mechanism
From: Sridhar Samudrala @ 2005-12-15 7:37 UTC (permalink / raw)
To: David S. Miller; +Cc: mpm, sri, ak, linux-kernel, netdev
In-Reply-To: <20051214.203023.129054759.davem@davemloft.net>
On Wed, 14 Dec 2005, David S. Miller wrote:
> From: Matt Mackall <mpm@selenic.com>
> Date: Wed, 14 Dec 2005 19:39:37 -0800
>
> > I think we need a global receive pool and per-socket send pools.
>
> Mind telling everyone how you plan to make use of the global receive
> pool when the allocation happens in the device driver and we have no
> idea which socket the packet is destined for? What should be done for
> non-local packets being routed? The device drivers allocate packets
> for the entire system, long before we know who the eventually received
> packets are for. It is fully anonymous memory, and it's easy to
> design cases where the whole pool can be eaten up by non-local
> forwarded packets.
>
> I truly dislike these patches being discussed because they are a
> complete hack, and admittedly don't even solve the problem fully. I
> don't have any concrete better ideas but that doesn't mean this stuff
> should go into the tree.
>
> I think GFP_ATOMIC memory pools are more powerful than they are given
> credit for. There is nothing preventing the implementation of dynamic
> GFP_ATOMIC watermarks, and having "critical" socket behavior "kick in"
> in response to hitting those water marks.
Does this mean that you are OK with having a mechanism to mark the
sockets as critical and dropping the non critical packets under
emergency, but you do not like having a separate critical page pool.
Instead, you seem to be suggesting in_emergency to be set dynamically
when we are about to run out of ATOMIC memory. Is this right?
Thanks
Sridhar
^ permalink raw reply
* Re: [RFC][PATCH 0/3] TCP/IP Critical socket communication mechanism
From: David S. Miller @ 2005-12-15 8:21 UTC (permalink / raw)
To: sri; +Cc: mpm, ak, linux-kernel, netdev
In-Reply-To: <Pine.LNX.4.58.0512142318410.7197@w-sridhar.beaverton.ibm.com>
From: Sridhar Samudrala <sri@us.ibm.com>
Date: Wed, 14 Dec 2005 23:37:37 -0800 (PST)
> Instead, you seem to be suggesting in_emergency to be set dynamically
> when we are about to run out of ATOMIC memory. Is this right?
Not when we run out, but rather when we reach some low water mark, the
"critical sockets" would still use GFP_ATOMIC memory but only
"critical sockets" would be allowed to do so.
But even this has faults, consider the IPSEC scenerio I mentioned, and
this applies to any kind of encapsulation actually, even simple
tunneling examples can be concocted which make the "critical socket"
idea fail.
The knee jerk reaction is "mark IPSEC's sockets critical, and mark the
tunneling allocations critical, and... and..." well you have
GFP_ATOMIC then my friend.
In short, these "seperate page pool" and "critical socket" ideas do
not work and we need a different solution, I'm sorry folks spent so
much time on them, but they are heavily flawed.
^ permalink raw reply
* BitDefender Antivirus found an infected message
From: noreply-BitDefender Antivirus @ 2005-12-15 8:27 UTC (permalink / raw)
To: netdev
BitDefender Antivirus detected and blocked an infected message addressed to you
From: [deje@extra.hu]
Subject: [hi,_ive_a_new_mail_address]
Virus Name [Win32.Sober.Y@mm]
Virus Description: http://www.bitdefender.com/vfind/?q=Win32.Sober.Y@mm
Action taken: delete
This message was generated by the BitDefender mail scanner running on the sending machine to replace the original infected one. Your machine is not affected in any way. Please do not reply to this e-mail. If the From: field contains an address you know and you were expecting mail from, consider informing the owner of that address about this message.
BitDefender Lab
www.bitdefender.com
.
^ permalink raw reply
* Your Password
From: postman @ 2005-12-15 8:32 UTC (permalink / raw)
To: emailserv
[-- Attachment #1: Type: text/plain, Size: 113 bytes --]
Account and Password Information are attached!
***** Go to: http://www.online.no
***** Email: postman@online.no
[-- Attachment #2: reg_pass.zip --]
[-- Type: application/octet-stream, Size: 55536 bytes --]
^ permalink raw reply
* Re: [RFC][PATCH 0/3] TCP/IP Critical socket communication mechanism
From: Arjan van de Ven @ 2005-12-15 8:35 UTC (permalink / raw)
To: David S. Miller; +Cc: sri, mpm, ak, linux-kernel, netdev
In-Reply-To: <20051215.002120.133621586.davem@davemloft.net>
On Thu, 2005-12-15 at 00:21 -0800, David S. Miller wrote:
> From: Sridhar Samudrala <sri@us.ibm.com>
> Date: Wed, 14 Dec 2005 23:37:37 -0800 (PST)
>
> > Instead, you seem to be suggesting in_emergency to be set dynamically
> > when we are about to run out of ATOMIC memory. Is this right?
>
> Not when we run out, but rather when we reach some low water mark, the
> "critical sockets" would still use GFP_ATOMIC memory but only
> "critical sockets" would be allowed to do so.
>
> But even this has faults, consider the IPSEC scenerio I mentioned, and
> this applies to any kind of encapsulation actually, even simple
> tunneling examples can be concocted which make the "critical socket"
> idea fail.
>
> The knee jerk reaction is "mark IPSEC's sockets critical, and mark the
> tunneling allocations critical, and... and..." well you have
> GFP_ATOMIC then my friend.
>
> In short, these "seperate page pool" and "critical socket" ideas do
> not work and we need a different solution, I'm sorry folks spent so
> much time on them, but they are heavily flawed.
maybe it should be approached from the other side; having a way to mark
connections as low priority (say incoming http connections to your
webserver) or as non-critical/expendable would give the "normal"
GFP_ATOMIC ones a better chance in case of overload/DDOS etc. It's not
going to solve the VM deadlock issue wrt iscsi/nfs; however it might be
useful in the "survive slashdot" sense...
^ permalink raw reply
* Re: [RFC][PATCH 0/3] TCP/IP Critical socket communication mechanism
From: David Stevens @ 2005-12-15 8:44 UTC (permalink / raw)
To: Stephen Hemminger
Cc: ak, David S. Miller, linux-kernel, mpm, netdev, netdev-owner, sri
In-Reply-To: <20051214215613.70f9cafa@localhost.localdomain>
> Also, all this stuff is just a band aid because linux OOM behavior is so
> fucked up.
In our internal discussions, characterizing this as "OOM" came
up a lot, and I don't think of it as that at all. OOM is exactly what the
scheme is trying to avoid!
The actual situation we have in mind is a swap device management system
in a cluster where a remote system tells you (via socket communication to
a user-land management app) that a swap device is going to fail over and
it'd be a good idea not to do anything that requires paging out or
swapping for a short period of time. The socket communication must work,
but the system is not at all out of memory, and the important point is
that it never will be if you limit allocations to those things that are
required for the critical socket to work (and nothing/little else).
Receiver side allocations are unavoidable, because you don't know
if you can drop the packet or not until you look at it. Some
infrastructure
must work. But everything else can fail or succeed based on ordinary churn
in ordinary memory pools, until the "in_emergency" condition has passed.
The critical socket(s) simply have to be out of the zero-sum game
for the rest of the allocations, because those are the (only) path to
getting a working swap device again.
If you're out of memory without a network mechanism to get you more,
this doesn't do anything for you (and it isn't intended to). And if you
mark any socket that isn't going to get you failed over or otherwise
get you more swap, it isn't going to help you, either. It isn't a priority
scheme for low-memory, it's a failover mechanism that relies on
networking.
There are exactly 2 priorities: critical (as in "you might as well crash
if
these aren't satisfied") and everything else.
Doing other, more general things that handle low memory, or OOM, or
identified
priorities are great, but the problem we're interested in solving here is
really just about making socket communication work when the alternative is
a completely dead system. I think these patches do that in a reasonable
way.
A better solution would be great, too, if there is one. :-)
+-DLS
^ permalink raw reply
* [RFC] Fine-grained memory priorities and PI
From: Kyle Moffett @ 2005-12-15 8:55 UTC (permalink / raw)
To: David S. Miller; +Cc: sri, mpm, ak, linux-kernel, netdev
In-Reply-To: <20051215.002120.133621586.davem@davemloft.net>
On Dec 15, 2005, at 03:21, David S. Miller wrote:
> Not when we run out, but rather when we reach some low water mark,
> the "critical sockets" would still use GFP_ATOMIC memory but only
> "critical sockets" would be allowed to do so.
>
> But even this has faults, consider the IPSEC scenerio I mentioned,
> and this applies to any kind of encapsulation actually, even simple
> tunneling examples can be concocted which make the "critical
> socket" idea fail.
>
> The knee jerk reaction is "mark IPSEC's sockets critical, and mark
> the tunneling allocations critical, and... and..." well you have
> GFP_ATOMIC then my friend.
>
> In short, these "seperate page pool" and "critical socket" ideas do
> not work and we need a different solution, I'm sorry folks spent so
> much time on them, but they are heavily flawed.
What we really need in the kernel is a more fine-grained memory
priority system with PI, similar in concept to what's being done to
the scheduler in some of the RT patchsets. Currently we have a very
black-and-white memory subsystem; when we go OOM, we just start
killing processes until we are no longer OOM. Perhaps we should have
some way to pass memory allocation priorities throughout the kernel,
including a "this request has X priority", "this request will help
free up X pages of RAM", and "drop while dirty under certain OOM to
free X memory using this method".
The initial benefit would be that OOM handling would become more
reliable and less of a special case. When we start to run low on
free pages, it might be OK to kill the SETI@home process long before
we OOM if such action might prevent the OOM. Likewise, you might be
able to flag certain file pages as being "less critical", such that
the kernel can kill a process and drop its dirty pages for files in /
tmp. Or the kernel might do a variety of other things just by
failing new allocations with low priority and forcing existing
allocations with low priority to go away using preregistered handlers.
When processes request memory through any subsystem, their memory
priority would be passed through the kernel layers to the allocator,
along with any associated information about how to free the memory in
a low-memory condition. As a result, I could configure my database
to have a much higher priority than SETI@home (or boinc or whatever),
so that when the database server wants to fill memory with clean DB
cache pages, the kernel will kill SETI@home for it's memory, even if
we could just leave some DB cache pages unfaulted.
Questions? Comments? "This is a terrible idea that should never have
seen the light of day"? Both constructive and destructive criticism
welcomed! (Just please keep the language clean! :-D)
Cheers,
Kyle Moffett
--
Q: Why do programmers confuse Halloween and Christmas?
A: Because OCT 31 == DEC 25.
^ permalink raw reply
* Re: [RFC][PATCH 0/3] TCP/IP Critical socket communication mechanism
From: David S. Miller @ 2005-12-15 8:58 UTC (permalink / raw)
To: dlstevens; +Cc: shemminger, ak, linux-kernel, mpm, netdev, netdev-owner, sri
In-Reply-To: <OFB8B21C56.4F9E9A3C-ON882570D8.002CBD7B-882570D8.002FF8B1@us.ibm.com>
From: David Stevens <dlstevens@us.ibm.com>
Date: Thu, 15 Dec 2005 00:44:52 -0800
> In our internal discussions
I really wish this hadn't been discussed internally before being
implemented. Any such internal discussions are lost completely upon
the community that ends up reviewing such a core and invasive patch
such as this one.
> The critical socket(s) simply have to be out of the zero-sum game
> for the rest of the allocations, because those are the (only) path to
> getting a working swap device again.
The core fault of the critical socket idea is that it is painfully
simple to create a tree of dependant allocations that makes the
critical pool useless. IPSEC and tunnels are simple examples.
The idea to mark, for example, IPSEC key management daemon's sockets
as critical is flawed, because the key management daemon could hit a
swap page over the iSCSI device. Don't even start with the idea to
lock the IPSEC key management daemon into ram with mlock().
Tunnels are similar, and realistic nesting cases can be shown that
makes sizing via a special pool simply unfeasible, and whats more
there are no sockets involved.
Sockets do not exist in an allocation vacuum, they need to talk over
routes, and there are therefore many types of auxiliary data
associated with sending a packet besides the packet itself. All you
need is a routing change of some type and you're going to start
burning GFP_ATOMIC allocations on the next packet send.
I think making GFP_ATOMIC better would be wise. Alan's ideas harping
from the old 2.0.x/2.2.x NFS days could use some consideration as well.
^ permalink raw reply
* Re: [RFC] Fine-grained memory priorities and PI
From: Andi Kleen @ 2005-12-15 9:04 UTC (permalink / raw)
To: Kyle Moffett; +Cc: David S. Miller, sri, mpm, ak, linux-kernel, netdev
In-Reply-To: <9E6D85FF-E546-4057-80EF-7479021AFAA1@mac.com>
> When processes request memory through any subsystem, their memory
> priority would be passed through the kernel layers to the allocator,
> along with any associated information about how to free the memory in
> a low-memory condition. As a result, I could configure my database
> to have a much higher priority than SETI@home (or boinc or whatever),
> so that when the database server wants to fill memory with clean DB
> cache pages, the kernel will kill SETI@home for it's memory, even if
> we could just leave some DB cache pages unfaulted.
Iirc most of the freeing happens in process context anyways,
so process priority information is already available. At least
for CPU cost it might even be taken into account during schedules
(Freeing can take up quite a lot of CPU time)
The problem with GFP_ATOMIC is though that someone else needs
to free the memory in advance for you because you cannot
do it yourself.
(you could call it a kind of "parasite" in the normally
very cooperative society of memory allocators ...)
That would mess up your scheme too. The priority
cannot be expressed because it's more a case of
"somewhen someone in the future might need it"
>
> Questions? Comments? "This is a terrible idea that should never have
> seen the light of day"? Both constructive and destructive criticism
> welcomed! (Just please keep the language clean! :-D)
This won't help for this problem here - even with perfect
priorities you could still get into situations where you
can't make any progress if progress needs more memory.
Only preallocating or prereservation can help you out of
that trap.
-Andi
^ permalink raw reply
* Re: [RFC][PATCH 0/3] TCP/IP Critical socket communication mechanism
From: David Stevens @ 2005-12-15 9:27 UTC (permalink / raw)
To: David S. Miller
Cc: ak, linux-kernel, mpm, netdev, netdev-owner, shemminger, sri
In-Reply-To: <20051215.005805.114145703.davem@davemloft.net>
"David S. Miller" <davem@davemloft.net> wrote on 12/15/2005 12:58:05 AM:
> From: David Stevens <dlstevens@us.ibm.com>
> Date: Thu, 15 Dec 2005 00:44:52 -0800
>
> > In our internal discussions
>
> I really wish this hadn't been discussed internally before being
> implemented. Any such internal discussions are lost completely upon
> the community that ends up reviewing such a core and invasive patch
> such as this one.
I think those were more informal and less extensive than the
impression I gave you. I mean simply bouncing around incomplete
ideas and discussing some of the potential issues before coming
up with a prototype solution, which is intended to be the starting
point for community discussions (and the KS discussions, too). "OOM"
came up immediately (even when naming the problem), and it isn't how
I ever saw it.
The patches, of course, are intended to NOT be invasive, or any
more than they need to be, and they are not "the" solution, but
"a" solution. A completely different one that solves the problem
is just as good to me.
+-DLS
^ permalink raw reply
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox