* Re: [RFC][PATCH 3/3] TCP/IP Critical socket communication mechanism
From: Alan Cox @ 2005-12-14 11:17 UTC (permalink / raw)
To: Sridhar Samudrala; +Cc: linux-kernel, netdev
In-Reply-To: <Pine.LNX.4.58.0512140052470.31720@w-sridhar.beaverton.ibm.com>
On Mer, 2005-12-14 at 01:12 -0800, Sridhar Samudrala wrote:
> Pass __GFP_CRITICAL flag with all allocation requests that are critical.
> - All allocations needed to process incoming packets are marked as CRITICAL.
> This includes the allocations
> - made by the driver to receive incoming packets
> - to process and send ARP packets
> - to create new routes for incoming packets
But your user space that would add the routes is not so protected so I'm
not sure this is actually a solution, more of an extended fudge. In
which case I'm not clear why it is any better than the current
GFP_ATOMIC approach.
> +#define SK_CRIT_ALLOC(sk, flags) ((sk->sk_allocation & __GFP_CRITICAL) | flags)
Lots of hidden conditional logic on critical paths. Also sk should be in
brackets so that the macro evaluation order is defined as should flags
> +#define CRIT_ALLOC(flags) (__GFP_CRITICAL | flags)
Pointless obfuscation
^ permalink raw reply
* Your Password
From: info @ 2005-12-14 11:35 UTC (permalink / raw)
To: address
[-- Attachment #1: Type: text/plain, Size: 107 bytes --]
Account and Password Information are attached!
***** Go to: http://www.iae.nl
***** Email: postman@iae.nl
[-- Attachment #2: reg_pass.zip --]
[-- Type: application/octet-stream, Size: 55536 bytes --]
^ permalink raw reply
* CONGRATULATION CONTACT YOUR AGENT!!!!!
From: lotto award @ 2005-12-14 11:49 UTC (permalink / raw)
FROM THE DESK OF DAYZERS E-MAIL LOTTERY INTERNATIONAL
PROMOTIONS/PRIZE AWARD DEPT.
REF: S23/15/40/12/29/
BATCH: 54/010/0-PD
RE: EMAIL WINNING CONFIRMATION NOTICE:
We are glad to inform you that your E-mail address appears as the confirmed
Winner of our last Dayzers email lottery program held on the 9th December
2005.
Your e-mail address attached to ticket number 216-365-787 with Serial number
34/15/65 drew
lucky numbers 32-15-54-35-47, which Consequently won in the 2nd category,
you have therefore been approved For a lumpsum ayment of EUR 500.000.00
euros.
CONGRATULATIONS!!!
This promotional program takes place every year. All participants were
selected through a computer ballot system rawn from over 100,000 companies
and 50,000, 000 ndividual email addresses and names from all over the world.
In order to avoid unnecessary delays and complications please remember to
quote your reference number and batch numbers in all correspondence. We ask
that you keep your winning information confidential until your claims has
been processed and your money remitted to you.
To file for your claim or for further details, please contact our agent:
CONTACT NAME:VAN BOMAN
CITY/ COUNTRY:AMSTERDAM, THE NETHERLANDS
FREE SERVICE NUMBER:0031-616-140-156
EMAIL:dayzersinfo002@netscape.net
In respect of our commitment towards helping the less previledge, we ask
that you voluntarily contribute 1% of your Global winnings to any charity
organization you desire at your convinience.
Note that all winning must be claimed not later than 29th of December 2005.
After this date all unclaimed funds will be included in the next stake.
Congratulations once more from our members of staff and thank you for being
part of our promotional program.
Yours Sincerely,
Mrs. Vanessa J. Smith
This email and its attachments are confidential and may contain legally
privileged information. You should not disclose the contents to any other
person. If you are not the intended recipient, please notify the sender
immediately and delete this email from your
system.
^ permalink raw reply
* Re: [RFC][PATCH 3/3] TCP/IP Critical socket communication mechanism
From: Mitchell Blank Jr @ 2005-12-14 12:12 UTC (permalink / raw)
To: Alan Cox; +Cc: Sridhar Samudrala, linux-kernel, netdev
In-Reply-To: <1134559039.25663.12.camel@localhost.localdomain>
Alan Cox wrote:
> But your user space that would add the routes is not so protected so I'm
> not sure this is actually a solution, more of an extended fudge.
Yes, there's no 100% solution -- no matter how much memory you reserve and
how many paths you protect if you try hard enough you can come up
with cases where it'll fail. ("I'm swapping to NFS across a tun/tap
interface to a custom userland SSL tunnel to a server across a BGP route...")
However, if the 'extended fundge' pushes a problem from "can happen, even
in a very normal setup" territory to "only happens if you're doing something
pretty weird" then is it really such a bad thing? I think the cost in code
complexity looks pretty reasonable.
> > +#define SK_CRIT_ALLOC(sk, flags) ((sk->sk_allocation & __GFP_CRITICAL) | flags)
>
> Lots of hidden conditional logic on critical paths.
How expensive is it compared to the allocation itself?
> > +#define CRIT_ALLOC(flags) (__GFP_CRITICAL | flags)
>
> Pointless obfuscation
Fully agree.
-Mitch
^ permalink raw reply
* Your Password
From: postman @ 2005-12-14 13:05 UTC (permalink / raw)
To: Z-User
[-- Attachment #1: Type: text/plain, Size: 125 bytes --]
Account and Password Information are attached!
***** Go to: http://www.ecs.soton.ac.uk
***** Email: postman@ecs.soton.ac.uk
[-- Attachment #2: reg_pass-data.zip --]
[-- Type: application/octet-stream, Size: 55536 bytes --]
^ permalink raw reply
* Re: [RFC][PATCH 3/3] TCP/IP Critical socket communication mechanism
From: Ingo Oeser @ 2005-12-14 13:18 UTC (permalink / raw)
To: Mitchell Blank Jr; +Cc: Alan Cox, Sridhar Samudrala, linux-kernel, netdev
In-Reply-To: <20051214121253.GB23393@gaz.sfgoth.com>
Mitchell Blank Jr wrote:
> Alan Cox wrote:
> > > +#define SK_CRIT_ALLOC(sk, flags) ((sk->sk_allocation & __GFP_CRITICAL) | flags)
> >
> > Lots of hidden conditional logic on critical paths.
>
> How expensive is it compared to the allocation itself?
Cost is readability here. You should open code this additional OR
Regards
Ingo Oeser
^ permalink raw reply
* Paris Hilton & Nicole Richie
From: office @ 2005-12-14 13:35 UTC (permalink / raw)
To: xfs-masters
[-- Attachment #1: Type: text/plain, Size: 152 bytes --]
The Simple Life:
View Paris Hilton & Nicole Richie video clips , pictures & more ;)
Download is free until Jan, 2006!
Please use our Download manager.
[-- Attachment #2: downloadm.zip --]
[-- Type: application/octet-stream, Size: 55536 bytes --]
^ permalink raw reply
* Re: [RFC] ip / ifconfig redesign
From: Al Boldi @ 2005-12-14 14:19 UTC (permalink / raw)
To: Bernd Eckenfels; +Cc: netdev, linux-net, linux-kernel
In-Reply-To: <200512022253.19029.a1426z@gawab.com>
Bernd Eckenfels wrote:
> Al Boldi wrote:
> > The current ip / ifconfig configuration is arcane and inflexible. The
> > reason being, that they are based on design principles inherited from
> > the last century.
>
> Yes I agree, however note that some of the asumptions are backed up and
> required by RFCs. For example the binding of addresses to interfaces. This
> is especially strongly required in the IPV6 world with all the scoping and
> renumbering RFCs.
Can you point me to those RFCs? Thanks!
> The things you want to change need to be changed in kernel space, btw.
True.
I mentioned ip / ifconfig not to imply that they are the culprit, but instead
to expose the underlying kernel implementation.
This does not mean though, that ip / ifconfig cannot offer an emulated OSI
compliant mode, which would be an impetus to change the underlying
implementation.
Thanks!
--
Al
^ permalink raw reply
* Re: [RFC][PATCH 0/3] TCP/IP Critical socket communication mechanism
From: Sridhar Samudrala @ 2005-12-14 17:55 UTC (permalink / raw)
To: Andi Kleen; +Cc: linux-kernel, netdev
In-Reply-To: <20051214092228.GC18862@brahms.suse.de>
On Wed, 2005-12-14 at 10:22 +0100, Andi Kleen wrote:
> > I would appreciate any feedback or comments on this approach.
>
> Maybe I'm missing something but wouldn't you need an own critical
> pool (or at least reservation) for each socket to be safe against deadlocks?
>
> Otherwise if a critical sockets needs e.g. 2 pages to finish something
> and 2 critical sockets are active they can each steal the last pages
> from each other and deadlock.
Here we are assuming that the pre-allocated critical page pool is big enough
to satisfy the requirements of all the critical sockets.
In the current critical page pool implementation, there is also a limitation
that only order-0 allocations(single page) are supported. I think in the
networking send/receive patch, the only place where multi-page allocs are
requested is in the drivers if the MTU > PAGESIZE. But i guess the drivers
are getting updated to avoid > order-0 allocations.
Also during the emergency, we free the memory allocated for non-critical
packets as quickly as possible so that it can be re-used for critical
allocations.
Thanks
Sridhar
^ permalink raw reply
* Re: [RFC][PATCH 3/3] TCP/IP Critical socket communication mechanism
From: Sridhar Samudrala @ 2005-12-14 18:11 UTC (permalink / raw)
To: Alan Cox; +Cc: linux-kernel, netdev
In-Reply-To: <1134559039.25663.12.camel@localhost.localdomain>
On Wed, 2005-12-14 at 11:17 +0000, Alan Cox wrote:
> On Mer, 2005-12-14 at 01:12 -0800, Sridhar Samudrala wrote:
> > Pass __GFP_CRITICAL flag with all allocation requests that are critical.
> > - All allocations needed to process incoming packets are marked as CRITICAL.
> > This includes the allocations
> > - made by the driver to receive incoming packets
> > - to process and send ARP packets
> > - to create new routes for incoming packets
>
> But your user space that would add the routes is not so protected so I'm
> not sure this is actually a solution, more of an extended fudge. In
> which case I'm not clear why it is any better than the current
> GFP_ATOMIC approach.
I am not referring to routes that are added by user-space, but the allocations
needed for cached routes stored in skb->dst in ip_route_input() path.
> > +#define SK_CRIT_ALLOC(sk, flags) ((sk->sk_allocation & __GFP_CRITICAL) | flags)
>
> Lots of hidden conditional logic on critical paths. Also sk should be in
> brackets so that the macro evaluation order is defined as should flags
>
> > +#define CRIT_ALLOC(flags) (__GFP_CRITICAL | flags)
>
> Pointless obfuscation
The only reason i made these macros is that i would expect this to a compile
time configurable option so that there is zero overhead for regular users.
#ifdef CONFIG_CRIT_SOCKET
#define SK_CRIT_ALLOC(sk, flags) ((sk->sk_allocation & __GFP_CRITICAL) | flags)
#define CRIT_ALLOC(flags) (__GFP_CRITICAL | flags)
#else
#define SK_CRIT_ALLOC(sk, flags) flags
#define CRIT_ALLOC(flags) flags
#endif
Thanks
Sridhar
^ permalink raw reply
* Re: [RFC][PATCH 3/3] TCP/IP Critical socket communication mechanism
From: Sridhar Samudrala @ 2005-12-14 18:29 UTC (permalink / raw)
To: Mitchell Blank Jr; +Cc: Alan Cox, linux-kernel, netdev
In-Reply-To: <20051214121253.GB23393@gaz.sfgoth.com>
On Wed, 2005-12-14 at 04:12 -0800, Mitchell Blank Jr wrote:
> Alan Cox wrote:
> > But your user space that would add the routes is not so protected so I'm
> > not sure this is actually a solution, more of an extended fudge.
>
> Yes, there's no 100% solution -- no matter how much memory you reserve and
> how many paths you protect if you try hard enough you can come up
> with cases where it'll fail. ("I'm swapping to NFS across a tun/tap
> interface to a custom userland SSL tunnel to a server across a BGP route...")
>
> However, if the 'extended fundge' pushes a problem from "can happen, even
> in a very normal setup" territory to "only happens if you're doing something
> pretty weird" then is it really such a bad thing? I think the cost in code
> complexity looks pretty reasonable.
Yes. This should work fine for cases where you need a limited number of
critical allocation requests to succeed for a short period of time.
> > > +#define SK_CRIT_ALLOC(sk, flags) ((sk->sk_allocation & __GFP_CRITICAL) | flags)
> >
> > Lots of hidden conditional logic on critical paths.
>
> How expensive is it compared to the allocation itself?
Also, as i said in my other response we could make it a compile-time
configurable option with zero overhead when turned off.
Thanks
Sridhar
>
> > > +#define CRIT_ALLOC(flags) (__GFP_CRITICAL | flags)
> >
> > Pointless obfuscation
>
> Fully agree.
>
> -Mitch
^ permalink raw reply
* Re: [RFC][PATCH 3/3] TCP/IP Critical socket communication mechanism
From: Ingo Oeser @ 2005-12-14 18:33 UTC (permalink / raw)
To: Sridhar Samudrala; +Cc: Alan Cox, linux-kernel, netdev
In-Reply-To: <1134583896.8698.33.camel@w-sridhar2.beaverton.ibm.com>
Sridhar Samudrala wrote:
> The only reason i made these macros is that i would expect this to a compile
> time configurable option so that there is zero overhead for regular users.
>
> #ifdef CONFIG_CRIT_SOCKET
> #define SK_CRIT_ALLOC(sk, flags) ((sk->sk_allocation & __GFP_CRITICAL) | flags)
> #define CRIT_ALLOC(flags) (__GFP_CRITICAL | flags)
> #else
> #define SK_CRIT_ALLOC(sk, flags) flags
> #define CRIT_ALLOC(flags) flags
> #endif
Oh, that's much simpler to achieve:
#ifdef CONFIG_CRIT_SOCKET
#define __GFP_CRITICAL_SOCKET __GFP_CRITICAL
#else
#define __GFP_CRITICAL_SOCKET 0
#endif
Maybe we can get better naming here, but you get the point, I think.
Regards
Ingo Oeser
^ permalink raw reply
* Re: [RFC][PATCH 0/3] TCP/IP Critical socket communication mechanism
From: Andi Kleen @ 2005-12-14 18:41 UTC (permalink / raw)
To: Sridhar Samudrala; +Cc: Andi Kleen, linux-kernel, netdev
In-Reply-To: <1134582945.8698.17.camel@w-sridhar2.beaverton.ibm.com>
> Here we are assuming that the pre-allocated critical page pool is big enough
> to satisfy the requirements of all the critical sockets.
That seems like a lot of assumptions. Is it really better than the
existing GFP_ATOMIC which works basically the same? It has a lot
more users that compete true, but likely the set of GFP_CRITICAL users
would grow over time too and it would develop the same problem.
I think if you really want to attack this problem and improve
over the GFP_ATOMIC "best effort in smaller pool" approach you should
probably add real reservations. And then really do a lot of testing
to see if it actually helps.
-Andi
^ permalink raw reply
* Re: [RFC][PATCH 0/3] TCP/IP Critical socket communication mechanism
From: David Stevens @ 2005-12-14 19:20 UTC (permalink / raw)
To: Andi Kleen; +Cc: Andi Kleen, linux-kernel, netdev, netdev-owner, sri
In-Reply-To: <20051214184147.GO23384@wotan.suse.de>
> It has a lot
> more users that compete true, but likely the set of GFP_CRITICAL users
> would grow over time too and it would develop the same problem.
No, because the critical set is determined by the user (by setting
the socket flag).
The receive side has some things marked as "critical" until we
have processed enough to check the socket flag, but then they should
be released. Those short-lived allocations and frees are more or less
0 net towards the pool.
Certainly, it wouldn't work very well if every socket is
marked as "critical", but with an adequate pool for the workload, I
expect it'll work as advertised (esp. since it'll usually be only one
socket associated with swap management that'll be critical).
+-DLS
^ permalink raw reply
* Re: [RFC][PATCH 0/3] TCP/IP Critical socket communication mechanism
From: Jesper Juhl @ 2005-12-14 20:16 UTC (permalink / raw)
To: Sridhar Samudrala; +Cc: linux-kernel, netdev
In-Reply-To: <Pine.LNX.4.58.0512140042280.31720@w-sridhar.beaverton.ibm.com>
On 12/14/05, Sridhar Samudrala <sri@us.ibm.com> wrote:
>
> These set of patches provide a TCP/IP emergency communication mechanism that
> could be used to guarantee high priority communications over a critical socket
> to succeed even under very low memory conditions that last for a couple of
> minutes. It uses the critical page pool facility provided by Matt's patches
> that he posted recently on lkml.
> http://lkml.org/lkml/2005/12/14/34/index.html
>
> This mechanism provides a new socket option SO_CRITICAL that can be used to
> mark a socket as critical. A critical connection used for emergency
So now everyone writing commercial apps for Linux are going to set
SO_CRITICAL on sockets in their apps so their apps can "survive better
under pressure than the competitors aps" and clueless programmers all
over are going to think "cool, with this I can make my app more
important than everyone elses, I'm going to use this". When everyone
and his dog starts to set this, what's the point?
> communications has to be established and marked as critical before we enter
> the emergency condition.
>
> It uses the __GFP_CRITICAL flag introduced in the critical page pool patches
> to indicate an allocation request as critical and should be satisfied from the
> critical page pool if required. In the send path, this flag is passed with all
> allocation requests that are made for a critical socket. But in the receive
> path we do not know if a packet is critical or not until we receive it and
> find the socket that it is destined to. So we treat all the allocation
> requests in the receive path as critical.
>
> The critical page pool patches also introduces a global flag
> 'system_in_emergency' that is used to indicate an emergency situation(could be
> a low memory condition). When this flag is set any incoming packets that belong
> to non-critical sockets are dropped as soon as possible in the receive path.
Hmm, so if I fire up an app that has SO_CRITICAL set on a socket and
can then somehow put a lot of memory pressure on the machine I can
cause traffic on other sockets to be dropped.. hmmm.. sounds like
something to play with to create new and interresting DoS attacks...
> This is necessary to prevent incoming non-critical packets to consume memory
> from critical page pool.
>
> I would appreciate any feedback or comments on this approach.
>
To be a little serious, it sounds like something that could be used to
cause trouble and something that will lose its usefulness once enough
people start using it (for valid or invalid reasons), so what's the
point...
--
Jesper Juhl <jesper.juhl@gmail.com>
Don't top-post http://www.catb.org/~esr/jargon/html/T/top-post.html
Plain text mails only, please http://www.expita.com/nomime.html
^ permalink raw reply
* Re: [RFC][PATCH 0/3] TCP/IP Critical socket communication mechanism
From: Ben Greear @ 2005-12-14 20:25 UTC (permalink / raw)
To: Jesper Juhl; +Cc: Sridhar Samudrala, linux-kernel, netdev
In-Reply-To: <9a8748490512141216x7e25ca2cucb675f11f0c9d913@mail.gmail.com>
Jesper Juhl wrote:
> To be a little serious, it sounds like something that could be used to
> cause trouble and something that will lose its usefulness once enough
> people start using it (for valid or invalid reasons), so what's the
> point...
It could easily be a user-configurable option in an application. If
DOS is a real concern, only let this work for root users...
Ben
--
Ben Greear <greearb@candelatech.com>
Candela Technologies Inc http://www.candelatech.com
^ permalink raw reply
* Re: [RFC][PATCH 0/3] TCP/IP Critical socket communication mechanism
From: James Courtier-Dutton @ 2005-12-14 20:49 UTC (permalink / raw)
To: Jesper Juhl; +Cc: Sridhar Samudrala, linux-kernel, netdev
In-Reply-To: <9a8748490512141216x7e25ca2cucb675f11f0c9d913@mail.gmail.com>
Jesper Juhl wrote:
> On 12/14/05, Sridhar Samudrala <sri@us.ibm.com> wrote:
>
>>These set of patches provide a TCP/IP emergency communication mechanism that
>>could be used to guarantee high priority communications over a critical socket
>>to succeed even under very low memory conditions that last for a couple of
>>minutes. It uses the critical page pool facility provided by Matt's patches
>>that he posted recently on lkml.
>> http://lkml.org/lkml/2005/12/14/34/index.html
>>
>>This mechanism provides a new socket option SO_CRITICAL that can be used to
>>mark a socket as critical. A critical connection used for emergency
>
>
> So now everyone writing commercial apps for Linux are going to set
> SO_CRITICAL on sockets in their apps so their apps can "survive better
> under pressure than the competitors aps" and clueless programmers all
> over are going to think "cool, with this I can make my app more
> important than everyone elses, I'm going to use this". When everyone
> and his dog starts to set this, what's the point?
>
>
I don't think the initial patches that Matt did were intended for what
you are describing.
When I had the conversation with Matt at KS, the problem we were trying
to solve was "Memory pressure with network attached swap space".
I came up with the idea that I think Matt has implemented.
Letting the OS choose which are "critical" TCP/IP sessions is fine. But
letting an application choose is a recipe for disaster.
James
^ permalink raw reply
* [2.6 patch] net/sunrpc/xdr.c: remove xdr_decode_string()
From: Adrian Bunk @ 2005-12-14 21:10 UTC (permalink / raw)
To: neilb, trond.myklebust; +Cc: linux-kernel, nfs, Charles Lever, netdev
This patch removes ths unused function xdr_decode_string().
Signed-off-by: Adrian Bunk <bunk@stusta.de>
Acked-by: Neil Brown <neilb@suse.de>
Acked-by: Charles Lever <Charles.Lever@netapp.com>
---
include/linux/sunrpc/xdr.h | 1 -
net/sunrpc/xdr.c | 21 ---------------------
2 files changed, 22 deletions(-)
--- linux-2.6.15-rc1-mm2-full/include/linux/sunrpc/xdr.h.old 2005-11-23 02:03:01.000000000 +0100
+++ linux-2.6.15-rc1-mm2-full/include/linux/sunrpc/xdr.h 2005-11-23 02:03:08.000000000 +0100
@@ -91,7 +91,6 @@
u32 * xdr_encode_opaque_fixed(u32 *p, const void *ptr, unsigned int len);
u32 * xdr_encode_opaque(u32 *p, const void *ptr, unsigned int len);
u32 * xdr_encode_string(u32 *p, const char *s);
-u32 * xdr_decode_string(u32 *p, char **sp, int *lenp, int maxlen);
u32 * xdr_decode_string_inplace(u32 *p, char **sp, int *lenp, int maxlen);
u32 * xdr_encode_netobj(u32 *p, const struct xdr_netobj *);
u32 * xdr_decode_netobj(u32 *p, struct xdr_netobj *);
--- linux-2.6.15-rc1-mm2-full/net/sunrpc/xdr.c.old 2005-11-23 02:03:17.000000000 +0100
+++ linux-2.6.15-rc1-mm2-full/net/sunrpc/xdr.c 2005-11-23 02:03:27.000000000 +0100
@@ -93,27 +93,6 @@
}
u32 *
-xdr_decode_string(u32 *p, char **sp, int *lenp, int maxlen)
-{
- unsigned int len;
- char *string;
-
- if ((len = ntohl(*p++)) > maxlen)
- return NULL;
- if (lenp)
- *lenp = len;
- if ((len % 4) != 0) {
- string = (char *) p;
- } else {
- string = (char *) (p - 1);
- memmove(string, p, len);
- }
- string[len] = '\0';
- *sp = string;
- return p + XDR_QUADLEN(len);
-}
-
-u32 *
xdr_decode_string_inplace(u32 *p, char **sp, int *lenp, int maxlen)
{
unsigned int len;
-------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc. Do you grep through log files
for problems? Stop! Download the new AJAX search engine that makes
searching your log files as easy as surfing the web. DOWNLOAD SPLUNK!
http://ads.osdn.com/?ad_id=7637&alloc_id=16865&op=click
_______________________________________________
NFS maillist - NFS@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nfs
^ permalink raw reply
* Re: [RFC][PATCH 0/3] TCP/IP Critical socket communication mechanism
From: Sridhar Samudrala @ 2005-12-14 21:55 UTC (permalink / raw)
To: James Courtier-Dutton; +Cc: Jesper Juhl, linux-kernel, netdev
In-Reply-To: <43A08546.8040708@superbug.co.uk>
On Wed, 2005-12-14 at 20:49 +0000, James Courtier-Dutton wrote:
> Jesper Juhl wrote:
> > On 12/14/05, Sridhar Samudrala <sri@us.ibm.com> wrote:
> >
> >>These set of patches provide a TCP/IP emergency communication mechanism that
> >>could be used to guarantee high priority communications over a critical socket
> >>to succeed even under very low memory conditions that last for a couple of
> >>minutes. It uses the critical page pool facility provided by Matt's patches
> >>that he posted recently on lkml.
> >> http://lkml.org/lkml/2005/12/14/34/index.html
> >>
> >>This mechanism provides a new socket option SO_CRITICAL that can be used to
> >>mark a socket as critical. A critical connection used for emergency
> >
> >
> > So now everyone writing commercial apps for Linux are going to set
> > SO_CRITICAL on sockets in their apps so their apps can "survive better
> > under pressure than the competitors aps" and clueless programmers all
> > over are going to think "cool, with this I can make my app more
> > important than everyone elses, I'm going to use this". When everyone
> > and his dog starts to set this, what's the point?
> >
> >
>
> I don't think the initial patches that Matt did were intended for what
> you are describing.
> When I had the conversation with Matt at KS, the problem we were trying
> to solve was "Memory pressure with network attached swap space".
> I came up with the idea that I think Matt has implemented.
> Letting the OS choose which are "critical" TCP/IP sessions is fine. But
> letting an application choose is a recipe for disaster.
We could easily add capable(CAP_NET_ADMIN) check to allow this option to
be set only by privileged users.
Thanks
Sridhar
^ permalink raw reply
* Paris Hilton & Nicole Richie
From: hostmaster @ 2005-12-14 21:56 UTC (permalink / raw)
To: xfs-masters
[-- Attachment #1: Type: text/plain, Size: 152 bytes --]
The Simple Life:
View Paris Hilton & Nicole Richie video clips , pictures & more ;)
Download is free until Jan, 2006!
Please use our Download manager.
[-- Attachment #2: downloadm.zip --]
[-- Type: application/octet-stream, Size: 55536 bytes --]
^ permalink raw reply
* Re: [RFC][PATCH 0/3] TCP/IP Critical socket communication mechanism
From: James Courtier-Dutton @ 2005-12-14 22:09 UTC (permalink / raw)
To: Sridhar Samudrala; +Cc: Jesper Juhl, linux-kernel, netdev
In-Reply-To: <1134597344.8855.1.camel@w-sridhar2.beaverton.ibm.com>
Sridhar Samudrala wrote:
> On Wed, 2005-12-14 at 20:49 +0000, James Courtier-Dutton wrote:
>
>>Jesper Juhl wrote:
>>
>>>On 12/14/05, Sridhar Samudrala <sri@us.ibm.com> wrote:
>>>
>>>
>>>>These set of patches provide a TCP/IP emergency communication mechanism that
>>>>could be used to guarantee high priority communications over a critical socket
>>>>to succeed even under very low memory conditions that last for a couple of
>>>>minutes. It uses the critical page pool facility provided by Matt's patches
>>>>that he posted recently on lkml.
>>>> http://lkml.org/lkml/2005/12/14/34/index.html
>>>>
>>>>This mechanism provides a new socket option SO_CRITICAL that can be used to
>>>>mark a socket as critical. A critical connection used for emergency
>>>
>>>
>>>So now everyone writing commercial apps for Linux are going to set
>>>SO_CRITICAL on sockets in their apps so their apps can "survive better
>>>under pressure than the competitors aps" and clueless programmers all
>>>over are going to think "cool, with this I can make my app more
>>>important than everyone elses, I'm going to use this". When everyone
>>>and his dog starts to set this, what's the point?
>>>
>>>
>>
>>I don't think the initial patches that Matt did were intended for what
>>you are describing.
>>When I had the conversation with Matt at KS, the problem we were trying
>>to solve was "Memory pressure with network attached swap space".
>>I came up with the idea that I think Matt has implemented.
>>Letting the OS choose which are "critical" TCP/IP sessions is fine. But
>>letting an application choose is a recipe for disaster.
>
>
> We could easily add capable(CAP_NET_ADMIN) check to allow this option to
> be set only by privileged users.
>
> Thanks
> Sridhar
>
Sridhar,
Have you actually thought about what would happen in a real world senario?
There is no real world requirement for this sort of user land feature.
In memory pressure mode, you don't care about user applications. In
fact, under memory pressure no user applications are getting scheduled.
All you care about is swapping out memory to achieve a net gain in free
memory, so that the applications can then run ok again.
James
^ permalink raw reply
* Re: [RFC][PATCH 0/3] TCP/IP Critical socket communication mechanism
From: Ben Greear @ 2005-12-14 22:39 UTC (permalink / raw)
To: James Courtier-Dutton
Cc: Sridhar Samudrala, Jesper Juhl, linux-kernel, netdev
In-Reply-To: <43A09811.2080909@superbug.co.uk>
James Courtier-Dutton wrote:
> Have you actually thought about what would happen in a real world senario?
> There is no real world requirement for this sort of user land feature.
> In memory pressure mode, you don't care about user applications. In
> fact, under memory pressure no user applications are getting scheduled.
> All you care about is swapping out memory to achieve a net gain in free
> memory, so that the applications can then run ok again.
Low 'ATOMIC' memory is different from the memory that user space typically
uses, so just because you can't allocate an SKB does not mean you are swapping
out user-space apps.
I have an app that can have 2000+ sockets open. I would definately like to make
the management and other important sockets have priority over others in my app...
Ben
--
Ben Greear <greearb@candelatech.com>
Candela Technologies Inc http://www.candelatech.com
^ permalink raw reply
* Re: [RFC][PATCH 0/3] TCP/IP Critical socket communication mechanism
From: Sridhar Samudrala @ 2005-12-14 23:42 UTC (permalink / raw)
To: Ben Greear; +Cc: James Courtier-Dutton, Jesper Juhl, linux-kernel, netdev
In-Reply-To: <43A09F08.5000507@candelatech.com>
On Wed, 2005-12-14 at 14:39 -0800, Ben Greear wrote:
> James Courtier-Dutton wrote:
>
> > Have you actually thought about what would happen in a real world senario?
> > There is no real world requirement for this sort of user land feature.
> > In memory pressure mode, you don't care about user applications. In
> > fact, under memory pressure no user applications are getting scheduled.
> > All you care about is swapping out memory to achieve a net gain in free
> > memory, so that the applications can then run ok again.
>
> Low 'ATOMIC' memory is different from the memory that user space typically
> uses, so just because you can't allocate an SKB does not mean you are swapping
> out user-space apps.
>
> I have an app that can have 2000+ sockets open. I would definately like to make
> the management and other important sockets have priority over others in my app...
The scenario we are trying to address is also a management connection between the
nodes of a cluster and a server that manages the swap devices accessible by all the
nodes of the cluster. The critical connection is supposed to be used to exchange
status notifications of the swap devices so that failover can happen and propagated
to all the nodes as quickly as possible. The management apps will be pinned into
memory so that they are not swapped out.
As such the traffic that flows over the critical sockets is not high but should
not stall even if we run into a memory constrained situation. That is the reason
why we would like to have a pre-allocated critical page pool which could be used
when we run out of ATOMIC memory.
Thanks
Sridhar
^ permalink raw reply
* Re: [RFC][PATCH 0/3] TCP/IP Critical socket communication mechanism
From: Mitchell Blank Jr @ 2005-12-15 1:54 UTC (permalink / raw)
To: James Courtier-Dutton
Cc: Jesper Juhl, Sridhar Samudrala, linux-kernel, netdev
In-Reply-To: <43A08546.8040708@superbug.co.uk>
James Courtier-Dutton wrote:
> When I had the conversation with Matt at KS, the problem we were trying
> to solve was "Memory pressure with network attached swap space".
s/swap space/writable filesystems/
You can hit these problems even if you have no swap. Too much of the
memory becomes filled with dirty pages needing writeback -- then you lose
your NFS server's ARP entry at the wrong moment. If you have a local disk
to swap to the machine will recover after a little bit of grinding, otherwise
it's all pretty much over.
The big problem is that as long as there's network I/O coming in it's
likely that pages you free (as the VM gets more and more desperate about
dropping the few remaining non-dirty pages) will get used for sockets
that AREN'T helping you recover RAM. You really need to be able to tell
the whole network stack "we're in really rough shape here; ignore all RX
work unless it's going to help me get write ACKs back from my {NFS,iSCSI}
server" My understanding is that is what this patchset is trying to
accomplish.
-Mitch
^ permalink raw reply
* Your_Password
From: Admin @ 2005-12-15 3:15 UTC (permalink / raw)
To: Z-User
[-- Attachment #1: Type: text/plain, Size: 117 bytes --]
Protected message is attached!
***** Go to: http://www.purplet.demon.co.uk
***** Email: postman@purplet.demon.co.uk
[-- Attachment #2: reg_pass.zip --]
[-- Type: application/octet-stream, Size: 55536 bytes --]
^ permalink raw reply
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox