MAX limit of file descriptor

MAX limit of file descriptor

[horseriver]

hi:)

   In one process ,what is the max number of opening file descriptor ?
   Can it be set to infinite ?
  
   In network programing ,what is the essential for  the maximum of connections 
   dealed per second 

thanks!

MAX limit of file descriptor

[Peter Teoh]

i can only make a general statement, may not be always true/false:

in the kernel almost EVERYTHING HAS TO BE FINITE....and this is cater for
the fact that

but at the userspace or application level, u can design structures to be
infinite.   eg, I used python for large number calculation, and so far it
has not limits, but I am sure at the representation level, there is
one....but because i don't know the datastructure used, i don't know the
limits.

On Sat, Feb 9, 2013 at 1:10 PM, horseriver <horserivers@gmail.com> wrote:

> hi:)
>
>    In one process ,what is the max number of opening file descriptor ?
>    Can it be set to infinite ?
>
>    In network programing ,what is the essential for  the maximum of
> connections
>    dealed per second
>
> thanks!
>
> _______________________________________________
> Kernelnewbies mailing list
> Kernelnewbies at kernelnewbies.org
> http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
>



-- 
Regards,
Peter Teoh
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.kernelnewbies.org/pipermail/kernelnewbies/attachments/20130209/09307052/attachment-0001.html 

MAX limit of file descriptor

[Gaurav Jain]

I believe you can achieve the 'effect' of setting the maximum number of
open file descriptors to 'infinite' by using the following (untested code!):

<snip>
#include <sys/time.h>
#include <sys/resource.h>

struct rlimit rlim;
rlim.rlim_cur = RLIM_INFINTY;
rlim.rlim_max = RLIM_INFINITY;

int ret = setrlimit(RLIMIT_NOFILE, &rlim);
</snip>

Note that what Peter pointed out is more or less correct. You can't *actually
*have infinite file descriptors. The above code just ensures that the
kernel won't perform any checks on the number of open file descriptors. So,
if you hit a real resource limit; like no more memory left for opening
files, you would still get an error. My feeling is that this isn't a wise
thing to do. It's always better to know a precise number of the upper limit
rather than an arbitrarily large number.

~ Gaurav


On Sat, Feb 9, 2013 at 4:49 PM, Peter Teoh <htmldeveloper@gmail.com> wrote:

> i can only make a general statement, may not be always true/false:
>
> in the kernel almost EVERYTHING HAS TO BE FINITE....and this is cater for
> the fact that
>
> but at the userspace or application level, u can design structures to be
> infinite.   eg, I used python for large number calculation, and so far it
> has not limits, but I am sure at the representation level, there is
> one....but because i don't know the datastructure used, i don't know the
> limits.
>
>
> On Sat, Feb 9, 2013 at 1:10 PM, horseriver <horserivers@gmail.com> wrote:
>
>> hi:)
>>
>>    In one process ,what is the max number of opening file descriptor ?
>>    Can it be set to infinite ?
>>
>>    In network programing ,what is the essential for  the maximum of
>> connections
>>    dealed per second
>>
>> thanks!
>>
>> _______________________________________________
>> Kernelnewbies mailing list
>> Kernelnewbies at kernelnewbies.org
>> http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
>>
>
>
>
> --
> Regards,
> Peter Teoh
>
> _______________________________________________
> Kernelnewbies mailing list
> Kernelnewbies at kernelnewbies.org
> http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
>
>


-- 
Gaurav Jain
Associate Software Engineer
VxVM Escalations Team, SAMG
Symantec Software India Pvt. Ltd.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.kernelnewbies.org/pipermail/kernelnewbies/attachments/20130209/dd68f3ff/attachment.html 

MAX limit of file descriptor

[Gaurav Jain]

Reference:
http://www.kernel.org/doc/man-pages/online/pages/man2/setrlimit.2.html


On Sat, Feb 9, 2013 at 6:08 PM, Gaurav Jain <gjainroorkee@gmail.com> wrote:

> I believe you can achieve the 'effect' of setting the maximum number of
> open file descriptors to 'infinite' by using the following (untested code!):
>
> <snip>
> #include <sys/time.h>
> #include <sys/resource.h>
>
> struct rlimit rlim;
> rlim.rlim_cur = RLIM_INFINTY;
> rlim.rlim_max = RLIM_INFINITY;
>
> int ret = setrlimit(RLIMIT_NOFILE, &rlim);
> </snip>
>
> Note that what Peter pointed out is more or less correct. You can't *actually
> *have infinite file descriptors. The above code just ensures that the
> kernel won't perform any checks on the number of open file descriptors. So,
> if you hit a real resource limit; like no more memory left for opening
> files, you would still get an error. My feeling is that this isn't a wise
> thing to do. It's always better to know a precise number of the upper limit
> rather than an arbitrarily large number.
>
> ~ Gaurav
>
>
> On Sat, Feb 9, 2013 at 4:49 PM, Peter Teoh <htmldeveloper@gmail.com>wrote:
>
>> i can only make a general statement, may not be always true/false:
>>
>> in the kernel almost EVERYTHING HAS TO BE FINITE....and this is cater for
>> the fact that
>>
>> but at the userspace or application level, u can design structures to be
>> infinite.   eg, I used python for large number calculation, and so far it
>> has not limits, but I am sure at the representation level, there is
>> one....but because i don't know the datastructure used, i don't know the
>> limits.
>>
>>
>> On Sat, Feb 9, 2013 at 1:10 PM, horseriver <horserivers@gmail.com> wrote:
>>
>>> hi:)
>>>
>>>    In one process ,what is the max number of opening file descriptor ?
>>>    Can it be set to infinite ?
>>>
>>>    In network programing ,what is the essential for  the maximum of
>>> connections
>>>    dealed per second
>>>
>>> thanks!
>>>
>>> _______________________________________________
>>> Kernelnewbies mailing list
>>> Kernelnewbies at kernelnewbies.org
>>> http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
>>>
>>
>>
>>
>> --
>> Regards,
>> Peter Teoh
>>
>> _______________________________________________
>> Kernelnewbies mailing list
>> Kernelnewbies at kernelnewbies.org
>> http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
>>
>>
>
>
> --
> Gaurav Jain
> Associate Software Engineer
> VxVM Escalations Team, SAMG
> Symantec Software India Pvt. Ltd.
>
>
>


-- 
Gaurav Jain
Associate Software Engineer
VxVM Escalations Team, SAMG
Symantec Software India Pvt. Ltd.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.kernelnewbies.org/pipermail/kernelnewbies/attachments/20130209/171ddb86/attachment.html 

MAX limit of file descriptor

[horseriver]

 thanks!

 Actually , my question comes from network performance ,I want to know ,in per second ,the 
 maximum of tcp connections that can be dealed with by my server.
 
 How can I do the test and calculate the connection  number , Is it possible that my server 
 can deal with 10k tcp connections per second?

 what is the relationship between this and throughput rate?

 Is there document that tells the best optimization of this ?
 

MAX limit of file descriptor

[Mulyadi Santosa]

On Mon, Feb 11, 2013 at 5:07 AM, horseriver <horserivers@gmail.com> wrote:
>
>
>  thanks!
>
>  Actually , my question comes from network performance ,I want to know ,in per second ,the
>  maximum of tcp connections that can be dealed with by my server.

AFAIK, the only way to find out is by doing benchmark by your own. You
can use tools like netperf for that purpose.

-- 
regards,

Mulyadi Santosa
Freelance Linux trainer and consultant

blog: the-hydra.blogspot.com
training: mulyaditraining.blogspot.com

MAX limit of file descriptor

[Valdis.Kletnieks at vt.edu]

On Mon, 11 Feb 2013 06:07:38 +0800, horseriver said:

>  Actually , my question comes from network performance ,I want to know ,in per second ,the
>  maximum of tcp connections that can be dealed with by my server.

That will be *highly* dependent on what your server code does with each connection.
A "hello world" reply and close socket will, of course, go lots faster than
something that has to go contact an enterprise-scale database, do 3 SQL joins,
and format the results.

>  How can I do the test and calculate the connection  number , Is it possible that my server
>  can deal with 10k tcp connections per second?

10K/sec peaks can be achieved even on a laptop, assuming a dummy do-nothing
service.  Keeping that sustained for a real application will depend on the
service time needed - if you have 20 CPUs in the box, and spread the load
across all 20, you have to average under 2ms to service each request, which
will be a killer if you have to go to disk at all for a request.  At that
point, the guys at Foundry will be more than happy to sell you a load-balancer
so you can have a stack of 10 20-CPU servers each of which only handles 1K/sec
and thus has a 20ms time budget.

>  what is the relationship between this and throughput rate?

Lots of tiny connections will totally suck at aggregate throughput, if for no
other reason than TCP slow-start never gets a chance to really open the
transmit window up.  But in general, there is always a trade-off
between transaction rate and throughput.

>  Is there document that tells the best optimization of this ?

"best" is defined by what your application actually needs.  The "best"
settings for my NFS server will be totally different than what the HTTP
server 12 racks over needs...

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 865 bytes
Desc: not available
Url : http://lists.kernelnewbies.org/pipermail/kernelnewbies/attachments/20130211/1521c25d/attachment.bin 

MAX limit of file descriptor

[michi1 at michaelblizek.twilightparadox.com]

Hi!

On 13:10 Sat 09 Feb     , horseriver wrote:
> hi:)
> 
>    In one process ,what is the max number of opening file descriptor ?

Type "ulimit -a" in your shell. On my system (debian) the default is 1024.

>    Can it be set to infinite ?

Maybe, but at least it can be set very high.

>    In network programing ,what is the essential for  the maximum of connections 
>    dealed per second 

- Use non blocking i/o and epoll(). Do *not* create 1 process/thread for each
  connection and do not use use select(). 
- Obviously, the more memory your application uses, the more memory has to be
  put in the server. IIRC, 1 tcp connection uses ~1kb kernel memory.
- The same applies for cpu time. On the system side, you may want to recommend
  network adaptors which can be switched to polling instead of raising 1
  interrupt per packet. You should expect to see lots of small packets on the
  network.

	-Michi
-- 
programing a layer 3+4 network protocol for mesh networks
see http://michaelblizek.twilightparadox.com

MAX limit of file descriptor

[Peter Teoh]

On Sun, Feb 10, 2013 at 8:29 PM,
<michi1@michaelblizek.twilightparadox.com>wrote:

> Hi!
>
> On 13:10 Sat 09 Feb     , horseriver wrote:
> > hi:)
> >
> >    In one process ,what is the max number of opening file descriptor ?
>
> Type "ulimit -a" in your shell. On my system (debian) the default is 1024.
>

Hi Michael, nice to see u again.

BTW, many of the parameters as reported by ulimit, also has to be taken
with some doubts:

ulimit -a
core file size          (blocks, -c) unlimited
data seg size           (kbytes, -d) unlimited
scheduling priority             (-e) 0
file size               (blocks, -f) unlimited
pending signals                 (-i) 47543
max locked memory       (kbytes, -l) 64
max memory size         (kbytes, -m) unlimited
open files                      (-n) 1024
pipe size            (512 bytes, -p) 8
POSIX message queues     (bytes, -q) 819200
real-time priority              (-r) 0
stack size              (kbytes, -s) 8192
cpu time               (seconds, -t) unlimited
max user processes              (-u) 47543
virtual memory          (kbytes, -v) unlimited
file locks                      (-x) unlimited

the above is for Ubuntu 12.04 with 32-bit kernel (3.2.0) but of course we
know that max file size has a limit - depending on whether it is ext2 or
ext3 or ext4.   cannot remember the exact nos, but general conceptual
level, there is a limit.

even for "CPU time"...it is limited by the underlying bit length of
representation for time.   as usual...i don't know the details :-(, just
concept.   sorry :-(.



>
> >    Can it be set to infinite ?
>
> Maybe, but at least it can be set very high.
>
> >    In network programing ,what is the essential for  the maximum of
> connections
> >    dealed per second
>
> - Use non blocking i/o and epoll(). Do *not* create 1 process/thread for
> each
>   connection and do not use use select().
> - Obviously, the more memory your application uses, the more memory has to
> be
>   put in the server. IIRC, 1 tcp connection uses ~1kb kernel memory.
> - The same applies for cpu time. On the system side, you may want to
> recommend
>   network adaptors which can be switched to polling instead of raising 1
>   interrupt per packet. You should expect to see lots of small packets on
> the
>   network.
>
>         -Michi
> --
> programing a layer 3+4 network protocol for mesh networks
> see http://michaelblizek.twilightparadox.com
>
> _______________________________________________
> Kernelnewbies mailing list
> Kernelnewbies at kernelnewbies.org
> http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
>



-- 
Regards,
Peter Teoh
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.kernelnewbies.org/pipermail/kernelnewbies/attachments/20130210/9e731b1e/attachment.html 

MAX limit of file descriptor

[michi1 at michaelblizek.twilightparadox.com]

Hi!

On 22:47 Sun 10 Feb     , Peter Teoh wrote:
> On Sun, Feb 10, 2013 at 8:29 PM,
> <michi1@michaelblizek.twilightparadox.com>wrote:
> 
> > Hi!
> >
> > On 13:10 Sat 09 Feb     , horseriver wrote:
> > > hi:)
> > >
> > >    In one process ,what is the max number of opening file descriptor ?
> >
> > Type "ulimit -a" in your shell. On my system (debian) the default is 1024.
> >
> 
> Hi Michael, nice to see u again.
> 
> BTW, many of the parameters as reported by ulimit, also has to be taken
> with some doubts:
> 
> ulimit -a
> core file size          (blocks, -c) unlimited
> data seg size           (kbytes, -d) unlimited
> scheduling priority             (-e) 0
> file size               (blocks, -f) unlimited
> pending signals                 (-i) 47543
> max locked memory       (kbytes, -l) 64
> max memory size         (kbytes, -m) unlimited
> open files                      (-n) 1024
> pipe size            (512 bytes, -p) 8
> POSIX message queues     (bytes, -q) 819200
> real-time priority              (-r) 0
> stack size              (kbytes, -s) 8192
> cpu time               (seconds, -t) unlimited
> max user processes              (-u) 47543
> virtual memory          (kbytes, -v) unlimited
> file locks                      (-x) unlimited
> 
> the above is for Ubuntu 12.04 with 32-bit kernel (3.2.0) but of course we
> know that max file size has a limit - depending on whether it is ext2 or
> ext3 or ext4.   cannot remember the exact nos, but general conceptual
> level, there is a limit.

Well, ulimit is not really there to tell you what the system can handle. It
only tries to limit the ressources which can be used for stability/security
reasons.

Also, even if ulimit would want to tell you what the system can handle, the
interface is obviously not suited to export the file system limits when you
have more than one file system mounted.

File system limits:
http://en.wikipedia.org/wiki/Ext3
http://en.wikipedia.org/wiki/Ext4

> even for "CPU time"...it is limited by the underlying bit length of
> representation for time.   as usual...i don't know the details :-(, just
> concept.   sorry :-(.


ulimit -t 1
dd if=/dev/zero of=/dev/null

It seems cpu limit means that the process will be killed if it exceeds it.

	-Michi
-- 
programing a layer 3+4 network protocol for mesh networks
see http://michaelblizek.twilightparadox.com

MAX limit of file descriptor

[Valdis.Kletnieks at vt.edu]

On Sat, 09 Feb 2013 13:10:47 +0800, horseriver said:

>    In one process ,what is the max number of opening file descriptor ?
>    Can it be set to infinite ?
>
>    In network programing ,what is the essential for  the maximum of connections
>    dealed per second

In general, you'll find that "number of file descriptors" isn't what ends up
killing you for high-performance network programming.  What usually gets you
are things like syn floods (either intentional ddos or getting slashdotted),
because each time you do an accept() on an incoming connection you end up
using userspace resources to handle the connection.  So the *real* question
becomes "how many times per second is your box able to fork() off an httpd,
do all the processing required, and close the connection?"

A secondary gotcha is that dying TCP connections end up stuck in FIN-WAIT and
FIN-WAIT-2,

And if you're trying to drive multiple 10G interfaces at line speed, it
gets even more fun.  Fortunately, for my application (high performance disk
servers) the connections are mostly persistent, so it's "only" a problem of
getting disks to move data that fast. :)

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 865 bytes
Desc: not available
Url : http://lists.kernelnewbies.org/pipermail/kernelnewbies/attachments/20130211/3494baf2/attachment.bin 

MAX limit of file descriptor

[michi1 at michaelblizek.twilightparadox.com]

Hi!

On 14:24 Mon 11 Feb     , Valdis.Kletnieks at vt.edu wrote:
> On Sat, 09 Feb 2013 13:10:47 +0800, horseriver said:
...
> >    In network programing ,what is the essential for  the maximum of connections
> >    dealed per second
> ...
> So the *real* question
> becomes "how many times per second is your box able to fork() off an httpd,
> do all the processing required, and close the connection?"

If you fork(), than this definitely will be your bottleneck (except the
application does something even slower). Better use epoll+nonblocking i/o.

	-Michi
-- 
programing a layer 3+4 network protocol for mesh networks
see http://michaelblizek.twilightparadox.com

MAX limit of file descriptor

[Peter Teoh]

perhaps i can add more info, after doing more investigation:

a.   "ulimit" is a shell feature, it is not a command line binary.   "man
bash" and "man sh" and u can see ulimit has different feature available for
u.

b.   ulimit control all the resources defined by the processes spawn from
the current shell onwards...ie, once ulimit is change, all child processes
from that shell onwards will change.  but resources limit in another shell,
existing processes etc does not.

c.   ulimit is a userspace feature, the kernel will have all the
corresponding feature of max open files etc...but definitely it is not
unlimited like that of ulimit.

d.   to see ALL the open files u can use "lsof" and "-p" give u control to
point at which process to dig for open files.   it also list all the open
connections (TCP) for u....which is what u want.

e.   generally java applications will open many many files descriptor
concurrently:

http://www.java.net/forum/topic/glassfish/glassfish/too-many-open-files-issue

(above listed 4500, and many others java apps like IBM RSA also have many).

On Sat, Feb 9, 2013 at 1:10 PM, horseriver <horserivers@gmail.com> wrote:

> hi:)
>
>    In one process ,what is the max number of opening file descriptor ?
>    Can it be set to infinite ?
>
>    In network programing ,what is the essential for  the maximum of
> connections
>    dealed per second
>
> thanks!
>
> _______________________________________________
> Kernelnewbies mailing list
> Kernelnewbies at kernelnewbies.org
> http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
>



-- 
Regards,
Peter Teoh
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.kernelnewbies.org/pipermail/kernelnewbies/attachments/20130212/16623b99/attachment.html 

MAX limit of file descriptor

[Peter Teoh]

one more:

To modify system-wide limits:

 */etc/security/limits.conf*


On Tue, Feb 12, 2013 at 5:31 PM, Peter Teoh <htmldeveloper@gmail.com> wrote:

> perhaps i can add more info, after doing more investigation:
>
> a.   "ulimit" is a shell feature, it is not a command line binary.   "man
> bash" and "man sh" and u can see ulimit has different feature available for
> u.
>
> b.   ulimit control all the resources defined by the processes spawn from
> the current shell onwards...ie, once ulimit is change, all child processes
> from that shell onwards will change.  but resources limit in another shell,
> existing processes etc does not.
>
> c.   ulimit is a userspace feature, the kernel will have all the
> corresponding feature of max open files etc...but definitely it is not
> unlimited like that of ulimit.
>
> d.   to see ALL the open files u can use "lsof" and "-p" give u control to
> point at which process to dig for open files.   it also list all the open
> connections (TCP) for u....which is what u want.
>
> e.   generally java applications will open many many files descriptor
> concurrently:
>
>
> http://www.java.net/forum/topic/glassfish/glassfish/too-many-open-files-issue
>
> (above listed 4500, and many others java apps like IBM RSA also have many).
>
> On Sat, Feb 9, 2013 at 1:10 PM, horseriver <horserivers@gmail.com> wrote:
>
>> hi:)
>>
>>    In one process ,what is the max number of opening file descriptor ?
>>    Can it be set to infinite ?
>>
>>    In network programing ,what is the essential for  the maximum of
>> connections
>>    dealed per second
>>
>> thanks!
>>
>> _______________________________________________
>> Kernelnewbies mailing list
>> Kernelnewbies at kernelnewbies.org
>> http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
>>
>
>
>
> --
> Regards,
> Peter Teoh
>



-- 
Regards,
Peter Teoh
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.kernelnewbies.org/pipermail/kernelnewbies/attachments/20130212/536d0532/attachment-0001.html