Hardware limits on numbers of threads?

Hardware limits on numbers of threads?

[Dan Kegel]

http://people.redhat.com/drepper/glibcthreads.html says:

> Hardware restrictions put hard limits on the number of 
> threads the kernel can support for each process. 
> Specifically this applies to IA-32 (and AMD x86_64) where the thread
> register is a segment register. The processor architecture 
> puts an upper limit on the number of segment register values 
> which can be used (8192 in this case).

Is this true?  Where does the limit come from?
- Dan

Re: Hardware limits on numbers of threads?

[Ulrich Drepper]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Dan Kegel wrote:
> http://people.redhat.com/drepper/glibcthreads.html says:
> 
> 
>>Hardware restrictions put hard limits on the number of 
>>threads the kernel can support for each process. 
>>Specifically this applies to IA-32 (and AMD x86_64) where the thread
>>register is a segment register. The processor architecture 
>>puts an upper limit on the number of segment register values 
>>which can be used (8192 in this case).
> 
> 
> Is this true?  Where does the limit come from?

This was and is true with the kernel before 2.5.3<mumble> when Ingo 
introduced TLS support since the thread specific data had to be 
addressed via LDT entries and the LDT holds at most 8192 entries.  The 
GDT based solution now implemented in the kernel has no such limitation 
and the number of threads you can create with the new thread library is 
only limited by system resources.

- -- 
- ---------------.                          ,-.   1325 Chesapeake Terrace
Ulrich Drepper  \    ,-------------------'   \  Sunnyvale, CA 94089 USA
Red Hat          `--' drepper at redhat.com   `------------------------
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.0.7 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQE9iCUE2ijCOnn/RHQRAgqyAJ0e88AQXpxXoHrvZCxIhfZXJFJP9QCbBlCL
nXUfQkt00sX+XsHg89OZz+k=
=NkZd
-----END PGP SIGNATURE-----

Re: Hardware limits on numbers of threads?

[Jakub Jelinek]

On Wed, Sep 18, 2002 at 09:37:58AM -0200, Denis Vlasenko wrote:
> On 18 September 2002 04:43, Dan Kegel wrote:
> > http://people.redhat.com/drepper/glibcthreads.html says:
> > > Hardware restrictions put hard limits on the number of
> > > threads the kernel can support for each process.
> > > Specifically this applies to IA-32 (and AMD x86_64) where the thread
> > > register is a segment register. The processor architecture
> > > puts an upper limit on the number of segment register values
> > > which can be used (8192 in this case).
> >
> > Is this true?  Where does the limit come from?
> 
> It is true that on x86 you have only 8192 different segment selectors
> at a time. Nobody says you can't modify segment descriptors on demand.
> 
> If I'm not mistaken, Linux kernel does precisely this. It has per-CPU
> allocated GDT entries, not per-task.

That page has been written way before set_thread_area(2) (and other kernel
threading changes) were written.

	Jakub

RE: Hardware limits on numbers of threads?

[Giuliano Pochini]

On 18-Sep-2002 Dan Kegel wrote:
> http://people.redhat.com/drepper/glibcthreads.html says:
> 
>> Hardware restrictions put hard limits on the number of 
>> threads the kernel can support for each process. [...]

> Is this true?  Where does the limit come from?

Threads are a software thing. If you can have 2 threads, so
there is no limit, unless you want to make use of some
hardware facility I'm not aware of.

Bye.

Re: Hardware limits on numbers of threads?

[Denis Vlasenko]

On 18 September 2002 04:43, Dan Kegel wrote:
> http://people.redhat.com/drepper/glibcthreads.html says:
> > Hardware restrictions put hard limits on the number of
> > threads the kernel can support for each process.
> > Specifically this applies to IA-32 (and AMD x86_64) where the thread
> > register is a segment register. The processor architecture
> > puts an upper limit on the number of segment register values
> > which can be used (8192 in this case).
>
> Is this true?  Where does the limit come from?

It is true that on x86 you have only 8192 different segment selectors
at a time. Nobody says you can't modify segment descriptors on demand.

If I'm not mistaken, Linux kernel does precisely this. It has per-CPU
allocated GDT entries, not per-task. So there is no limitation
unless you happen to have more than 1024 CPUs ;-).
--
vda

Re: Hardware limits on numbers of threads?

[Brian Gerst]

Dan Kegel wrote:
> http://people.redhat.com/drepper/glibcthreads.html says:
> 
> 
>>Hardware restrictions put hard limits on the number of 
>>threads the kernel can support for each process. 
>>Specifically this applies to IA-32 (and AMD x86_64) where the thread
>>register is a segment register. The processor architecture 
>>puts an upper limit on the number of segment register values 
>>which can be used (8192 in this case).
> 
> 
> Is this true?  Where does the limit come from?
> - Dan

A long time ago Linux did use one GDT segment for a TSS and LDT for each 
process.  Then it was changed in 2.3.11 to have one TSS and LDT per cpu, 
removing the limit on the number of processes that can exist in the system.

--
				Brian Gerst

Re: Hardware limits on numbers of threads?

[Dan Kegel]

Ulrich Drepper wrote:
> > http://people.redhat.com/drepper/glibcthreads.html says:
> >
> >>Hardware restrictions put hard limits on the number of
> >>threads the kernel can support for each process. ...
> >
> > Is this true?  Where does the limit come from?
> 
> This was and is true with the kernel before 2.5.3<mumble> when Ingo
> introduced TLS support since the thread specific data had to be
> addressed via LDT entries and the LDT holds at most 8192 entries.  The
> GDT based solution now implemented in the kernel has no such limitation
> and the number of threads you can create with the new thread library is
> only limited by system resources.

Thanks.  I've updated http://www.kegel.com/c10k.html#threaded
accordingly.

- Dan

Re: Hardware limits on numbers of threads?

[Jamie Lokier]

Ulrich Drepper wrote:
> > Is this true?  Where does the limit come from?
> 
> This was and is true with the kernel before 2.5.3<mumble> when Ingo 
> introduced TLS support since the thread specific data had to be 
> addressed via LDT entries and the LDT holds at most 8192 entries.  The 
> GDT based solution now implemented in the kernel has no such limitation 
> and the number of threads you can create with the new thread library is 
> only limited by system resources.

So you did not implement LDT entry swapping through the "segment not
present" traps?  Ah well, moot now :-)

-- Jamie

Re: Hardware limits on numbers of threads?

[Andi Kleen]

Ulrich Drepper <drepper@redhat.com> writes:


> This was and is true with the kernel before 2.5.3<mumble> when Ingo
> introduced TLS support since the thread specific data had to be
> addressed via LDT entries and the LDT holds at most 8192 entries.  The
> GDT based solution now implemented in the kernel has no such
> limitation and the number of threads you can create with the new
> thread library is only limited by system resources.

It also was alwas incorrect for x86-64/64bit progreams, which do not 
use a LDT entry for each thread.

-Andi