Re: Better value for chunk_size when threaded

[Nicolas Pitre <nico@cam.org>]

On Thu, 6 Dec 2007, Jon Smirl wrote:

> I tried some various ideas out for chunk_size and the best strategy I
> found was to simply set it to a constant. How does 20,000 work on
> other CPUs?

That depends on the object size.  If you have a repo with big objects 
but only 1000 of them for example, then the constant doesn't work.

Ideally I'd opt for a value that tend towards around 5 seconds worth of 
work per segment, or something like that.  Maybe using the actual 
objects size could be another way.

> I'd turn on default threaded support with this change. With threads=1
> versus non-threaded there is no appreciable difference in the time.

Would need a way to determine pthreads availability from Makefile.

> Is there an API to ask how many CPUs are in the system? It would be
> nice to default the number of threads equal to the number of CPUs and
> only use pack.threads=X to override.

If there is one besides futzing with /proc/cpuinfo I'd like to know 
about it.  Bonus points if it is portable.


Nicolas