Re: [PATCH] 2.5.21 Nonlinear CPU support

[Andreas Dilger <adilger@clusterfs.com>]

On Jun 12, 2002  19:34 +0100, Anton Altaparmakov wrote:
> At 18:36 12/06/02, Andreas Dilger wrote:
> >1) Allocate an array of NULL pointers which is NR_CPUs in size (you could 
> >   do this all the time, as it would only be a few bytes)
> 
> Yes, that is fine.
> 
> >2) If you need to do decompression on a cpu you check the array entry
> >   for that CPU and if is NULL you vmalloc() the decompression buffers once
> >   for that CPU.  This avoid vmalloc() overhead for each read.
> 
> The vmalloc() sleeps and by the time you get control back you are executing 
> on a different CPU. Ooops. The only valid way of treating per-cpu data is:
> 
> - disable preemption
> - get the cpu number = START OF CRITICAL SECTION: no sleep/schedule allowed
> - do work using the cpu number
> - reenable preemption = END OF CRITICAL SECTION
> 
> The only thing that could possibly be used inside the critical region is 
> kmalloc(GFP_ATOMIC) but we are allocating 64kiB so that is not an option. 
> (It would fail very quickly due to memory fragmentation, the order of the 
> allocation is too high.)

Well, then you can still do the one-time allocation for that CPU slot,
and re-check the CPU number after vmalloc() returns.  If it is different
(or always, for that matter) then you jump back to the "is the array for
this CPU allocated" check until the array _is_ allocated for that CPU
and you don't need to allocate it (so you won't sleep).  At most you
will need to loop once for each available CPU if you are unlucky enough
to be rescheduled to a different CPU after each call to vmalloc().

Like:
	int cpunum = this_cpu();
	char *newbuf = NULL;

	while (unlikely(NTFS_SB(sb)->s_compr_array[cpunum] == NULL)) {
		newbuf = vmalloc(NTFS_DECOMPR_BUFFER_SIZE);

		/* Re-check the buffer case we slept in vmalloc() and
		 * someone else already allocated a buffer for "this" CPU.
		 */
		if (likely(NTFS_SB(sb)->s_compr_array[cpunum] == NULL)) {
			NTFS_SB(sb)->s_compr_array[cpunum] = newbuf;
			newbuf = NULL;
		}
		cpunum = this_cpu();
	}
	/* Hmm, we slept in vmalloc and we don't need the new buffer */
	if (unlikely(newbuf != NULL))
		vfree(newbuf);

> >3) Any allocated buffers are freed in the same manner they are now -
> >   when the last compressed volume is unmounted.  There may be some or
> >   all entries that are still NULL.
> >
> >This also avoids allocating buffers when there are no files which are
> >actually compressed.
> 
> True it does, but unfortunately it doesn't work. )-:

Now it does... ;-).

Cheers, Andreas
--
Andreas Dilger
http://www-mddsp.enel.ucalgary.ca/People/adilger/
http://sourceforge.net/projects/ext2resize/