Re: [PATCH RFC] sched: add notifier for process migration

[Peter Zijlstra <peterz@infradead.org>]

On Fri, 2009-10-09 at 15:43 -0700, Jeremy Fitzhardinge wrote:

> OK, concretely:
> 
>    1. allocate a page and fixmap it into userspace
>    2. keep an array of structures containing tsc->cycle_t
>       (pvclock_vcpu_time_info) params, indexed by cpu
>    3. register those structures with the hypervisor so it can update
>       them as either the pcpus change freq and/or the vcpus get moved to
>       different pcpus
>    4. associate a "migration_count" with each structure (ie, how many
>       times this cpu has had tasks migrated off it)
> 
> The algorithm is basically:
> 
>     do {
>         cpu = vgetcpu();    	/* get current cpu */
>         ti = &timeinfo[cpu];	/* get scaling+offset for tsc */
> 
>         /* !!! migration race */
> 
>         migration_count = ti->migration_count;
>         version = ti->version;
> 
>         barrier();
> 
>         local_time_info = *ti;
> 
>         tsc = rdtsc();
>         cycles = compute_cycles_from_tsc(tsc, &local_time_info);
> 
>         barrier();
> 
>         cpu1 = vgetcpu();
> 
>     /* loop if anything changed under our feet:
>         - we changed cpus (if we got migrated at "!!! migration race" above
>            then the migration_count test won't pick it up)
>         - the time info changed
>         - we got migrated to a different cpu (we need to check this as well
>            as cpu != cpu1 in case we got migrated from A->B->A)
>      */
> 
>     } while(unlikely(cpu1 != cpu ||
>     		 timeinfo->version != version ||
>     		 timeinfo->migration_count != migration_count));
> 
>     return cycles;
>       
> 
> This is executed in usermode as part of vsyscall gettimeofday via the
> clocksource.vread function.

Why not do something like:

    struct {
	u64 tsc;
	u32 aux;
    } tscp = rdtscp();

    local_time_info = timeinfo[tscp_cpu(tscp)];

    /* yay, consistent tsc and timeinfo !! */

?