Re: [PATCH v6] perf: Sharing PMU counters across compatible events

Song Liu
2019-11-06T17:40:46Z

> On Nov 6, 2019, at 1:14 AM, Peter Zijlstra <peterz@infradead.org> wrote:
>
> On Tue, Nov 05, 2019 at 11:06:06PM +0000, Song Liu wrote:
>>
>>
>>> On Nov 5, 2019, at 12:16 PM, Peter Zijlstra <peterz@infradead.org> wrote:
>>>
>>> On Tue, Nov 05, 2019 at 05:11:08PM +0000, Song Liu wrote:
>>>
>>>>> I think we can use one of the event as master. We need to be careful when
>>>>> the master event is removed, but it should be doable. Let me try.
>>>>
>>>> Actually, there is a bigger issue when we use one event as the master: what
>>>> shall we do if the master event is not running? Say it is an cgroup event,
>>>> and the cgroup is not running on this cpu. An extra master (and all these
>>>> array hacks) help us get O(1) complexity in such scenario.
>>>>
>>>> Do you have suggestions on how to solve this problem? Maybe we can keep the
>>>> extra master, and try get rid of the double alloc?
>>>
>>> Right, you have to consider scope when sharing. The master should be the
>>> largest scope event and any slaves should be complete subsets.
>>>
>>> Without much thought this seems a fairly straight forward constraint;
>>> that is, given cgroups I'm not immediately seeing how we can violate
>>> that.
>>>
>>> Basically, pick the cgroup event nearest to the root as the master.
>>> We have to have logic to re-elect the master anyway for deletion, so
>>> changing it on add shouldn't be different.
>>>
>>> (obviously the root-cgroup is cpu-wide and always on, and if you have
>>> two events from disjoint subtrees they have no overlap, so it doesn't
>>> make sense to share anyway)
>>
>> Hmm... I didn't think about cgroup structure with this much detail. And
>> this is very interesting idea.
>>
>> OTOH, non-cgroup event could also be inactive. For example, when we have
>> to rotate events, we may schedule slave before master.
>
> Right, although I suppose in that case you can do what you did in your
> patch here. If someone did IOC_DISABLE on the master, we'd have to
> re-elect a master -- obviously (and IOC_ENABLE).

Re-elect master on IOC_DISABLE is good. But we still need work for ctx
rotation. Otherwise, we need keep the master on at all time.

>
>> And if the master is in an event group, it will be more complicated...
>
> Hurmph, do you actually have that use-case? And yes, this one is tricky.
>
> Would it be sufficient if we disallow group events to be master (but
> allow them to be slaves) ?

Maybe we can solve this with an extra "first_active" pointer in perf_event.
first_active points to the first event that being added by event_pmu_add().
Then we need something like:

event_pmu_add(event)
{
if (event->dup_master->first_active) {
/* sync with first_active */
} else {
/* this event will be the first_active */
event->dup_master->first_active = event;
pmu->add(event);
}
}

However, I just realized the event_pmu_del() path need some more thoughts,
because first_active is likely the first one get sched_out().

Merging another email here:

>> If we do GFP_ATOMIC in perf_event_alloc(), maybe with an extra option, we
>> don't need the tmp_master hack. So we only allocate master when we will
>> use it.
>
> You can't, that's broken on -RT. ctx->lock is a raw_spinlock_t and
> allocator locks are spinlock_t.

How about we add another step in __perf_install_in_context(), like

__perf_install_in_context()
{
bool alloc_master;

perf_ctx_lock();
alloc_master = find_new_sharing(event, ctx);
perf_ctx_unlock();

if (alloc_master)
event->dup_master = perf_event_alloc();

/* existing logic of __perf_install_in_context() */

}

In this way, we only allocate the master event when necessary, and it
is outside of the locks.

Thanks,
Song