Use of Indirect function calls

Use of Indirect function calls

[Rao Shoaib]

I am working on a change which introduces a couple of indirect function 
calls in the fast path. I used indirect function calls instead of 
"if/else" as it keeps the code cleaner and more readable and provides 
for extensibility.

I do not expect any measurable overhead as modern CPU's use pre-fetching 
and multiple parallel execution engines. Indirect jump prediction has 
gotten very good too. Assembly shows that the difference is an 
instruction to get the table address followed by a call to *callq 
instead of callq. I could not find official Intel numbers[1] but third 
party numbers suggest that *callq may have an over head of 1 clock 
cycle. Over head of accessing the table address is similar to reading a 
variable or testing a flag.

Will a patch that introduces indirect function calls be rejected just 
because it uses indirect function calls in the fast path or is the 
use/impact evaluated on a case by case basis ? I do see usage of 
indirect function calls in TCP fast path and Ethernet drivers so it 
can't be that bad.

Regards,

Shoaib

[1] I believe this is because the instruction is broken down into 
multiple uops and due to perfecting and parallel execution the latency 
is hard to measure.

Re: Use of Indirect function calls

[David Miller]

From: Rao Shoaib <rao.shoaib@oracle.com>
Date: Tue, 6 Mar 2018 19:35:46 -0800

> I do not expect any measurable overhead as modern CPU's use
> pre-fetching and multiple parallel execution engines.

Please see Spectre and retpolines, all of this parallel execution and
prefetching is essentially disabled to address those vulnerabilities
and side-channel exploits.

Indirect calls are terrible and we are now looking at ways in which
we can remove them from as many parts of the networking as possible.

Re: Use of Indirect function calls

[Rao Shoaib]

David,

Thanks a lot for your prompt response. Do you have a specific solution 
in mind or will the calls be replaced with simple checks ?

Also while I have your attention can I ask your opinion about breaking 
up some TCP functions, mostly control functions into smaller units so 
that if a little different behavior is desired it can be achieved and 
common code could still be shared. Of course you can not say much 
without looking at the code but will you even entertain such a change ?

Regards,

Rao.


On 03/06/2018 08:43 PM, David Miller wrote:
> From: Rao Shoaib <rao.shoaib@oracle.com>
> Date: Tue, 6 Mar 2018 19:35:46 -0800
>
>> I do not expect any measurable overhead as modern CPU's use
>> pre-fetching and multiple parallel execution engines.
> Please see Spectre and retpolines, all of this parallel execution and
> prefetching is essentially disabled to address those vulnerabilities
> and side-channel exploits.
>
> Indirect calls are terrible and we are now looking at ways in which
> we can remove them from as many parts of the networking as possible.

Re: Use of Indirect function calls

[Eric Dumazet]

On Tue, 2018-03-06 at 21:53 -0800, Rao Shoaib wrote:
> David,
> 
> Thanks a lot for your prompt response. Do you have a specific
> solution 
> in mind or will the calls be replaced with simple checks ?

There is upcoming work for that, but not specific to TCP stack.

> 
> Also while I have your attention can I ask your opinion about
> breaking 
> up some TCP functions, mostly control functions into smaller units
> so 
> that if a little different behavior is desired it can be achieved
> and 
> common code could still be shared. Of course you can not say much 
> without looking at the code but will you even entertain such a change
> ?

I am sorry, but I would prefer no code refactoring unless you fix a
serious bug, or prepare for something really new (and having noticeable
impact)

We have to maintain stable trees, and such code churns are adding
maintenance hassles.

Of course, you can submit patches, but be warned that you can not
expect us spending hours reviewing patches that might bring serious
regressions.

I suggest you start with small patches first.

Re: Use of Indirect function calls

[Rao Shoaib]

On 03/06/2018 10:32 PM, Eric Dumazet wrote:
> On Tue, 2018-03-06 at 21:53 -0800, Rao Shoaib wrote:
>> David,
>>
>> Thanks a lot for your prompt response. Do you have a specific
>> solution
>> in mind or will the calls be replaced with simple checks ?
> There is upcoming work for that, but not specific to TCP stack.
>
>> Also while I have your attention can I ask your opinion about
>> breaking
>> up some TCP functions, mostly control functions into smaller units
>> so
>> that if a little different behavior is desired it can be achieved
>> and
>> common code could still be shared. Of course you can not say much
>> without looking at the code but will you even entertain such a change
>> ?
> I am sorry, but I would prefer no code refactoring unless you fix a
> serious bug, or prepare for something really new (and having noticeable
> impact)
>
> We have to maintain stable trees, and such code churns are adding
> maintenance hassles.
>
> Of course, you can submit patches, but be warned that you can not
> expect us spending hours reviewing patches that might bring serious
> regressions.
>
> I suggest you start with small patches first.
>
Thanks a lot Eric, I absolutely understand your concerns. I asked the 
question as I did not want to spend the time if you guys were not 
willing to even entertain the idea. Thanks a lot for your flexibility.
The changes I am making are very simple and very unlikely to cause any 
issues. You can always reject the patch as too big and I will work to 
make it smaller.

Regards,

Shoaib

Re: Use of Indirect function calls

[David Miller]

From: Rao Shoaib <rao.shoaib@oracle.com>
Date: Tue, 6 Mar 2018 21:53:19 -0800

> Also while I have your attention can I ask your opinion about
> breaking up some TCP functions, mostly control functions into
> smaller units so that if a little different behavior is desired it
> can be achieved and common code could still be shared. Of course you
> can not say much without looking at the code but will you even
> entertain such a change ?

You answered your own question, I'd have to see what the change
looks like to make a determination.