Re: x86 - cpu_relax - why nop vs. pause?

[Michael Breuer <mbreuer@majjas.com>]

On 2/7/2010 4:15 PM, Michael Breuer wrote:
> On 02/07/2010 03:08 PM, Michael Breuer wrote:
>> On 2/7/2010 1:14 PM, Mike Galbraith wrote:
>> ...
>> Case1 - asm volatile("pause" ::: "memory");
>> 0000000000400480 <main>:
>>   400480:    f3 90                    pause
>>   400482:    c3                       retq
>>   400483:    90                       nop
>>
>> ...
>>
>> Case3 - asm volitile("rep;pause" ::: "memory")
>> 0000000000400480 <main>:
>>   400480:    f3 f3 90                 pause
>>   400483:    c3                       retq
>>   400484:    90                       nop
>> _______
>> Note the difference between opcodes case 1 and case 3, and the mess 
>> made by the compiler in case 2.
>>
>> As to benchmarks  - I've checked a few things, no formal or lasting 
>> stuff... but striking at first glance:
>>
>> 1) At idle, perf top shows time spent in _raw_spin_lock dropping from 
>> ~35% to ~25%.
>> 2) Running a media transcode (single core - handbrakecli): frame rate 
>> increased by about 5-10%.
>> 3) During file-intensive operations (#2, above, or copying large 
>> files - ext4 on software raid6) - latencytop shows a decerase on 
>> writing a page to disc from about 120ms to about 90ms.
>>
> Disregard case 2 - was missing -O3. With -O3 or -O2 rep;nop and pause 
> are identical. The interesting case is rep;pause which is different 
> and seems more efficient.
Just to move away from this... totally perplexed, I retested a bit. 
Seems something else had gone wrong causing me to try 'rep;pause' vs. 
'pause'. The resulting opcode is f3 f3 90, as noted above.

I do see what seems to be a small but noticeable performance improvement 
- no idea if there's a downside, and also no idea what f3 f3 90 does vs. 
f3 90. Might be something interesting, or maybe not.
Test scenario:

Boot clean to single user mode. perform tiotest -8 five times.
%cpu is %usr + %sys as reported by tiotest.

Results:
Writes
pause:         1.14 sec; 72.01MB/sec; 322.44%cpu
rep;pause:   1.12 sec; 70.4MB/sec; 311.58%cpu
Random Writes
pause:         3.7 sec; 8.51MB/sec; 66.48%cpu
rep;pause    3.46sec; 9.04MB/sec; 72.34%cpu
Reads
pause:         11557.48MB/sec; 6040.74%cpu
rep;pause    11620.15MB/sec; 5974.90%cpu
Random Reads
pause:          11416.9MB/sec; 5330.50%cpu
rep;pause     11786.99MB/sec; 5118.66%cpu