Hi everyone, I wrote a patch that tries to get some data where we spent our time. This should give us some data to think about when we talk about virtualization on non hardware assisted platforms like 440. A good example what this data might help us to discuss is the comparison between paravirtualized and non paravirtualized guest (using the magic page mechanism I posted recently). You see that the pv extension reduces the number of exits for instruction emulation by 50%, but unfortunately that only reduced the time we spend for the emulation by ~20%. This data is for you information and comments/discussions are welcome (I expect that we need to trace the data of the emulation per instr. to identify the slow ones and think about their optimization). The basic approach is to read TB[U&L] when entering the guest and when exiting it. The time from exit->enter is the time we spent to handle the exit and accounted per exit type. The time enter->exit is the time we spent in the guest and accounted too. For details I attached the patch I used to measure the timing. I verified it using with serveral assertions I now removed in the final version like "testing for double accounting of an exit" or "testing if we missed to account an exit". Additionally I verified the exit counts with the data reported by our kvm_stat support and that matches. Also the sum of accounted time matches the real time spent to run the test (Test was booting a ubuntu guest, logon as root, executing a ls, kill guest). I think this pretty much explains why the guest is sometimes feeling slow :-). Handle this data as semi-experimental and let me know what you think. NO PV support unit is usec traced duration 202,244,233 usec count min max sum avg stddev time% EMULINST 6,423,699 12 247,582 91,732,705 14.2804 241.200 45.36 ITLBVIRT 1,777,242 21 264,257 47,116,557 26.5111 286.040 23.30 DTLBVIRT 1,544,241 22 263,947 41,765,447 27.0459 218.997 20.65 TIMEINGUEST 10,000,000 0 3,673 10,861,792 1.0862 4.790 5.37 MMIO 9,318 47 10,686 3,544,934 380.4394 507.292 1.75 EXTINT 3,523 33 4,475 2,557,183 725.8538 619.026 1.26 DEC 16,937 50 4,866 1,441,084 85.0850 203.856 0.71 DTLBREAL 95,497 13 105 1,349,255 14.1288 2.811 0.67 ITLBREAL 80,622 13 146 1,079,052 13.3841 2.870 0.53 DSI 22,498 11 99 249,554 11.0923 1.975 0.12 DCR 4,854 41 3,758 232,657 47.9310 55.802 0.12 ISI 11,167 11 67 124,210 11.1230 2.242 0.06 SYSCALL 9,459 11 100 113,972 12.0491 2.763 0.06 SIGNAL 635 66 6,025 72,438 114.0756 335.235 0.04 FP_UNAVAIL 308 11 12 3,393 11.0162 0.126 0.00 *never hit USR_PR_INST, HCALL, HALT, DTLBPV, DEBUG With PV Support unit is usec traced duration 136,640,181 usec count min max sum avg stddev time% EMULINST 3,160,786 12 247,587 50,808,972 16.0748 205.445 37.18 ITLBVIRT 1,333,725 21 264,336 35,094,300 26.3130 235.204 25.68 DTLBVIRT 1,212,505 22 16,753 32,382,302 26.7069 51.475 23.70 TIMEINGUEST 6,000,000 0 3,846 7259,180 1.2099 9.317 5.31 MMIO 9,955 47 12,289 3570,654 358.6795 539.759 2.61 EXTINT 2,380 31 5,285 1791,705 752.8172 665.021 1.31 ITLBREAL 80,690 13 247,582 1320,621 16.3666 871.535 0.97 DTLBREAL 93,213 13 108 1315,333 14.1110 2.799 0.96 DTLBPV 46,941 20 3,696 1100,399 23.4422 50.127 0.81 DEC 10,376 50 3,672 928,878 89.5218 217.384 0.68 SIGNAL 658 66 264,544 339,911 516.5821 10303.957 0.25 DSI 21,799 11 87 242,343 11.1172 2.227 0.18 DCR 5,042 41 258 233,095 46.2307 6.465 0.17 ISI 11,582 11 66 128,625 11.1056 2.133 0.09 SYSCALL 10,052 11 59 120,597 11.9973 2.415 0.09 FP_UNAVAIL 295 11 12 3,247 11.0068 0.082 0.00 HCALL 1 19 19 19 19.0000 0.000 0.00 *never hit USR_PR_INST, HALT, DEBUG -- Grüsse / regards, Christian Ehrhardt IBM Linux Technology Center, Open Virtualization