From mboxrd@z Thu Jan 1 00:00:00 1970 From: Raindog Subject: Re: debugging windows guests Date: Tue, 15 Dec 2009 20:07:20 -0800 Message-ID: <4B285CF8.8030401@macrohmasheen.com> References: <4B26D775.90809@macrohmasheen.com> <4B281E4A.1050608@web.de> Mime-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit To: kvm@vger.kernel.org Return-path: Received: from macrohmasheen.com ([206.123.88.147]:41427 "EHLO macrohmasheen.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1754163AbZLPEHa (ORCPT ); Tue, 15 Dec 2009 23:07:30 -0500 Received: from [10.0.1.102] (unknown [209.90.234.203]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by macrohmasheen.com (Postfix) with ESMTPSA id D7FB32D80075 for ; Tue, 15 Dec 2009 20:07:27 -0800 (PST) In-Reply-To: <4B281E4A.1050608@web.de> Sender: kvm-owner@vger.kernel.org List-ID: On 12/15/2009 3:39 PM, Jan Kiszka wrote: > Raindog wrote: > > Hello, > > > > I am researching KVM as a malware analysis platform and had some > > questions about debugging the guest OS. In my case I intend to use > > windows guests. So my questsions are as follows: > > > > Questions: > > > > 1. What instrumentation facilities are their available? > > > > 2. Is it possible to extend the debugging interface so that debugging is > > more transparent to the guest OS? IE: there is still a limit of 4 HW > > breakpoints (which makes me wonder why a LIST is used for them...) > > In accelerated KVM mode, the x86 architecture restricts us to 4 break- > or watchpoints that can be active at the same time. If you switch to > emulation mode, there are no such limits. Actually, I just made use of > this for debugging a subtle stack corruption in a guest, and I had more > than 70 watchpoints active at the same time. It's just "slightly" slower > than KVM... > Ok, is there anything special that needs to be done to enable additional watchpoints as they are being called? How are these set btw? Is it accomplished transparently through gdb? IE: if you set a watchpoint at a specific address, under emulation mode, they simulate HW bps in that no code is modified via the injection of an int 3? > > > > 3. I'm not finding any published API for interfacing with KVM/KQEMU/QEMU > > at a low level, for example, for writing custom tracers, etc. Is there > > one? Or is there something similar? > > KVM provides tracepoints for the Linux ftrace framework, see related > documentation of the kernel. I found this http://lxr.linux.no/#linux+v2.6.27/Documentation/ftrace.txt but that can hardly be accused of being called documentation. I don't think something like this: http://www.pintool.org/tutorials/asplos08/slides/PinTutorial.pdf is unreasonable. > If you extend your guest Windows is by design not extensible. > to issue certain > events that the hypervisor sees and traces (e.g. writes to pseudo I/O > ports), you can also trace things inside the guest that are otherwise > invisible to the host. I once hacked up an ad-hoc tracing by means of > hypercalls (required some kvm patching). That also worked from guest > userspace - and revealed that even more hypercalls could be called that > way (that's fixed in KVM now). > > > > > > > Bugs: > > > > 1. I hit a bug w/ instruction logging using a RAM based temp folder. If > > I ran w/ the following command line: > > (Version info: QEMU PC emulator version 0.10.50 (qemu-kvm-devel-88)) > > > > qemu-system-x86_64 -hda debian.img -enable-nesting -d in_asm > > -d only works in emulation mode as it relies on dynamic code translation > (TCG). For qemu-kvm, you need to switch to emulation via -no-kvm (for > upstream QEMU, it's the other way around). > Hence why running w/out admin rights enables the logging. I was confused because the logfile is still created w/ the kvm module disabled. > > > > It would successfully log to the tmp log file, but obviously, KVM would > > be disabled. > > > > If I use sudo, it won't log to the file, is this a known issue? > > > > 2. -enable-nesting on AMD hardware using a xen guest OS causes xen to > > GPF somewhere in svm_cpu_up. Is nesting supposed to work w/ Xen based > > guests? > > If your host kernel or kvm-kmod is not 2.6.32 based, update first. A lot > of nested SVM fixes went in recently. If it still fails, put Alex (Graf) > and Joerg (Roedel) on CC. > Upgrading now... > Jan > >