Re: [Qemu-devel] [RFC PATCH 0/5] FAULT INJECTION FRAMEWORK

[Stefan Hajnoczi <stefanha@gmail.com>]

[-- Attachment #1: Type: text/plain, Size: 2705 bytes --]

On Wed, Jul 03, 2019 at 05:47:47PM +0200, Damien Hedde wrote:
> On 7/3/19 11:29 AM, Stefan Hajnoczi wrote:
> > On Mon, Jul 01, 2019 at 12:16:44PM +0200, Philippe Mathieu-Daudé wrote:
> >> On 7/1/19 10:37 AM, Stefan Hajnoczi wrote:
> >>> On Fri, Jun 28, 2019 at 02:45:29PM +0200, Damien Hedde wrote:
> >>>> This series adds a python framework aiming to provide some ways to do fault
> >>>> injection in a running vm. In its current state, it allows to easily interact
> >>>> with memory, change gpios and qom properties.
> >>>>
> >>>> The framework consists in a python script based on the qmp existing module
> >>>> which allows to interact with the vm.
> >>>
> >>> How does this compare to qtest?  There seems to be a lot of overlap
> >>> between them.
> >>>
> >>> Why is it called "fault injection"?  The commands seem to be
> >>> general-purpose device testing functions (like qtest and libqos), not
> >>> functions for testing error code paths as would be expected from a fault
> >>> injection framework.
> >>
> >> I understand qtest is to test QEMU, while this framework/command is to
> >> test how the guest react to an hardware faults.
> > 
> > The commands seems to be equivalent to qtest commands, just implemented
> > as QMP commands.
> > 
> > Damien: Can you explain the use case more and show some example test
> > cases?
> 
> The goal is to test and validate the software running on the vp. We want
> to generate some fault to test if the software behave correctly. We
> target corner cases that do not happen otherwise on the vp. Basically we
> would like, using some scripts, to run some specific scenarios and check
> that the expected behavior happens.
> 
> Regarding qtest, I was not aware that it provided such commands. I'm
> sorry I've missed that. Just checked after reading your feedback,
> commands seem indeed equivalent. I don't know if running the vp with
> qtest enabled has some hidden drawbacks. But if that's not the case, we
> can work to extend the existing qtest commands (or switch some of them
> to QMP like Philippe proposed, I don't know what's best).

I'm not 100% sure that qtest is the right tool for the job.  Maybe you
really need to add QMP commands as you have done.

Could you share some test cases so reviewers have an idea of how these
new commands are used for fault injection?

qtest is special in that no guest code executes.  QEMU allocates guest
RAM and initializes devices as usual but TCG/KVM do not execute guest
CPU instructions.  Does your use case require guest execution?

Here is a presentation on qtest if you want to get an overview:
https://www.youtube.com/watch?v=4TSaMmrnHy8

Stefan

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]