Re: [Qemu-devel] QEMU postcopy-test failing on ppc64

[Greg Kurz <groug@kaod.org>]

On Tue, 15 Nov 2016 19:24:24 +0100
Laurent Vivier <lvivier@redhat.com> wrote:

> On 15/11/2016 19:01, Greg Kurz wrote:
> > On Tue, 15 Nov 2016 16:07:40 +0100
> > Laurent Vivier <lvivier@redhat.com> wrote:
> >   
> >> On 15/11/2016 16:03, Greg Kurz wrote:  
> >>> On Tue, 15 Nov 2016 14:48:30 +0000
> >>> Stefan Hajnoczi <stefanha@gmail.com> wrote:
> >>>     
> >>>> On Tue, Nov 15, 2016 at 01:58:38PM +0100, Laurent Vivier wrote:    
> >>>>>
> >>>>>
> >>>>> On 15/11/2016 13:20, Dr. David Alan Gilbert wrote:      
> >>>>>> * Stefan Hajnoczi (stefanha@gmail.com) wrote:      
> >>>>>>> On Tue, Nov 15, 2016 at 10:09 AM, Greg Kurz <groug@kaod.org> wrote:      
> >>>>>>>> On Tue, 15 Nov 2016 10:53:35 +0100
> >>>>>>>> Laurent Vivier <lvivier@redhat.com> wrote:
> >>>>>>>>      
> >>>>>>>>> On 14/11/2016 21:52, Stefan Hajnoczi wrote:      
> >>>>>>>>>> I hit a failure running "make check" on ppc64 for the first time.  Ideas?
> >>>>>>>>>>
> >>>>>>>>>> Stefan
> >>>>>>>>>>
> >>>>>>>>>> commit 682df581c65ed2c1b9e77093e332214ecaa1ee93
> >>>>>>>>>>
> >>>>>>>>>>   GTESTER check-qtest-ppc64
> >>>>>>>>>> Memory content inconsistency at 5af4000 first_byte = 1b last_byte = 1a
> >>>>>>>>>> current = 7c hit_edge = 1
> >>>>>>>>>> Memory content inconsistency at 5af5000 first_byte = 1b last_byte = 7c
> >>>>>>>>>> current = 1b hit_edge = 1
> >>>>>>>>>> Memory content inconsistency at 5e59000 first_byte = 1b last_byte = 1b
> >>>>>>>>>> current = 1a hit_edge = 1
> >>>>>>>>>> **
> >>>>>>>>>> ERROR:tests/postcopy-test.c:345:check_guests_ram: 'bad' should be FALSE
> >>>>>>>>>> GTester: last random seed: R02S9d79166a1ca7e21940a0f4b0b1255d5b
> >>>>>>>>>>      
> >>>>>>>>>
> >>>>>>>>> Are you using KVM PR?
> >>>>>>>>>
> >>>>>>>>> it  was working fine with TCG and KVM HV.
> >>>>>>>>>
> >>>>>>>>> Apparently, USERFAULTFD doesn't work with KVM PR.
> >>>>>>>>>
> >>>>>>>>> I've already seen this kind of error with nested KVM on Power:
> >>>>>>>>> guest in guest with KVM PR in host.
> >>>>>>>>>
> >>>>>>>>> This problem was reported on IRC by Greg if I remember correctly (CC:)
> >>>>>>>>>      
> >>>>>>>>
> >>>>>>>> Yeah I hit this when running make check in a PPC64 BE guest which
> >>>>>>>> has kvm_pr loaded. I did not find time to investigate though... I've
> >>>>>>>> switched to run make check on bare metal POWER7 instead.      
> >>>>>>>
> >>>>>>> Right, it's POWER7 PPC64 BE with kvm_pr.
> >>>>>>>
> >>>>>>> If this should be fixed for QEMU 2.8 please add it to
> >>>>>>> http://qemu-project.org/Planning/2.8 and I'll track it (i.e. won't
> >>>>>>> release before it's resolved).
> >>>>>>>
> >>>>>>> If this is a known issue that will be in QEMU 2.8 please add it to
> >>>>>>> http://qemu-project.org/ChangeLog/2.8#Known_issues.      
> >>>>>>
> >>>>>> I don't think it's new with 2.8 (Laurent?) and I think it's actually
> >>>>>> a kernel issue that needs fixing.      
> >>>>>
> >>>>> I's not a regression and I think it's a kernel issue.
> >>>>>
> >>>>> As it has never worked with KVM PR and can't be fixed at QEMU level, we
> >>>>> can't solve it for 2.8 (no tracking).
> >>>>>
> >>>>> I'm going to update the know issues for 2.8.
> >>>>> [and I'm going to try to understand what really happens]      
> >>>>
> >>>> Should the test be skipped on KVM PR?
> >>>>
> >>>> It's a shame that make check fails.
> >>>>    
> >>>
> >>> Yeah I agree this is lame but KVM PR isn't very cherished these days...
> >>>
> >>> Since postcopy-test runs QEMU with -accel kvm:tcg, a possible workaround is
> >>> to kick KVM out of the environment you're running make check in.    
> >>
> >> Moreover, using "-accel kvm:tcg" induces an error when the test is
> >> running in cross-arch case (ppc on intel, or intel on ppc).
> >>  
> > 
> > You're talking about the '"kvm" accelerator not found' error ?  
> 
> Yes
> 
> >   
> >> But I think postcopy is typically something we should test with KVM, not
> >> TCG, as there are some kernel dependencies (USERFAULTFD).
> >>  
> > 
> > I agree it should be tested with KVM when applicable but I don't understand
> > the 'not TCG' since it is the only way to test when you don't have access
> > to specific hardware...  
> 
> I agree; What I mean is we must test this preferably with KVM because it
> is more likely to have a problem with KVM than with TCG because of the
> MMU virtualization.
> 

Fair enough but this doesn't address the problem we currently have: a broken
KVM can cause make check to fail... and BTW, until the root cause is found,
I'm not sure if the culprit is KVM PR alone or QEMU+KVM PR...

Cheers.

--
Greg

> >> It has been suggested a while to use an environment variable, like for
> >> kvm-unit-tests, something like QTEST_ACCEL="kvm".
> >>  
> > 
> > Yeah, I think it would be better than "-accel kvm:tcg"
> >   
> >> Laurent  
> > 
> > Cheers.
> > 
> > --
> > Greg  
> 
> Thanks,
> Laurent