audit_p2m errors, p2m_mmio_direct turns into p2m_ram_logdirty

audit_p2m errors, p2m_mmio_direct turns into p2m_ram_logdirty

[Olaf Hering]

Hello,

audit_p2m appears to be unmaintained, given that it did not even compile
up to a few weeks ago (and it still does not compile in 4.0 tree).

Today I tried it again in the hope to get some help for xenpaging
debugging in xen-4.0, but it ran into a BUG right away.
For some reason the p2m type set by set_mmio_p2m_entry for gfn 0xfee00
is 2 instead of 5 when audit_p2m checks it.

How can the BUG get fixed properly?

(I turned all P2M_PRINTK into printk, and dumped the p2mt in a few
places):

(XEN) p2m: p2m_alloc_table(1645): allocating p2m table
(XEN) p2m: p2m_alloc_table(1669): populating p2m table
(XEN) p2m: p2m_alloc_table(1697): p2m table initialised (0 pages)
(XEN) p2mdebug: set_mmio_p2m_entry(2399): set mmio fee00 134012
(XEN) cpupool_add_domain(dom=1,pool=0) n_dom 2
(XEN) p2m: audit_p2m(1762): p2m audit starts
(XEN) p2m: audit_p2m(1777): auditing guest page, mfn=0x133b86
(XEN) p2m: audit_p2m(1821): map mismatch mfn 0x133b86 -> gfn 0xed47f -> mfn 0xffffffffffffffff (-> gfn 0xffffffff) p2mt 4
(XEN) p2m: audit_p2m(1839): OK: mfn=0x133b86, gfn=0xed47f, p2mfn=0xffffffffffffffff, lp2mfn=0x0
(XEN) p2m: audit_p2m(1941): mismatch: gfn 0xfee00 -> mfn 0x134012 -> gfn 0xffffffffffffffff (p2mt 2)
(XEN) Xen BUG at p2m.c:1942
(XEN) Debugging connection not set up.


Olaf

Re: audit_p2m errors, p2m_mmio_direct turns into p2m_ram_logdirty

[Tim Deegan]

Hi, 

At 16:54 +0000 on 02 Nov (1288716869), Olaf Hering wrote:
> audit_p2m appears to be unmaintained, given that it did not even compile
> up to a few weeks ago (and it still does not compile in 4.0 tree).

Yes, the p2m audit code was brought up to date with the
populate-on-demand patches but it looks like the paging/sharing patches
didn't touch it at all.

> Today I tried it again in the hope to get some help for xenpaging
> debugging in xen-4.0, but it ran into a BUG right away.
> For some reason the p2m type set by set_mmio_p2m_entry for gfn 0xfee00
> is 2 instead of 5 when audit_p2m checks it.

I wans't able to reproduce this on xen-unstable tip.  Were you using the
paging code at the time?

> How can the BUG get fixed properly?
> 
> (I turned all P2M_PRINTK into printk, and dumped the p2mt in a few
> places):
> 
> (XEN) p2m: p2m_alloc_table(1645): allocating p2m table
> (XEN) p2m: p2m_alloc_table(1669): populating p2m table
> (XEN) p2m: p2m_alloc_table(1697): p2m table initialised (0 pages)
> (XEN) p2mdebug: set_mmio_p2m_entry(2399): set mmio fee00 134012
> (XEN) cpupool_add_domain(dom=1,pool=0) n_dom 2
> (XEN) p2m: audit_p2m(1762): p2m audit starts
> (XEN) p2m: audit_p2m(1777): auditing guest page, mfn=0x133b86
> (XEN) p2m: audit_p2m(1821): map mismatch mfn 0x133b86 -> gfn 0xed47f -> mfn 0xffffffffffffffff (-> gfn 0xffffffff) p2mt 4

This might be benign - a page owned by the domain that's not in the p2m
at all yet, with a stale m2p entry from its previous owner.  I'll write
up a patch to remove these warning for freshly allocated pages.

> (XEN) p2m: audit_p2m(1839): OK: mfn=0x133b86, gfn=0xed47f, p2mfn=0xffffffffffffffff, lp2mfn=0x0
> (XEN) p2m: audit_p2m(1941): mismatch: gfn 0xfee00 -> mfn 0x134012 -> gfn 0xffffffffffffffff (p2mt 2)

That's a bug, though. 

Your line numbers don't match my tree, but I take it this is the third
(i.e. non-superpage) instance of this check?  In that case it looks like
it's found a real bug - for any log-dirty RAM p2m entry the m2p ought to
match the p2m.  Also an MMIO entry shouldn't become without becoming
normal RAM in the meantime. 

The next step, I'm afraid, is to trace where the type is getting
changed.  If the pfn is the same on each run (and it look like it should
be) then you should be able to put an appropriate WARN_ON in
paging_write_p2m_entry() to catch all changes to that pfn's type.

Tim.

-- 
Tim Deegan <Tim.Deegan@citrix.com>
Principal Software Engineer, Xen Platform Team
Citrix Systems UK Ltd.  (Company #02937203, SL9 0BG)

Re: audit_p2m errors, p2m_mmio_direct turns into p2m_ram_logdirty

[Olaf Hering]

On Wed, Nov 03, Tim Deegan wrote:

> > Today I tried it again in the hope to get some help for xenpaging
> > debugging in xen-4.0, but it ran into a BUG right away.
> > For some reason the p2m type set by set_mmio_p2m_entry for gfn 0xfee00
> > is 2 instead of 5 when audit_p2m checks it.
> 
> I wans't able to reproduce this on xen-unstable tip.  Were you using the
> paging code at the time?

Its with the 4.0 tree, and xenpaging starts much later. Its the early
domain setup code.

> > (XEN) p2m: audit_p2m(1839): OK: mfn=0x133b86, gfn=0xed47f, p2mfn=0xffffffffffffffff, lp2mfn=0x0
> > (XEN) p2m: audit_p2m(1941): mismatch: gfn 0xfee00 -> mfn 0x134012 -> gfn 0xffffffffffffffff (p2mt 2)
> 
> That's a bug, though. 
> 
> Your line numbers don't match my tree, but I take it this is the third
> (i.e. non-superpage) instance of this check?  In that case it looks like
> it's found a real bug - for any log-dirty RAM p2m entry the m2p ought to
> match the p2m.  Also an MMIO entry shouldn't become without becoming
> normal RAM in the meantime. 
> 
> The next step, I'm afraid, is to trace where the type is getting
> changed.  If the pfn is the same on each run (and it look like it should
> be) then you should be able to put an appropriate WARN_ON in
> paging_write_p2m_entry() to catch all changes to that pfn's type.

Thanks for looking into this. I will try to trace this and see where the
type changes.

Olaf