[PATCH v1] drm/i915: Enhanced for initialize partially filled pagetables

[PATCH v1] drm/i915: Enhanced for initialize partially filled pagetables

[Xiaolin Zhang]

if vgpu active, the page table entry should be initialized after
allocation and then the hypersivor can ping pages succesuffly,
otherwise hypervisor will ping pages failed and the host will print
a lot of annoying errors such as “ERROR gvt: guest page write error -22,
gfn 0x7ada8, pa 0x7ada89a8, var 0x6, len 1” when create linux guest.

Signed-off-by: Xiaolin Zhang <xiaolin.zhang@intel.com>
---
 drivers/gpu/drm/i915/i915_gem_gtt.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/drivers/gpu/drm/i915/i915_gem_gtt.c b/drivers/gpu/drm/i915/i915_gem_gtt.c
index 731ce229a923..be52b139817d 100644
--- a/drivers/gpu/drm/i915/i915_gem_gtt.c
+++ b/drivers/gpu/drm/i915/i915_gem_gtt.c
@@ -1175,7 +1175,7 @@ static int gen8_ppgtt_alloc_pd(struct i915_address_space *vm,
 			if (IS_ERR(pt))
 				goto unwind;
 
-			if (count < GEN8_PTES)
+			if (count < GEN8_PTES || intel_vgpu_active(vm->i915))
 				gen8_initialize_pt(vm, pt);
 
 			gen8_ppgtt_set_pde(vm, pd, pt, pde);
-- 
2.14.1

_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

✓ Fi.CI.BAT: success for drm/i915: Enhanced for initialize partially filled pagetables

[Patchwork]

== Series Details ==

Series: drm/i915: Enhanced for initialize partially filled pagetables
URL   : https://patchwork.freedesktop.org/series/31029/
State : success

== Summary ==

Series 31029v1 drm/i915: Enhanced for initialize partially filled pagetables
https://patchwork.freedesktop.org/api/1.0/series/31029/revisions/1/mbox/

Test drv_module_reload:
        Subgroup basic-reload:
                dmesg-warn -> PASS       (fi-glk-1) fdo#102777

fdo#102777 https://bugs.freedesktop.org/show_bug.cgi?id=102777

fi-bdw-5557u     total:289  pass:268  dwarn:0   dfail:0   fail:0   skip:21  time:439s
fi-bdw-gvtdvm    total:289  pass:265  dwarn:0   dfail:0   fail:0   skip:24  time:475s
fi-blb-e6850     total:289  pass:224  dwarn:1   dfail:0   fail:0   skip:64  time:420s
fi-bsw-n3050     total:289  pass:243  dwarn:0   dfail:0   fail:0   skip:46  time:513s
fi-bwr-2160      total:289  pass:184  dwarn:0   dfail:0   fail:0   skip:105 time:279s
fi-bxt-j4205     total:289  pass:260  dwarn:0   dfail:0   fail:0   skip:29  time:517s
fi-byt-j1900     total:289  pass:254  dwarn:1   dfail:0   fail:0   skip:34  time:494s
fi-cfl-s         total:289  pass:223  dwarn:34  dfail:0   fail:0   skip:32  time:540s
fi-cnl-y         total:289  pass:259  dwarn:0   dfail:0   fail:3   skip:27  time:644s
fi-elk-e7500     total:289  pass:230  dwarn:0   dfail:0   fail:0   skip:59  time:417s
fi-glk-1         total:289  pass:260  dwarn:0   dfail:0   fail:0   skip:29  time:567s
fi-hsw-4770      total:289  pass:263  dwarn:0   dfail:0   fail:0   skip:26  time:426s
fi-hsw-4770r     total:289  pass:263  dwarn:0   dfail:0   fail:0   skip:26  time:403s
fi-ilk-650       total:289  pass:229  dwarn:0   dfail:0   fail:0   skip:60  time:429s
fi-ivb-3520m     total:289  pass:261  dwarn:0   dfail:0   fail:0   skip:28  time:491s
fi-ivb-3770      total:289  pass:261  dwarn:0   dfail:0   fail:0   skip:28  time:466s
fi-kbl-7500u     total:289  pass:264  dwarn:1   dfail:0   fail:0   skip:24  time:474s
fi-kbl-7560u     total:289  pass:270  dwarn:0   dfail:0   fail:0   skip:19  time:583s
fi-kbl-r         total:289  pass:262  dwarn:0   dfail:0   fail:0   skip:27  time:591s
fi-pnv-d510      total:289  pass:223  dwarn:1   dfail:0   fail:0   skip:65  time:550s
fi-skl-6260u     total:289  pass:269  dwarn:0   dfail:0   fail:0   skip:20  time:454s
fi-skl-6700k     total:289  pass:265  dwarn:0   dfail:0   fail:0   skip:24  time:754s
fi-skl-6770hq    total:289  pass:269  dwarn:0   dfail:0   fail:0   skip:20  time:491s
fi-skl-gvtdvm    total:289  pass:266  dwarn:0   dfail:0   fail:0   skip:23  time:472s
fi-snb-2520m     total:289  pass:251  dwarn:0   dfail:0   fail:0   skip:38  time:560s
fi-snb-2600      total:289  pass:250  dwarn:0   dfail:0   fail:0   skip:39  time:416s

7f93222785e4887c083c85b76fcbb391bb1991d9 drm-tip: 2017y-09m-27d-20h-05m-25s UTC integration manifest
ef12d5353059 drm/i915: Enhanced for initialize partially filled pagetables

== Logs ==

For more details see: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_5840/
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

✓ Fi.CI.IGT: success for drm/i915: Enhanced for initialize partially filled pagetables

[Patchwork]

== Series Details ==

Series: drm/i915: Enhanced for initialize partially filled pagetables
URL   : https://patchwork.freedesktop.org/series/31029/
State : success

== Summary ==

shard-hsw        total:2429 pass:1332 dwarn:4   dfail:0   fail:10  skip:1083 time:9949s

== Logs ==

For more details see: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_5840/shards.html
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

Re: [PATCH v1] drm/i915: Enhanced for initialize partially filled pagetables

[Joonas Lahtinen]

On Thu, 2017-09-28 at 10:09 +0800, Xiaolin Zhang wrote:
> if vgpu active, the page table entry should be initialized after
> allocation and then the hypersivor can ping pages succesuffly,
> otherwise hypervisor will ping pages failed and the host will print
> a lot of annoying errors such as “ERROR gvt: guest page write error -22,
> gfn 0x7ada8, pa 0x7ada89a8, var 0x6, len 1” when create linux guest.
> 
> Signed-off-by: Xiaolin Zhang <xiaolin.zhang@intel.com>

Why does the hypervisor try to access the entries prior to them being
made valid for hardware?

Regards, Joonas
-- 
Joonas Lahtinen
Open Source Technology Center
Intel Corporation
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

Re: [PATCH v1] drm/i915: Enhanced for initialize partially filled pagetables

[Zhang, Xiaolin]

[-- Attachment #1.1: Type: text/plain, Size: 1235 bytes --]

On 09/28/2017 10:25 PM, Joonas Lahtinen wrote:

On Thu, 2017-09-28 at 10:09 +0800, Xiaolin Zhang wrote:


if vgpu active, the page table entry should be initialized after
allocation and then the hypersivor can ping pages succesuffly,
otherwise hypervisor will ping pages failed and the host will print
a lot of annoying errors such as “ERROR gvt: guest page write error -22,
gfn 0x7ada8, pa 0x7ada89a8, var 0x6, len 1” when create linux guest.

Signed-off-by: Xiaolin Zhang <xiaolin.zhang@intel.com><mailto:xiaolin.zhang@intel.com>


Why does the hypervisor try to access the entries prior to them being
made valid for hardware?

Regards, Joonas


Hi Joonas,

thanks your comment.

I think what you ask is the point we got the error message in gvt since the current gvt

implementation is that page under write protection and trapped should be valid

with correct shadow page setup and p2m translation. Actually, to work with

“initialize partially filled pagetables" , there is a certain refine work to do in gvt side

(maybe less or maybe large). but before refine work done, this patch is trying to bring back

gvt behavior as before “initialize partially filled pagetables"patch.

BRs, Xiaolin

[-- Attachment #1.2: Type: text/html, Size: 1772 bytes --]

[-- Attachment #2: Type: text/plain, Size: 160 bytes --]

_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

Re: [PATCH v1] drm/i915: Enhanced for initialize partially filled pagetables

[Joonas Lahtinen]

+ Zhenyu and Zhi

On Sat, 2017-09-30 at 02:58 +0000, Zhang, Xiaolin wrote:
> On 09/28/2017 10:25 PM, Joonas Lahtinen wrote:
> > On Thu, 2017-09-28 at 10:09 +0800, Xiaolin Zhang wrote:
> > > if vgpu active, the page table entry should be initialized after
> > > allocation and then the hypersivor can ping pages succesuffly,
> > > otherwise hypervisor will ping pages failed and the host will
> > > print
> > > a lot of annoying errors such as “ERROR gvt: guest page write
> > > error -22,
> > > gfn 0x7ada8, pa 0x7ada89a8, var 0x6, len 1” when create linux
> > > guest.
> > > 
> > > Signed-off-by: Xiaolin Zhang <xiaolin.zhang@intel.com>
> > 
> > Why does the hypervisor try to access the entries prior to them
> > being
> > made valid for hardware?
> > 
> > Regards, Joonas
> 
> Hi Joonas,
> thanks your comment. 
> I think what you ask is the point we got the error message in gvt since the current gvt
> implementation is that page under write protection and trapped should be valid 
> with correct shadow page setup and p2m translation.

My question is that if the hardware doesn't care about them being
uninitialized at this point, how can GVT? If the GVT implementation
relies heavily on how the i915 driver currently happens to behave, not
on what contracts it has with the hardware, these breakages are bound
to happen repeatedly. The code is being transformed and optimized on
daily basis, how it currently behaves is not a solid foundation for
implementing the host side virtualization.

> Actually, to work with 
> “initialize partially filled pagetables" , there is a certain refine work to do in gvt side
> (maybe less or maybe large). but before refine work done, this patch is trying to bring back 
> gvt behavior as before “initialize partially filled pagetables"patch. 

We should first minimize and then keep the vgpu specific checks to
minimum. So this would need to be fixed on the GVT side of code.

Regards, Joonas
-- 
Joonas Lahtinen
Open Source Technology Center
Intel Corporation
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

Re: [PATCH v1] drm/i915: Enhanced for initialize partially filled pagetables

[Wang, Zhi A]

It's because there is an optimization of pre-shadow in guest i915. When an i915 PPGTT is created, guest i915 will notify the hypervisor about this. Hypervisor will start to track the page table at this time. It's like the page table has already been valid on HW after being tracked by hypervisor. So when a PTE with invalid content is linked into the PDE, HV will report some errors.

We tried to remove this optimization and shadow the whole PPGTT page table before a vGPU workload was submitted. It ends up with huge performance drop since the PPGTT shadow has to be created and destroyed for every submit. It cost so much CPU% time and wastes a lot of GPU bandwidth.

A better solution from HV side is proactively stopping track a page with invalid content when it's linked into the page table and performing the shadow for this page before HV dispatches a vGPU workload (of course, if it's still invalid, HV will report error at this time). The lazy shadow feature stills needs some efforts to land into GVT-g. So we needs a short term solution.

From my opinion, we can check:

a) If the workaround can stay in GVT-g.
b) Gaps if the workaround has to stay in i915.

If we can move to a), then this patch is not necessary.

For long term solution, I'm working on that. It still needs quite some re-factors of code structure since once HV bites an invalid entry, it has to unwind the shadow has been done in that page recursively.

Thanks,
Zhi.

-----Original Message-----
From: Joonas Lahtinen [mailto:joonas.lahtinen@linux.intel.com] 
Sent: Monday, October 2, 2017 12:03 PM
To: Zhang, Xiaolin <xiaolin.zhang@intel.com>; intel-gvt-dev@lists.freedesktop.org; intel-gfx@lists.freedesktop.org
Cc: Zhenyu Wang <zhenyuw@linux.intel.com>; Wang, Zhi A <zhi.a.wang@intel.com>
Subject: Re: [Intel-gfx] [PATCH v1] drm/i915: Enhanced for initialize partially filled pagetables

+ Zhenyu and Zhi

On Sat, 2017-09-30 at 02:58 +0000, Zhang, Xiaolin wrote:
> On 09/28/2017 10:25 PM, Joonas Lahtinen wrote:
> > On Thu, 2017-09-28 at 10:09 +0800, Xiaolin Zhang wrote:
> > > if vgpu active, the page table entry should be initialized after 
> > > allocation and then the hypersivor can ping pages succesuffly, 
> > > otherwise hypervisor will ping pages failed and the host will 
> > > print a lot of annoying errors such as “ERROR gvt: guest page 
> > > write error -22, gfn 0x7ada8, pa 0x7ada89a8, var 0x6, len 1” when 
> > > create linux guest.
> > > 
> > > Signed-off-by: Xiaolin Zhang <xiaolin.zhang@intel.com>
> > 
> > Why does the hypervisor try to access the entries prior to them 
> > being made valid for hardware?
> > 
> > Regards, Joonas
> 
> Hi Joonas,
> thanks your comment. 
> I think what you ask is the point we got the error message in gvt 
> since the current gvt implementation is that page under write 
> protection and trapped should be valid with correct shadow page setup and p2m translation.

My question is that if the hardware doesn't care about them being uninitialized at this point, how can GVT? If the GVT implementation relies heavily on how the i915 driver currently happens to behave, not on what contracts it has with the hardware, these breakages are bound to happen repeatedly. The code is being transformed and optimized on daily basis, how it currently behaves is not a solid foundation for implementing the host side virtualization.

> Actually, to work with
> “initialize partially filled pagetables" , there is a certain refine 
> work to do in gvt side (maybe less or maybe large). but before refine 
> work done, this patch is trying to bring back gvt behavior as before “initialize partially filled pagetables"patch.

We should first minimize and then keep the vgpu specific checks to minimum. So this would need to be fixed on the GVT side of code.

Regards, Joonas
--
Joonas Lahtinen
Open Source Technology Center
Intel Corporation
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

✗ Fi.CI.BAT: failure for drm/i915: Enhanced for initialize partially filled pagetables

[Patchwork]

== Series Details ==

Series: drm/i915: Enhanced for initialize partially filled pagetables
URL   : https://patchwork.freedesktop.org/series/31029/
State : failure

== Summary ==

Series 31029v1 drm/i915: Enhanced for initialize partially filled pagetables
https://patchwork.freedesktop.org/api/1.0/series/31029/revisions/1/mbox/

Test gem_exec_reloc:
        Subgroup basic-cpu-active:
                pass       -> FAIL       (fi-gdg-551) fdo#102582 +4
Test kms_busy:
        Subgroup basic-flip-c:
                incomplete -> PASS       (fi-bxt-j4205)
Test kms_pipe_crc_basic:
        Subgroup read-crc-pipe-b:
                incomplete -> PASS       (fi-cnl-y)
        Subgroup suspend-read-crc-pipe-a:
                pass       -> INCOMPLETE (fi-glk-dsi)

fdo#102582 https://bugs.freedesktop.org/show_bug.cgi?id=102582

fi-bdw-5557u     total:289  pass:268  dwarn:0   dfail:0   fail:0   skip:21  time:443s
fi-bdw-gvtdvm    total:289  pass:265  dwarn:0   dfail:0   fail:0   skip:24  time:453s
fi-blb-e6850     total:289  pass:223  dwarn:1   dfail:0   fail:0   skip:65  time:371s
fi-bsw-n3050     total:289  pass:243  dwarn:0   dfail:0   fail:0   skip:46  time:530s
fi-bwr-2160      total:289  pass:183  dwarn:0   dfail:0   fail:0   skip:106 time:261s
fi-bxt-dsi       total:289  pass:259  dwarn:0   dfail:0   fail:0   skip:30  time:502s
fi-bxt-j4205     total:289  pass:260  dwarn:0   dfail:0   fail:0   skip:29  time:501s
fi-byt-j1900     total:289  pass:253  dwarn:1   dfail:0   fail:0   skip:35  time:486s
fi-byt-n2820     total:289  pass:249  dwarn:1   dfail:0   fail:0   skip:39  time:473s
fi-cfl-s         total:289  pass:253  dwarn:4   dfail:0   fail:0   skip:32  time:554s
fi-cnl-y         total:289  pass:262  dwarn:0   dfail:0   fail:0   skip:27  time:611s
fi-elk-e7500     total:289  pass:229  dwarn:0   dfail:0   fail:0   skip:60  time:418s
fi-gdg-551       total:289  pass:173  dwarn:1   dfail:0   fail:6   skip:109 time:250s
fi-glk-1         total:289  pass:261  dwarn:0   dfail:0   fail:0   skip:28  time:578s
fi-glk-dsi       total:245  pass:217  dwarn:0   dfail:0   fail:1   skip:26 
fi-hsw-4770      total:289  pass:262  dwarn:0   dfail:0   fail:0   skip:27  time:429s
fi-hsw-4770r     total:289  pass:262  dwarn:0   dfail:0   fail:0   skip:27  time:424s
fi-ilk-650       total:289  pass:228  dwarn:0   dfail:0   fail:0   skip:61  time:429s
fi-ivb-3520m     total:289  pass:260  dwarn:0   dfail:0   fail:0   skip:29  time:494s
fi-ivb-3770      total:289  pass:260  dwarn:0   dfail:0   fail:0   skip:29  time:463s
fi-kbl-7500u     total:289  pass:264  dwarn:1   dfail:0   fail:0   skip:24  time:493s
fi-kbl-7560u     total:289  pass:270  dwarn:0   dfail:0   fail:0   skip:19  time:572s
fi-kbl-7567u     total:289  pass:269  dwarn:0   dfail:0   fail:0   skip:20  time:476s
fi-kbl-r         total:289  pass:262  dwarn:0   dfail:0   fail:0   skip:27  time:578s
fi-pnv-d510      total:289  pass:222  dwarn:1   dfail:0   fail:0   skip:66  time:543s
fi-skl-6260u     total:289  pass:269  dwarn:0   dfail:0   fail:0   skip:20  time:452s
fi-skl-6600u     total:289  pass:262  dwarn:0   dfail:0   fail:0   skip:27  time:593s
fi-skl-6700hq    total:289  pass:263  dwarn:0   dfail:0   fail:0   skip:26  time:646s
fi-skl-6700k     total:289  pass:265  dwarn:0   dfail:0   fail:0   skip:24  time:516s
fi-skl-6770hq    total:289  pass:269  dwarn:0   dfail:0   fail:0   skip:20  time:506s
fi-skl-gvtdvm    total:289  pass:266  dwarn:0   dfail:0   fail:0   skip:23  time:454s
fi-snb-2520m     total:289  pass:250  dwarn:0   dfail:0   fail:0   skip:39  time:566s
fi-snb-2600      total:289  pass:249  dwarn:0   dfail:0   fail:0   skip:40  time:413s

5e39e3539f6330cf7d08cc83d34bc693f0be8920 drm-tip: 2017y-10m-25d-13h-19m-05s UTC integration manifest
f44ab6a666e2 drm/i915: Enhanced for initialize partially filled pagetables

== Logs ==

For more details see: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_6183/
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx