[PATCH] drm/i915: enable semaphores on gen6 if dmar is not active

[PATCH] drm/i915: enable semaphores on gen6 if dmar is not active

[Daniel Vetter]

Inspired by the recent ppgtt regression report, where switching of
dmar only for the gpu seems to fix things completely, I've looked
again at the semaphores+vt-d situation.

Contrary to my earlier testing a few months back my system is now
stable with dmar disabled for the igd, and not only when disabling
dmar completely.

So I'm rather hopeful that all our recent fixes for snb have changed
things for code and it's time to try enabling semaphores again. We've
also had issues with enabling semaphores which are not vt-d related,
but I guess these are all fixed by the autoreport-disabling and lazy
request fix. And there's only one way to find out whether there are
still other issues ...

Signed-Off-by: Daniel Vetter <daniel.vetter@ffwll.ch>

---

If no further vt-d regressions show up in the 3.4 cycle I plan to
merge this into -next for 3.5 (in a month or so). Comments about how
unfeasibly you deem this highly welcome.
-Daniel
---
 drivers/gpu/drm/i915/i915_gem_execbuffer.c |    8 +++++---
 1 files changed, 5 insertions(+), 3 deletions(-)

diff --git a/drivers/gpu/drm/i915/i915_gem_execbuffer.c b/drivers/gpu/drm/i915/i915_gem_execbuffer.c
index 8e0b686..ac52433 100644
--- a/drivers/gpu/drm/i915/i915_gem_execbuffer.c
+++ b/drivers/gpu/drm/i915/i915_gem_execbuffer.c
@@ -849,9 +849,11 @@ intel_enable_semaphores(struct drm_device *dev)
 	if (i915_semaphores >= 0)
 		return i915_semaphores;
 
-	/* Disable semaphores on SNB */
-	if (INTEL_INFO(dev)->gen == 6)
-		return 0;
+#ifdef CONFIG_INTEL_IOMMU
+	/* Disable semaphores on SNB if VT-d is on. */
+	if (INTEL_INFO(dev)->gen == 6 && intel_iommu_gfx_mapped)
+		return false;
+#endif
 
 	return 1;
 }
-- 
1.7.7.6

Re: [Intel-gfx] [PATCH] drm/i915: enable semaphores on gen6 if dmar is not active

[Andrew Lutomirski]

On Mon, Apr 2, 2012 at 11:48 AM, Daniel Vetter <daniel.vetter@ffwll.ch> wrote:
> Inspired by the recent ppgtt regression report, where switching of
> dmar only for the gpu seems to fix things completely, I've looked
> again at the semaphores+vt-d situation.
>
> Contrary to my earlier testing a few months back my system is now
> stable with dmar disabled for the igd, and not only when disabling
> dmar completely.
>
> So I'm rather hopeful that all our recent fixes for snb have changed
> things for code and it's time to try enabling semaphores again. We've
> also had issues with enabling semaphores which are not vt-d related,
> but I guess these are all fixed by the autoreport-disabling and lazy
> request fix. And there's only one way to find out whether there are
> still other issues ...
>
> Signed-Off-by: Daniel Vetter <daniel.vetter@ffwll.ch>
>
> ---
>
> If no further vt-d regressions show up in the 3.4 cycle I plan to
> merge this into -next for 3.5 (in a month or so). Comments about how
> unfeasibly you deem this highly welcome.
> -Daniel
> ---
>  drivers/gpu/drm/i915/i915_gem_execbuffer.c |    8 +++++---
>  1 files changed, 5 insertions(+), 3 deletions(-)
>
> diff --git a/drivers/gpu/drm/i915/i915_gem_execbuffer.c b/drivers/gpu/drm/i915/i915_gem_execbuffer.c
> index 8e0b686..ac52433 100644
> --- a/drivers/gpu/drm/i915/i915_gem_execbuffer.c
> +++ b/drivers/gpu/drm/i915/i915_gem_execbuffer.c
> @@ -849,9 +849,11 @@ intel_enable_semaphores(struct drm_device *dev)
>        if (i915_semaphores >= 0)
>                return i915_semaphores;
>
> -       /* Disable semaphores on SNB */
> -       if (INTEL_INFO(dev)->gen == 6)
> -               return 0;
> +#ifdef CONFIG_INTEL_IOMMU
> +       /* Disable semaphores on SNB if VT-d is on. */
> +       if (INTEL_INFO(dev)->gen == 6 && intel_iommu_gfx_mapped)
> +               return false;

It might be nice to have a printk here giving instructions for how to work
around this.  For example:

i915: Disabling semaphores to work around a DMAR issue.  As an
alternative, boot with intel_iommu=igfx_off [or on, igfx_off, or
whatever it's supposed to be].

The documentation in kernel-parameters.txt is at best unclear to the
uninitiated.

--Andy

Re: [PATCH] drm/i915: enable semaphores on gen6 if dmar is not active

[Daniel Vetter]

On Mon, Apr 02, 2012 at 01:45:39PM -0700, Andrew Lutomirski wrote:
> On Mon, Apr 2, 2012 at 11:48 AM, Daniel Vetter <daniel.vetter@ffwll.ch> wrote:
> > Inspired by the recent ppgtt regression report, where switching of
> > dmar only for the gpu seems to fix things completely, I've looked
> > again at the semaphores+vt-d situation.
> >
> > Contrary to my earlier testing a few months back my system is now
> > stable with dmar disabled for the igd, and not only when disabling
> > dmar completely.
> >
> > So I'm rather hopeful that all our recent fixes for snb have changed
> > things for code and it's time to try enabling semaphores again. We've
> > also had issues with enabling semaphores which are not vt-d related,
> > but I guess these are all fixed by the autoreport-disabling and lazy
> > request fix. And there's only one way to find out whether there are
> > still other issues ...
> >
> > Signed-Off-by: Daniel Vetter <daniel.vetter@ffwll.ch>
> >
> > ---
> >
> > If no further vt-d regressions show up in the 3.4 cycle I plan to
> > merge this into -next for 3.5 (in a month or so). Comments about how
> > unfeasibly you deem this highly welcome.
> > -Daniel
> > ---
> >  drivers/gpu/drm/i915/i915_gem_execbuffer.c |    8 +++++---
> >  1 files changed, 5 insertions(+), 3 deletions(-)
> >
> > diff --git a/drivers/gpu/drm/i915/i915_gem_execbuffer.c b/drivers/gpu/drm/i915/i915_gem_execbuffer.c
> > index 8e0b686..ac52433 100644
> > --- a/drivers/gpu/drm/i915/i915_gem_execbuffer.c
> > +++ b/drivers/gpu/drm/i915/i915_gem_execbuffer.c
> > @@ -849,9 +849,11 @@ intel_enable_semaphores(struct drm_device *dev)
> >        if (i915_semaphores >= 0)
> >                return i915_semaphores;
> >
> > -       /* Disable semaphores on SNB */
> > -       if (INTEL_INFO(dev)->gen == 6)
> > -               return 0;
> > +#ifdef CONFIG_INTEL_IOMMU
> > +       /* Disable semaphores on SNB if VT-d is on. */
> > +       if (INTEL_INFO(dev)->gen == 6 && intel_iommu_gfx_mapped)
> > +               return false;
> 
> It might be nice to have a printk here giving instructions for how to work
> around this.  For example:
> 
> i915: Disabling semaphores to work around a DMAR issue.  As an
> alternative, boot with intel_iommu=igfx_off [or on, igfx_off, or
> whatever it's supposed to be].
> 
> The documentation in kernel-parameters.txt is at best unclear to the
> uninitiated.

If this really does work, I'll look into this. Atm it's still unclear
where to exactly put the plain, we need to annoy internal hardware people
to analyze this. Once we have enough evidence that the combination of gpu
with various features enable plus VT-d just doesn't seem to work reliably.

So can you check whether things do indeed work with this patch, atop
kernel 3.4-rc1? Iirc you've been the one with the most trouble when
semaphores are enabled ...

Yours, Daniel
-- 
Daniel Vetter
Mail: daniel@ffwll.ch
Mobile: +41 (0)79 365 57 48

Re: [PATCH] drm/i915: enable semaphores on gen6 if dmar is not active

[Andrew Lutomirski]

On Mon, Apr 2, 2012 at 1:52 PM, Daniel Vetter <daniel@ffwll.ch> wrote:
> On Mon, Apr 02, 2012 at 01:45:39PM -0700, Andrew Lutomirski wrote:
>> On Mon, Apr 2, 2012 at 11:48 AM, Daniel Vetter <daniel.vetter@ffwll.ch> wrote:
>> > Inspired by the recent ppgtt regression report, where switching of
>> > dmar only for the gpu seems to fix things completely, I've looked
>> > again at the semaphores+vt-d situation.
>> >
>> > Contrary to my earlier testing a few months back my system is now
>> > stable with dmar disabled for the igd, and not only when disabling
>> > dmar completely.
>> >
>> > So I'm rather hopeful that all our recent fixes for snb have changed
>> > things for code and it's time to try enabling semaphores again. We've
>> > also had issues with enabling semaphores which are not vt-d related,
>> > but I guess these are all fixed by the autoreport-disabling and lazy
>> > request fix. And there's only one way to find out whether there are
>> > still other issues ...
>> >
>> > Signed-Off-by: Daniel Vetter <daniel.vetter@ffwll.ch>
>> >
>> > ---
>> >
>> > If no further vt-d regressions show up in the 3.4 cycle I plan to
>> > merge this into -next for 3.5 (in a month or so). Comments about how
>> > unfeasibly you deem this highly welcome.
>> > -Daniel
>> > ---
>> >  drivers/gpu/drm/i915/i915_gem_execbuffer.c |    8 +++++---
>> >  1 files changed, 5 insertions(+), 3 deletions(-)
>> >
>> > diff --git a/drivers/gpu/drm/i915/i915_gem_execbuffer.c b/drivers/gpu/drm/i915/i915_gem_execbuffer.c
>> > index 8e0b686..ac52433 100644
>> > --- a/drivers/gpu/drm/i915/i915_gem_execbuffer.c
>> > +++ b/drivers/gpu/drm/i915/i915_gem_execbuffer.c
>> > @@ -849,9 +849,11 @@ intel_enable_semaphores(struct drm_device *dev)
>> >        if (i915_semaphores >= 0)
>> >                return i915_semaphores;
>> >
>> > -       /* Disable semaphores on SNB */
>> > -       if (INTEL_INFO(dev)->gen == 6)
>> > -               return 0;
>> > +#ifdef CONFIG_INTEL_IOMMU
>> > +       /* Disable semaphores on SNB if VT-d is on. */
>> > +       if (INTEL_INFO(dev)->gen == 6 && intel_iommu_gfx_mapped)
>> > +               return false;
>>
>> It might be nice to have a printk here giving instructions for how to work
>> around this.  For example:
>>
>> i915: Disabling semaphores to work around a DMAR issue.  As an
>> alternative, boot with intel_iommu=igfx_off [or on, igfx_off, or
>> whatever it's supposed to be].
>>
>> The documentation in kernel-parameters.txt is at best unclear to the
>> uninitiated.
>
> If this really does work, I'll look into this. Atm it's still unclear
> where to exactly put the plain, we need to annoy internal hardware people
> to analyze this. Once we have enough evidence that the combination of gpu
> with various features enable plus VT-d just doesn't seem to work reliably.
>
> So can you check whether things do indeed work with this patch, atop
> kernel 3.4-rc1? Iirc you've been the one with the most trouble when
> semaphores are enabled ...

I've had a hard time reproducing the problems lately.  The
unconditional instant hard hang went away a few months ago, I think.
I'll give it another shot when I get home to that machine.

--Andy

Re: [Intel-gfx] [PATCH] drm/i915: enable semaphores on gen6 if dmar is not active

[Daniel Vetter]

On Mon, Apr 02, 2012 at 02:44:14PM -0700, Andrew Lutomirski wrote:
> On Mon, Apr 2, 2012 at 1:52 PM, Daniel Vetter <daniel@ffwll.ch> wrote:
> > On Mon, Apr 02, 2012 at 01:45:39PM -0700, Andrew Lutomirski wrote:
> >> On Mon, Apr 2, 2012 at 11:48 AM, Daniel Vetter <daniel.vetter@ffwll.ch> wrote:
> >> > Inspired by the recent ppgtt regression report, where switching of
> >> > dmar only for the gpu seems to fix things completely, I've looked
> >> > again at the semaphores+vt-d situation.
> >> >
> >> > Contrary to my earlier testing a few months back my system is now
> >> > stable with dmar disabled for the igd, and not only when disabling
> >> > dmar completely.
> >> >
> >> > So I'm rather hopeful that all our recent fixes for snb have changed
> >> > things for code and it's time to try enabling semaphores again. We've
> >> > also had issues with enabling semaphores which are not vt-d related,
> >> > but I guess these are all fixed by the autoreport-disabling and lazy
> >> > request fix. And there's only one way to find out whether there are
> >> > still other issues ...
> >> >
> >> > Signed-Off-by: Daniel Vetter <daniel.vetter@ffwll.ch>
> >> >
> >> > ---
> >> >
> >> > If no further vt-d regressions show up in the 3.4 cycle I plan to
> >> > merge this into -next for 3.5 (in a month or so). Comments about how
> >> > unfeasibly you deem this highly welcome.
> >> > -Daniel
> >> > ---
> >> >  drivers/gpu/drm/i915/i915_gem_execbuffer.c |    8 +++++---
> >> >  1 files changed, 5 insertions(+), 3 deletions(-)
> >> >
> >> > diff --git a/drivers/gpu/drm/i915/i915_gem_execbuffer.c b/drivers/gpu/drm/i915/i915_gem_execbuffer.c
> >> > index 8e0b686..ac52433 100644
> >> > --- a/drivers/gpu/drm/i915/i915_gem_execbuffer.c
> >> > +++ b/drivers/gpu/drm/i915/i915_gem_execbuffer.c
> >> > @@ -849,9 +849,11 @@ intel_enable_semaphores(struct drm_device *dev)
> >> >        if (i915_semaphores >= 0)
> >> >                return i915_semaphores;
> >> >
> >> > -       /* Disable semaphores on SNB */
> >> > -       if (INTEL_INFO(dev)->gen == 6)
> >> > -               return 0;
> >> > +#ifdef CONFIG_INTEL_IOMMU
> >> > +       /* Disable semaphores on SNB if VT-d is on. */
> >> > +       if (INTEL_INFO(dev)->gen == 6 && intel_iommu_gfx_mapped)
> >> > +               return false;
> >>
> >> It might be nice to have a printk here giving instructions for how to work
> >> around this.  For example:
> >>
> >> i915: Disabling semaphores to work around a DMAR issue.  As an
> >> alternative, boot with intel_iommu=igfx_off [or on, igfx_off, or
> >> whatever it's supposed to be].
> >>
> >> The documentation in kernel-parameters.txt is at best unclear to the
> >> uninitiated.
> >
> > If this really does work, I'll look into this. Atm it's still unclear
> > where to exactly put the plain, we need to annoy internal hardware people
> > to analyze this. Once we have enough evidence that the combination of gpu
> > with various features enable plus VT-d just doesn't seem to work reliably.
> >
> > So can you check whether things do indeed work with this patch, atop
> > kernel 3.4-rc1? Iirc you've been the one with the most trouble when
> > semaphores are enabled ...
> 
> I've had a hard time reproducing the problems lately.  The
> unconditional instant hard hang went away a few months ago, I think.
> I'll give it another shot when I get home to that machine.

Well, I've managed to kill my machine shortly after login with semaphores,
rc6 and dmar enabled. It hasn't died in the same configuration after a few
hours of light usage (in my case that seems to be the best killer) with
dmar disabled for just the gpu.

So if you can give your machine some serious beating with the different
options, that's really great.
-Daniel
-- 
Daniel Vetter
Mail: daniel@ffwll.ch
Mobile: +41 (0)79 365 57 48

Re: [Intel-gfx] [PATCH] drm/i915: enable semaphores on gen6 if dmar is not active

[Daniel Vetter]

On Tue, Apr 03, 2012 at 12:32:05AM +0200, Daniel Vetter wrote:
> On Mon, Apr 02, 2012 at 02:44:14PM -0700, Andrew Lutomirski wrote:
> > On Mon, Apr 2, 2012 at 1:52 PM, Daniel Vetter <daniel@ffwll.ch> wrote:
> > > On Mon, Apr 02, 2012 at 01:45:39PM -0700, Andrew Lutomirski wrote:
> > >> On Mon, Apr 2, 2012 at 11:48 AM, Daniel Vetter <daniel.vetter@ffwll.ch> wrote:
> > >> > Inspired by the recent ppgtt regression report, where switching of
> > >> > dmar only for the gpu seems to fix things completely, I've looked
> > >> > again at the semaphores+vt-d situation.
> > >> >
> > >> > Contrary to my earlier testing a few months back my system is now
> > >> > stable with dmar disabled for the igd, and not only when disabling
> > >> > dmar completely.
> > >> >
> > >> > So I'm rather hopeful that all our recent fixes for snb have changed
> > >> > things for code and it's time to try enabling semaphores again. We've
> > >> > also had issues with enabling semaphores which are not vt-d related,
> > >> > but I guess these are all fixed by the autoreport-disabling and lazy
> > >> > request fix. And there's only one way to find out whether there are
> > >> > still other issues ...
> > >> >
> > >> > Signed-Off-by: Daniel Vetter <daniel.vetter@ffwll.ch>
> > >> >
> > >> > ---
> > >> >
> > >> > If no further vt-d regressions show up in the 3.4 cycle I plan to
> > >> > merge this into -next for 3.5 (in a month or so). Comments about how
> > >> > unfeasibly you deem this highly welcome.
> > >> > -Daniel
> > >> > ---
> > >> >  drivers/gpu/drm/i915/i915_gem_execbuffer.c |    8 +++++---
> > >> >  1 files changed, 5 insertions(+), 3 deletions(-)
> > >> >
> > >> > diff --git a/drivers/gpu/drm/i915/i915_gem_execbuffer.c b/drivers/gpu/drm/i915/i915_gem_execbuffer.c
> > >> > index 8e0b686..ac52433 100644
> > >> > --- a/drivers/gpu/drm/i915/i915_gem_execbuffer.c
> > >> > +++ b/drivers/gpu/drm/i915/i915_gem_execbuffer.c
> > >> > @@ -849,9 +849,11 @@ intel_enable_semaphores(struct drm_device *dev)
> > >> >        if (i915_semaphores >= 0)
> > >> >                return i915_semaphores;
> > >> >
> > >> > -       /* Disable semaphores on SNB */
> > >> > -       if (INTEL_INFO(dev)->gen == 6)
> > >> > -               return 0;
> > >> > +#ifdef CONFIG_INTEL_IOMMU
> > >> > +       /* Disable semaphores on SNB if VT-d is on. */
> > >> > +       if (INTEL_INFO(dev)->gen == 6 && intel_iommu_gfx_mapped)
> > >> > +               return false;
> > >>
> > >> It might be nice to have a printk here giving instructions for how to work
> > >> around this.  For example:
> > >>
> > >> i915: Disabling semaphores to work around a DMAR issue.  As an
> > >> alternative, boot with intel_iommu=igfx_off [or on, igfx_off, or
> > >> whatever it's supposed to be].
> > >>
> > >> The documentation in kernel-parameters.txt is at best unclear to the
> > >> uninitiated.
> > >
> > > If this really does work, I'll look into this. Atm it's still unclear
> > > where to exactly put the plain, we need to annoy internal hardware people
> > > to analyze this. Once we have enough evidence that the combination of gpu
> > > with various features enable plus VT-d just doesn't seem to work reliably.
> > >
> > > So can you check whether things do indeed work with this patch, atop
> > > kernel 3.4-rc1? Iirc you've been the one with the most trouble when
> > > semaphores are enabled ...
> > 
> > I've had a hard time reproducing the problems lately.  The
> > unconditional instant hard hang went away a few months ago, I think.
> > I'll give it another shot when I get home to that machine.
> 
> Well, I've managed to kill my machine shortly after login with semaphores,
> rc6 and dmar enabled. It hasn't died in the same configuration after a few
> hours of light usage (in my case that seems to be the best killer) with
> dmar disabled for just the gpu.
> 
> So if you can give your machine some serious beating with the different
> options, that's really great.

Do you already have some preliminary results or has your machine not yet
died with this patch applied?

Thanks, Daniel
-- 
Daniel Vetter
Mail: daniel@ffwll.ch
Mobile: +41 (0)79 365 57 48

Re: [PATCH] drm/i915: enable semaphores on gen6 if dmar is not active

[Daniel Vetter]

On Tue, Apr 17, 2012 at 11:31:00AM +0200, Daniel Vetter wrote:
> On Tue, Apr 03, 2012 at 12:32:05AM +0200, Daniel Vetter wrote:
> > On Mon, Apr 02, 2012 at 02:44:14PM -0700, Andrew Lutomirski wrote:
> > > On Mon, Apr 2, 2012 at 1:52 PM, Daniel Vetter <daniel@ffwll.ch> wrote:
> > > > On Mon, Apr 02, 2012 at 01:45:39PM -0700, Andrew Lutomirski wrote:
> > > I've had a hard time reproducing the problems lately.  The
> > > unconditional instant hard hang went away a few months ago, I think.
> > > I'll give it another shot when I get home to that machine.
> > 
> > Well, I've managed to kill my machine shortly after login with semaphores,
> > rc6 and dmar enabled. It hasn't died in the same configuration after a few
> > hours of light usage (in my case that seems to be the best killer) with
> > dmar disabled for just the gpu.
> > 
> > So if you can give your machine some serious beating with the different
> > options, that's really great.
> 
> Do you already have some preliminary results or has your machine not yet
> died with this patch applied?

Ping?
-Daniel
-- 
Daniel Vetter
Mail: daniel@ffwll.ch
Mobile: +41 (0)79 365 57 48

Re: [PATCH] drm/i915: enable semaphores on gen6 if dmar is not active

[Daniel Vetter]

On Fri, Apr 27, 2012 at 07:54:59PM +0200, Daniel Vetter wrote:
> On Tue, Apr 17, 2012 at 11:31:00AM +0200, Daniel Vetter wrote:
> > On Tue, Apr 03, 2012 at 12:32:05AM +0200, Daniel Vetter wrote:
> > > On Mon, Apr 02, 2012 at 02:44:14PM -0700, Andrew Lutomirski wrote:
> > > > On Mon, Apr 2, 2012 at 1:52 PM, Daniel Vetter <daniel@ffwll.ch> wrote:
> > > > > On Mon, Apr 02, 2012 at 01:45:39PM -0700, Andrew Lutomirski wrote:
> > > > I've had a hard time reproducing the problems lately.  The
> > > > unconditional instant hard hang went away a few months ago, I think.
> > > > I'll give it another shot when I get home to that machine.
> > > 
> > > Well, I've managed to kill my machine shortly after login with semaphores,
> > > rc6 and dmar enabled. It hasn't died in the same configuration after a few
> > > hours of light usage (in my case that seems to be the best killer) with
> > > dmar disabled for just the gpu.
> > > 
> > > So if you can give your machine some serious beating with the different
> > > options, that's really great.
> > 
> > Do you already have some preliminary results or has your machine not yet
> > died with this patch applied?
> 
> Ping?

Even more ping. I'd really like to merge this, and a tested-by from you
would be awesome for this. Otherwise I'll just merge and we'll wait if it
actually blows up.
-Daniel
-- 
Daniel Vetter
Mail: daniel@ffwll.ch
Mobile: +41 (0)79 365 57 48