Questions about some PFB registers on NVAC cards

Questions about some PFB registers on NVAC cards

[pierre.morrow-GANU6spQydw]

(Sending it to the correct Nvidia mailing list, sorry for the spam)

Hi,

When using acceleration with Nouveau on MacBook Pros with an 9400M (NVAC) card,
a PFIFO interrupt 0x00400000 is thrown during the initialisation of that card
(sometime after PFIFO and PGRAPH initialisation) and the laptop will lockup [1],
forcing users to load Nouveau without acceleration.

After some investigation, I found that enabling bit 1 of register 100c14 fixes
the issue on that card. Other NVAC cards are working great without that trick,
and it seems they have that bit enabled by default. What is the role of that
bit, and when should it be turned on?

Before enabling 100c14's bit 1, the Nvidia driver writes some value into 100c1c.
Leaving the default value, or writing some random value seemed to have no
effect. What is this register used for?

Thanks in advance for your help!

Best regards,

Pierre Moreau


[1]: https://bugs.freedesktop.org/show_bug.cgi?id=27501

Re: Questions about some PFB registers on NVAC cards

[Robert Morell]

On Wed, Oct 22, 2014 at 12:55:23AM +0200, pierre.morrow@free.fr wrote:
[...]
> After some investigation, I found that enabling bit 1 of register 100c14 fixes
> the issue on that card. Other NVAC cards are working great without that trick,
> and it seems they have that bit enabled by default. What is the role of that
> bit, and when should it be turned on?

Register 100c14 controls a feature of the integrated GPU's memory interface
called the "poller".  The pollers exist on MCP77/78 and MCP79/7A to ensure
that certain types of writes have flushed all the way to system memory.  (I
think this is needed to ensure proper ordering of memory transactions.)

There are actually three important bits in that register, which control three
pollers:

NV_PFB_NISO_POLLER_CFG                                                0x00100c14
NV_PFB_NISO_POLLER_CFG_DNISO_ENABLE                                          0:0
NV_PFB_NISO_POLLER_CFG_DNISO_ENABLE_DISABLED                          0x00000000
NV_PFB_NISO_POLLER_CFG_DNISO_ENABLE_ENABLED                           0x00000001
NV_PFB_NISO_POLLER_CFG_HOSTNB_ENABLE                                         1:1
NV_PFB_NISO_POLLER_CFG_HOSTNB_ENABLE_DISABLED                         0x00000000
NV_PFB_NISO_POLLER_CFG_HOSTNB_ENABLE_ENABLED                          0x00000001
NV_PFB_NISO_POLLER_CFG_FLUSH_ENABLE                                        16:16
NV_PFB_NISO_POLLER_CFG_FLUSH_ENABLE_DISABLED                          0x00000000
NV_PFB_NISO_POLLER_CFG_FLUSH_ENABLE_ENABLED                           0x00000001

> Before enabling 100c14's bit 1, the Nvidia driver writes some value into 100c1c.
> Leaving the default value, or writing some random value seemed to have no
> effect. What is this register used for?

100c1c is one of three registers which control the (upper bits of the 32-byte
aligned) memory locations that the pollers use:

#define NV_PFB_NISO_POLLER_DNISO_BASE_ADR                     0x00100C18
#define NV_PFB_NISO_POLLER_HOSTNB_BASE_ADR                    0x00100C1C
#define NV_PFB_NISO_FLUSH_CARVEOUT_ADR                        0x00100C24

Each of these should point to at least 32 bytes of otherwise-unused FB
memory, if the poller is enabled.

The proprietary driver enables all three pollers for GPUs that have them, when
memory is not local (i.e., when using a sysmem carveout rather than dedicated
video memory).

I'm not sure why this seems to only be necessary on some systems and not
others.

- Robert
_______________________________________________
Nouveau mailing list
Nouveau@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/nouveau

Re: Questions about some PFB registers on NVAC cards

[Pierre Moreau]

Hi Robert,

Thanks a lot for your response!

----- Mail original -----
> On Wed, Oct 22, 2014 at 12:55:23AM +0200, pierre.morrow@free.fr
> wrote:
> [...]
> > After some investigation, I found that enabling bit 1 of register
> > 100c14 fixes
> > the issue on that card. Other NVAC cards are working great without
> > that trick,
> > and it seems they have that bit enabled by default. What is the
> > role of that
> > bit, and when should it be turned on?
> 
> Register 100c14 controls a feature of the integrated GPU's memory
> interface
> called the "poller".  The pollers exist on MCP77/78 and MCP79/7A to
> ensure
> that certain types of writes have flushed all the way to system
> memory.  (I
> think this is needed to ensure proper ordering of memory
> transactions.)
> 
> There are actually three important bits in that register, which
> control three
> pollers:
> 
> NV_PFB_NISO_POLLER_CFG
>                                                0x00100c14
> NV_PFB_NISO_POLLER_CFG_DNISO_ENABLE
>                                          0:0
> NV_PFB_NISO_POLLER_CFG_DNISO_ENABLE_DISABLED
>                          0x00000000
> NV_PFB_NISO_POLLER_CFG_DNISO_ENABLE_ENABLED
>                           0x00000001
> NV_PFB_NISO_POLLER_CFG_HOSTNB_ENABLE
>                                         1:1
> NV_PFB_NISO_POLLER_CFG_HOSTNB_ENABLE_DISABLED
>                         0x00000000
> NV_PFB_NISO_POLLER_CFG_HOSTNB_ENABLE_ENABLED
>                          0x00000001
> NV_PFB_NISO_POLLER_CFG_FLUSH_ENABLE
>                                        16:16
> NV_PFB_NISO_POLLER_CFG_FLUSH_ENABLE_DISABLED
>                          0x00000000
> NV_PFB_NISO_POLLER_CFG_FLUSH_ENABLE_ENABLED
>                           0x00000001
> 
> > Before enabling 100c14's bit 1, the Nvidia driver writes some value
> > into 100c1c.
> > Leaving the default value, or writing some random value seemed to
> > have no
> > effect. What is this register used for?
> 
> 100c1c is one of three registers which control the (upper bits of the
> 32-byte
> aligned) memory locations that the pollers use:
> 
> #define NV_PFB_NISO_POLLER_DNISO_BASE_ADR
>                     0x00100C18
> #define NV_PFB_NISO_POLLER_HOSTNB_BASE_ADR
>                    0x00100C1C
> #define NV_PFB_NISO_FLUSH_CARVEOUT_ADR
>                        0x00100C24
> 
> Each of these should point to at least 32 bytes of otherwise-unused
> FB
> memory, if the poller is enabled.
> 
> The proprietary driver enables all three pollers for GPUs that have
> them, when
> memory is not local (i.e., when using a sysmem carveout rather than
> dedicated
> video memory).
> 
> I'm not sure why this seems to only be necessary on some systems and
> not
> others.

I didn't meant that it wasn't necessary on other systems, just that it was enabled by default on those systems so they would work even if Nouveau didn't set the pollers.

> 
> - Robert
> 

Regards,

Pierre
_______________________________________________
Nouveau mailing list
Nouveau@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/nouveau

Re: Questions about some PFB registers on NVAC cards

[Pierre Moreau]

Hi Robert,

After trying to write a proper patch for Nouveau, we realised we were missing a few pieces of information.

----- Mail original -----
> Hi Robert,
>
> Thanks a lot for your response!
>
> ----- Mail original -----
> > On Wed, Oct 22, 2014 at 12:55:23AM +0200, pierre.morrow@free.fr
> > wrote:
> > [...]
> > > After some investigation, I found that enabling bit 1 of register
> > > 100c14 fixes
> > > the issue on that card. Other NVAC cards are working great
> > > without
> > > that trick,
> > > and it seems they have that bit enabled by default. What is the
> > > role of that
> > > bit, and when should it be turned on?
> >
> > Register 100c14 controls a feature of the integrated GPU's memory
> > interface
> > called the "poller".  The pollers exist on MCP77/78 and MCP79/7A to
> > ensure
> > that certain types of writes have flushed all the way to system
> > memory.  (I
> > think this is needed to ensure proper ordering of memory
> > transactions.)
> >
> > There are actually three important bits in that register, which
> > control three
> > pollers:
> >
> > NV_PFB_NISO_POLLER_CFG
> >                                                0x00100c14
> > NV_PFB_NISO_POLLER_CFG_DNISO_ENABLE
> >                                          0:0
> > NV_PFB_NISO_POLLER_CFG_DNISO_ENABLE_DISABLED
> >                          0x00000000
> > NV_PFB_NISO_POLLER_CFG_DNISO_ENABLE_ENABLED
> >                           0x00000001
> > NV_PFB_NISO_POLLER_CFG_HOSTNB_ENABLE
> >                                         1:1
> > NV_PFB_NISO_POLLER_CFG_HOSTNB_ENABLE_DISABLED
> >                         0x00000000
> > NV_PFB_NISO_POLLER_CFG_HOSTNB_ENABLE_ENABLED
> >                          0x00000001
> > NV_PFB_NISO_POLLER_CFG_FLUSH_ENABLE
> >                                        16:16
> > NV_PFB_NISO_POLLER_CFG_FLUSH_ENABLE_DISABLED
> >                          0x00000000
> > NV_PFB_NISO_POLLER_CFG_FLUSH_ENABLE_ENABLED
> >                           0x00000001

Out of curiosity, what do DNISO and HOSTNB mean?

> >
> > > Before enabling 100c14's bit 1, the Nvidia driver writes some
> > > value
> > > into 100c1c.
> > > Leaving the default value, or writing some random value seemed to
> > > have no
> > > effect. What is this register used for?
> >
> > 100c1c is one of three registers which control the (upper bits of
> > the
> > 32-byte
> > aligned) memory locations that the pollers use:

So, as the memory location is 32-byte aligned the last 8 bits of the address are uncessary. 100c1c and the others hold 32 bits. So are the missing 24 bits of the address forced to zero or do we need to specify them somewhere?

Here is the patch so far, where priv->r100c08 is an address to an empty dma page:

    diff --git a/nvkm/subdev/fb/nv50.c b/nvkm/subdev/fb/nv50.c
    index 4150b0d..1a0a94e 100644
    --- a/nvkm/subdev/fb/nv50.c
    +++ b/nvkm/subdev/fb/nv50.c
    @@ -295,6 +296,19 @@ nv50_fb_init(struct nouveau_object *object)
             */
            nv_wr32(priv, 0x100c08, priv->r100c08 >> 8);
    
    +       /* Enable NISO poller for various clients and set their associated
    +        * read address, only for MCP77/78 and MCP79/7A. (fd#25701)
    +        */
    +       if ((nv_device(priv)->chipset == 0xaa || nv_device(priv)->chipset == 0xac)
    +                       && !nv_rd32(priv, 0x100c14)) {
    +               nv_wr32(priv, 0x100c18, priv->r100c08 >> 8);
    +               nv_mask(priv, 0x100c14, 0x00000000, 0x00000001);
    +               nv_wr32(priv, 0x100c1c, priv->r100c08 >> 8);
    +               nv_mask(priv, 0x100c14, 0x00000000, 0x00000002);
    +               nv_wr32(priv, 0x100c24, priv->r100c08 >> 8);
    +               nv_mask(priv, 0x100c14, 0x00000000, 0x00010000);
    +       }
    +
            /* This is needed to get meaningful information from 100c90
             * on traps. No idea what these values mean exactly. */
            nv_wr32(priv, 0x100c90, impl->trap);

Thanks,

Pierre


> >
> > #define NV_PFB_NISO_POLLER_DNISO_BASE_ADR
> >                     0x00100C18
> > #define NV_PFB_NISO_POLLER_HOSTNB_BASE_ADR
> >                    0x00100C1C
> > #define NV_PFB_NISO_FLUSH_CARVEOUT_ADR
> >                        0x00100C24
> >
> > Each of these should point to at least 32 bytes of otherwise-unused
> > FB
> > memory, if the poller is enabled.
> >
> > The proprietary driver enables all three pollers for GPUs that have
> > them, when
> > memory is not local (i.e., when using a sysmem carveout rather than
> > dedicated
> > video memory).
> >
> > I'm not sure why this seems to only be necessary on some systems
> > and
> > not
> > others.
>
> I didn't meant that it wasn't necessary on other systems, just that
> it was enabled by default on those systems so they would work even
> if Nouveau didn't set the pollers.
>
> >
> > - Robert
> >
>
> Regards,
>
> Pierre
> _______________________________________________
> Nouveau mailing list
> Nouveau@lists.freedesktop.org
> http://lists.freedesktop.org/mailman/listinfo/nouveau
> 
_______________________________________________
Nouveau mailing list
Nouveau@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/nouveau

Re: Questions about some PFB registers on NVAC cards

[Robert Morell]

On Mon, Dec 01, 2014 at 08:08:22PM +0100, Pierre Moreau wrote:
> Hi Robert,
> 
> After trying to write a proper patch for Nouveau, we realised we were missing a few pieces of information.
> 
> ----- Mail original -----
> > Hi Robert,
> >
> > Thanks a lot for your response!
> >
> > ----- Mail original -----
> > > On Wed, Oct 22, 2014 at 12:55:23AM +0200, pierre.morrow@free.fr
> > > wrote:
> > > [...]
> > > > After some investigation, I found that enabling bit 1 of register 100c14 fixes
> > > > the issue on that card. Other NVAC cards are working great without that trick,
> > > > and it seems they have that bit enabled by default. What is the role of that
> > > > bit, and when should it be turned on?
> > >
> > > Register 100c14 controls a feature of the integrated GPU's memory interface
> > > called the "poller".  The pollers exist on MCP77/78 and MCP79/7A to ensure
> > > that certain types of writes have flushed all the way to system memory.  (I
> > > think this is needed to ensure proper ordering of memory transactions.)
> > >
> > > There are actually three important bits in that register, which
> > > control three pollers:
> > >
> > > NV_PFB_NISO_POLLER_CFG                                                0x00100c14
> > > NV_PFB_NISO_POLLER_CFG_DNISO_ENABLE                                          0:0
> > > NV_PFB_NISO_POLLER_CFG_DNISO_ENABLE_DISABLED                          0x00000000
> > > NV_PFB_NISO_POLLER_CFG_DNISO_ENABLE_ENABLED                           0x00000001
> > > NV_PFB_NISO_POLLER_CFG_HOSTNB_ENABLE                                         1:1
> > > NV_PFB_NISO_POLLER_CFG_HOSTNB_ENABLE_DISABLED                         0x00000000
> > > NV_PFB_NISO_POLLER_CFG_HOSTNB_ENABLE_ENABLED                          0x00000001
> > > NV_PFB_NISO_POLLER_CFG_FLUSH_ENABLE                                        16:16
> > > NV_PFB_NISO_POLLER_CFG_FLUSH_ENABLE_DISABLED                          0x00000000
> > > NV_PFB_NISO_POLLER_CFG_FLUSH_ENABLE_ENABLED                           0x00000001
> 
> Out of curiosity, what do DNISO and HOSTNB mean?

DNISO is "display non-iso" (i.e., display reads or writes that don't
need hard realtime guarantees for scanout, like semaphore accesses).

I don't know what HOSTNB stands for, sorry.

> > > > Before enabling 100c14's bit 1, the Nvidia driver writes some value into 100c1c.
> > > > Leaving the default value, or writing some random value seemed to have no
> > > > effect. What is this register used for?
> > >
> > > 100c1c is one of three registers which control the (upper bits of the 32-byte
> > > aligned) memory locations that the pollers use:
> 
> So, as the memory location is 32-byte aligned the last 8 bits of the
> address are uncessary. 100c1c and the others hold 32 bits. So are the
> missing 24 bits of the address forced to zero or do we need to specify
> them somewhere?

I should have been more specific about how these registers are
interpreted.  The hardware will calculate the physical address by
subtracting from the top of carveout.  If I'm reading this correctly,
the equation involving 100c18, for example, is:

physical address = carveout_base + carveout_size - ((r100c18 + 1) * 32)

where:
carveout_base is (r100e10 << 12)
carveout_size is (r100e14 << 12)
(and "r100e10" means "the value in register 100e10")

> Here is the patch so far, where priv->r100c08 is an address to an empty dma page:
> 
>     diff --git a/nvkm/subdev/fb/nv50.c b/nvkm/subdev/fb/nv50.c
>     index 4150b0d..1a0a94e 100644
>     --- a/nvkm/subdev/fb/nv50.c
>     +++ b/nvkm/subdev/fb/nv50.c
>     @@ -295,6 +296,19 @@ nv50_fb_init(struct nouveau_object *object)
>              */
>             nv_wr32(priv, 0x100c08, priv->r100c08 >> 8);
>     
>     +       /* Enable NISO poller for various clients and set their associated
>     +        * read address, only for MCP77/78 and MCP79/7A. (fd#25701)
>     +        */
>     +       if ((nv_device(priv)->chipset == 0xaa || nv_device(priv)->chipset == 0xac)
>     +                       && !nv_rd32(priv, 0x100c14)) {
>     +               nv_wr32(priv, 0x100c18, priv->r100c08 >> 8);
>     +               nv_mask(priv, 0x100c14, 0x00000000, 0x00000001);
>     +               nv_wr32(priv, 0x100c1c, priv->r100c08 >> 8);

I think you'll want to use different unique offsets for 100c1c and
100c14, rather than reusing the same one.

- Robert

>     +               nv_mask(priv, 0x100c14, 0x00000000, 0x00000002);
>     +               nv_wr32(priv, 0x100c24, priv->r100c08 >> 8);
>     +               nv_mask(priv, 0x100c14, 0x00000000, 0x00010000);
>     +       }
>     +
>             /* This is needed to get meaningful information from 100c90
>              * on traps. No idea what these values mean exactly. */
>             nv_wr32(priv, 0x100c90, impl->trap);
> 
> Thanks,
> 
> Pierre
> 
> 
> > >
> > > #define NV_PFB_NISO_POLLER_DNISO_BASE_ADR
> > >                     0x00100C18
> > > #define NV_PFB_NISO_POLLER_HOSTNB_BASE_ADR
> > >                    0x00100C1C
> > > #define NV_PFB_NISO_FLUSH_CARVEOUT_ADR
> > >                        0x00100C24
> > >
> > > Each of these should point to at least 32 bytes of otherwise-unused
> > > FB
> > > memory, if the poller is enabled.
> > >
> > > The proprietary driver enables all three pollers for GPUs that have
> > > them, when
> > > memory is not local (i.e., when using a sysmem carveout rather than
> > > dedicated
> > > video memory).
> > >
> > > I'm not sure why this seems to only be necessary on some systems
> > > and
> > > not
> > > others.
> >
> > I didn't meant that it wasn't necessary on other systems, just that
> > it was enabled by default on those systems so they would work even
> > if Nouveau didn't set the pollers.
> >
> > >
> > > - Robert
> > >
> >
> > Regards,
> >
> > Pierre
> > _______________________________________________
> > Nouveau mailing list
> > Nouveau@lists.freedesktop.org
> > http://lists.freedesktop.org/mailman/listinfo/nouveau
> > 
> 
_______________________________________________
Nouveau mailing list
Nouveau@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/nouveau

Questions about some PFB registers on NVAC cards

[pierre.morrow-GANU6spQydw]

Hi,

When using acceleration with Nouveau on MacBook Pros with an 9400M (NVAC) card,
a PFIFO interrupt 0x00400000 is thrown during the initialisation of that card
(sometime after PFIFO and PGRAPH initialisation) and the laptop will lockup [1],
forcing users to load Nouveau without acceleration.

After some investigation, I found that enabling bit 1 of register 100c14 fixes
the issue on that card. Other NVAC cards are working great without that trick,
and it seems they have that bit enabled by default. What is the role of that
bit, and when should it be turned on?

Before enabling 100c14's bit 1, the Nvidia driver writes some value into 100c1c.
Leaving the default value, or writing some random value seemed to have no
effect. What is this register used for?

Thanks in advance for your help!

Best regards,

Pierre Moreau


[1]: https://bugs.freedesktop.org/show_bug.cgi?id=27501