CONFIG_PPC_VAS depends on 64k pages...?

CONFIG_PPC_VAS depends on 64k pages...?

[Will Springer]

I learned about the POWER9 gzip accelerator a few months ago when the 
support hit upstream Linux 5.8. However, for some reason the Kconfig 
dictates that VAS depends on a 64k page size, which is problematic as I 
run Void Linux, which uses a 4k-page kernel.

Some early poking by others indicated there wasn't an obvious page size 
dependency in the code, and suggested I try modifying the config to switch 
it on. I did so, but was stopped by a minor complaint of an "unexpected DT 
configuration" by the VAS code. I wasn't equipped to figure out exactly what 
this meant, even after finding the offending condition, so after writing a 
very drawn-out forum post asking for help, I dropped the subject.

Fast forward to today, when I was reminded of the whole thing again, and 
decided to debug a bit further. Apparently the VAS platform device 
(derived from the DT node) has 5 resources on my 4k kernel, instead of 4 
(which evidently works for others who have had success on 64k kernels). I 
have no idea what this means in practice (I don't know how to introspect 
it), but after making a tiny patch[1], everything came up smoothly and I 
was doing blazing-fast gzip (de)compression in no time.

Everything seems to work fine on 4k pages. So, what's up? Are there 
pitfalls lurking around that I've yet to stumble over? More reasonably, 
I'm curious as to why the feature supposedly depends on 64k pages, or if 
there's anything else I should be concerned about.

I do have to say I'm quite satisfied with the results of the NX 
accelerator, though. Being able to shuffle data to a RaptorCS box over gigE 
and get compressed data back faster than most software gzip could ever
hope to achieve is no small feat, let alone the instantaneous results locally.
:)

Cheers,
Will Springer [she/her]

[1]: https://github.com/Skirmisher/void-packages/blob/vas-4k-pages/srcpkgs/linux5.9/patches/ppc-vas-on-4k.patch

Re: CONFIG_PPC_VAS depends on 64k pages...?

[Christophe Leroy]

Hi,

Le 19/11/2020 à 11:58, Will Springer a écrit :
> I learned about the POWER9 gzip accelerator a few months ago when the
> support hit upstream Linux 5.8. However, for some reason the Kconfig
> dictates that VAS depends on a 64k page size, which is problematic as I
> run Void Linux, which uses a 4k-page kernel.
> 
> Some early poking by others indicated there wasn't an obvious page size
> dependency in the code, and suggested I try modifying the config to switch
> it on. I did so, but was stopped by a minor complaint of an "unexpected DT
> configuration" by the VAS code. I wasn't equipped to figure out exactly what
> this meant, even after finding the offending condition, so after writing a
> very drawn-out forum post asking for help, I dropped the subject.
> 
> Fast forward to today, when I was reminded of the whole thing again, and
> decided to debug a bit further. Apparently the VAS platform device
> (derived from the DT node) has 5 resources on my 4k kernel, instead of 4
> (which evidently works for others who have had success on 64k kernels). I
> have no idea what this means in practice (I don't know how to introspect
> it), but after making a tiny patch[1], everything came up smoothly and I
> was doing blazing-fast gzip (de)compression in no time.
> 
> Everything seems to work fine on 4k pages. So, what's up? Are there
> pitfalls lurking around that I've yet to stumble over? More reasonably,
> I'm curious as to why the feature supposedly depends on 64k pages, or if
> there's anything else I should be concerned about.
> 

Maybe ask Sukadev who did the implementation and is maintaining it ?

> I do have to say I'm quite satisfied with the results of the NX
> accelerator, though. Being able to shuffle data to a RaptorCS box over gigE
> and get compressed data back faster than most software gzip could ever
> hope to achieve is no small feat, let alone the instantaneous results locally.
> :)
> 
> Cheers,
> Will Springer [she/her]
> 
> [1]: https://github.com/Skirmisher/void-packages/blob/vas-4k-pages/srcpkgs/linux5.9/patches/ppc-vas-on-4k.patch
> 


Christophe

Re: CONFIG_PPC_VAS depends on 64k pages...?

[Sukadev Bhattiprolu]

Christophe Leroy [christophe.leroy@csgroup.eu] wrote:
> Hi,
> 
> Le 19/11/2020 à 11:58, Will Springer a écrit :
> > I learned about the POWER9 gzip accelerator a few months ago when the
> > support hit upstream Linux 5.8. However, for some reason the Kconfig
> > dictates that VAS depends on a 64k page size, which is problematic as I
> > run Void Linux, which uses a 4k-page kernel.
> > 
> > Some early poking by others indicated there wasn't an obvious page size
> > dependency in the code, and suggested I try modifying the config to switch
> > it on. I did so, but was stopped by a minor complaint of an "unexpected DT
> > configuration" by the VAS code. I wasn't equipped to figure out exactly what
> > this meant, even after finding the offending condition, so after writing a
> > very drawn-out forum post asking for help, I dropped the subject.
> > 
> > Fast forward to today, when I was reminded of the whole thing again, and
> > decided to debug a bit further. Apparently the VAS platform device
> > (derived from the DT node) has 5 resources on my 4k kernel, instead of 4
> > (which evidently works for others who have had success on 64k kernels). I
> > have no idea what this means in practice (I don't know how to introspect
> > it), but after making a tiny patch[1], everything came up smoothly and I
> > was doing blazing-fast gzip (de)compression in no time.
> > 
> > Everything seems to work fine on 4k pages. So, what's up? Are there
> > pitfalls lurking around that I've yet to stumble over? More reasonably,
> > I'm curious as to why the feature supposedly depends on 64k pages, or if
> > there's anything else I should be concerned about.

Will,

The reason I put in that config check is because we were only able to
test 64K pages at that point.

It is interesting that it is working for you. Following code in skiboot
https://github.com/open-power/skiboot/blob/master/hw/vas.c should restrict
it to 64K pages. IIRC there is also a corresponding change in some NX 
registers that should also be configured to allow 4K pages.


	static int init_north_ctl(struct proc_chip *chip)
	{
		uint64_t val = 0ULL;

		val = SETFIELD(VAS_64K_MODE_MASK, val, true);
		val = SETFIELD(VAS_ACCEPT_PASTE_MASK, val, true);
		val = SETFIELD(VAS_ENABLE_WC_MMIO_BAR, val, true);
		val = SETFIELD(VAS_ENABLE_UWC_MMIO_BAR, val, true);
		val = SETFIELD(VAS_ENABLE_RMA_MMIO_BAR, val, true);

		return vas_scom_write(chip, VAS_MISC_N_CTL, val);
	}

I am copying Bulent Albali and Haren Myneni who have been working with
VAS/NX for their thoughts/experience.

> > 
> 
> Maybe ask Sukadev who did the implementation and is maintaining it ?
> 
> > I do have to say I'm quite satisfied with the results of the NX
> > accelerator, though. Being able to shuffle data to a RaptorCS box over gigE
> > and get compressed data back faster than most software gzip could ever
> > hope to achieve is no small feat, let alone the instantaneous results locally.
> > :)
> > 
> > Cheers,
> > Will Springer [she/her]
> > 
> > [1]: https://github.com/Skirmisher/void-packages/blob/vas-4k-pages/srcpkgs/linux5.9/patches/ppc-vas-on-4k.patch
> > 
> 
> 
> Christophe

Re: CONFIG_PPC_VAS depends on 64k pages...?

[Bulent Abali]

[-- Attachment #1: Type: text/plain, Size: 4652 bytes --]

I don't know anything about VAS page size requirements in the kernel.  I 
checked the user compression library and saw that we do a sysconf to get 
the page size; so the library should be immune to page size by design.
But it wouldn't surprise me if a 64KB constant is inadvertently hardcoded 
somewhere else in the library.  Giving heads up to Tulio and Raphael who 
are owners of the github repo.

https://github.com/libnxz/power-gzip/blob/master/lib/nx_zlib.c#L922

If we got this wrong in the library it might manifest itself as an error 
message of the sort "excessive page faults".  The library must touch pages 
ahead to make them present in the memory; occasional page faults is 
acceptable. It will retry.


Bulent




From:   "Sukadev Bhattiprolu" <sukadev@linux.ibm.com>
To:     "Christophe Leroy" <christophe.leroy@csgroup.eu>
Cc:     "Will Springer" <skirmisher@protonmail.com>, 
linuxppc-dev@lists.ozlabs.org, daniel@octaforge.org, Bulent 
Abali/Watson/IBM@IBM, haren@linux.ibm.com
Date:   12/01/2020 12:53 AM
Subject:        Re: CONFIG_PPC_VAS depends on 64k pages...?




Christophe Leroy [christophe.leroy@csgroup.eu] wrote:
> Hi,
> 
> Le 19/11/2020 à 11:58, Will Springer a écrit :
> > I learned about the POWER9 gzip accelerator a few months ago when the
> > support hit upstream Linux 5.8. However, for some reason the Kconfig
> > dictates that VAS depends on a 64k page size, which is problematic as 
I
> > run Void Linux, which uses a 4k-page kernel.
> > 
> > Some early poking by others indicated there wasn't an obvious page 
size
> > dependency in the code, and suggested I try modifying the config to 
switch
> > it on. I did so, but was stopped by a minor complaint of an 
"unexpected DT
> > configuration" by the VAS code. I wasn't equipped to figure out 
exactly what
> > this meant, even after finding the offending condition, so after 
writing a
> > very drawn-out forum post asking for help, I dropped the subject.
> > 
> > Fast forward to today, when I was reminded of the whole thing again, 
and
> > decided to debug a bit further. Apparently the VAS platform device
> > (derived from the DT node) has 5 resources on my 4k kernel, instead of 
4
> > (which evidently works for others who have had success on 64k 
kernels). I
> > have no idea what this means in practice (I don't know how to 
introspect
> > it), but after making a tiny patch[1], everything came up smoothly and 
I
> > was doing blazing-fast gzip (de)compression in no time.
> > 
> > Everything seems to work fine on 4k pages. So, what's up? Are there
> > pitfalls lurking around that I've yet to stumble over? More 
reasonably,
> > I'm curious as to why the feature supposedly depends on 64k pages, or 
if
> > there's anything else I should be concerned about.

Will,

The reason I put in that config check is because we were only able to
test 64K pages at that point.

It is interesting that it is working for you. Following code in skiboot
https://github.com/open-power/skiboot/blob/master/hw/vas.c should restrict
it to 64K pages. IIRC there is also a corresponding change in some NX 
registers that should also be configured to allow 4K pages.


                 static int init_north_ctl(struct proc_chip *chip)
                 {
                                 uint64_t val = 0ULL;

                                 val = SETFIELD(VAS_64K_MODE_MASK, val, 
true);
                                 val = SETFIELD(VAS_ACCEPT_PASTE_MASK, 
val, true);
                                 val = SETFIELD(VAS_ENABLE_WC_MMIO_BAR, 
val, true);
                                 val = SETFIELD(VAS_ENABLE_UWC_MMIO_BAR, 
val, true);
                                 val = SETFIELD(VAS_ENABLE_RMA_MMIO_BAR, 
val, true);

                                 return vas_scom_write(chip, 
VAS_MISC_N_CTL, val);
                 }

I am copying Bulent Albali and Haren Myneni who have been working with
VAS/NX for their thoughts/experience.

> > 
> 
> Maybe ask Sukadev who did the implementation and is maintaining it ?
> 
> > I do have to say I'm quite satisfied with the results of the NX
> > accelerator, though. Being able to shuffle data to a RaptorCS box over 
gigE
> > and get compressed data back faster than most software gzip could ever
> > hope to achieve is no small feat, let alone the instantaneous results 
locally.
> > :)
> > 
> > Cheers,
> > Will Springer [she/her]
> > 
> > [1]: 
https://github.com/Skirmisher/void-packages/blob/vas-4k-pages/srcpkgs/linux5.9/patches/ppc-vas-on-4k.patch

> > 
> 
> 
> Christophe





[-- Attachment #2: Type: text/html, Size: 7501 bytes --]

Re: CONFIG_PPC_VAS depends on 64k pages...?

[Will Springer]

On Tuesday, December 1, 2020 5:16:51 AM PST Bulent Abali wrote:
> I don't know anything about VAS page size requirements in the kernel.  I
> checked the user compression library and saw that we do a sysconf to
> get the page size; so the library should be immune to page size by
> design. But it wouldn't surprise me if a 64KB constant is inadvertently
> hardcoded somewhere else in the library.  Giving heads up to Tulio and
> Raphael who are owners of the github repo.
> 
> https://github.com/libnxz/power-gzip/blob/master/lib/nx_zlib.c#L922
> 
> If we got this wrong in the library it might manifest itself as an error
> message of the sort "excessive page faults".  The library must touch
> pages ahead to make them present in the memory; occasional page faults
> is acceptable. It will retry.

Hm, good to know. As I said I haven't noticed any problems so far, over a 
few different days of testing. My change is now in the Void Linux kernel 
package, and is working for others as well (including the Void maintainer 
Daniel/q66 who I CC'd initially).

> 
> Bulent
> 
> 
> 
> 
> From:        "Sukadev Bhattiprolu" <sukadev@linux.ibm.com>
> To:        "Christophe Leroy" <christophe.leroy@csgroup.eu>
> Cc:        "Will Springer" <skirmisher@protonmail.com>,
> linuxppc-dev@lists.ozlabs.org, daniel@octaforge.org, Bulent
> Abali/Watson/IBM@IBM, haren@linux.ibm.com Date:        12/01/2020 12:53
> AM
> Subject:        Re: CONFIG_PPC_VAS depends on 64k pages...?
> 
> Christophe Leroy [christophe.leroy@csgroup.eu] wrote:
> > Hi,
> > 
> > Le 19/11/2020 à 11:58, Will Springer a écrit :
> > > I learned about the POWER9 gzip accelerator a few months ago when
> > > the
> > > support hit upstream Linux 5.8. However, for some reason the Kconfig
> > > dictates that VAS depends on a 64k page size, which is problematic
> > > as I
> > > run Void Linux, which uses a 4k-page kernel.
> > > 
> > > Some early poking by others indicated there wasn't an obvious page
> > > size
> > > dependency in the code, and suggested I try modifying the config to
> > > switch it on. I did so, but was stopped by a minor complaint of an
> > > "unexpected DT configuration" by the VAS code. I wasn't equipped to
> > > figure out exactly what this meant, even after finding the
> > > offending condition, so after writing a very drawn-out forum post
> > > asking for help, I dropped the subject.
> > > 
> > > Fast forward to today, when I was reminded of the whole thing again,
> > > and decided to debug a bit further. Apparently the VAS platform
> > > device (derived from the DT node) has 5 resources on my 4k kernel,
> > > instead of 4 (which evidently works for others who have had success
> > > on 64k kernels). I have no idea what this means in practice (I
> > > don't know how to introspect it), but after making a tiny patch[1],
> > > everything came up smoothly and I was doing blazing-fast gzip
> > > (de)compression in no time.
> > > 
> > > Everything seems to work fine on 4k pages. So, what's up? Are there
> > > pitfalls lurking around that I've yet to stumble over? More
> > > reasonably,
> > > I'm curious as to why the feature supposedly depends on 64k pages,
> > > or if there's anything else I should be concerned about.
> 
> Will,
> 
> The reason I put in that config check is because we were only able to
> test 64K pages at that point.
> 
> It is interesting that it is working for you. Following code in skiboot
> https://github.com/open-power/skiboot/blob/master/hw/vas.cshould
> restrict it to 64K pages. IIRC there is also a corresponding change in
> some NX registers that should also be configured to allow 4K pages. 

Huh, that is interesting indeed. As far as the kernel code, the only thing 
specific to 64k pages I could find was in [1], where 
VAS_XLATE_LPCR_PAGE_SIZE is set. There is also NX_PAGE_SIZE in drivers/
crypto/nx/nx.h, which is set to 4096, but I don't know if that's related to 
kernel page size at all. Without a better idea of the code base, I didn't
examine more thoroughly.

[1]: https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/tree/arch/powerpc/platforms/powernv/vas-window.c#n293

>                 static int init_north_ctl(struct proc_chip *chip)
>                 {
>                                  uint64_t val = 0ULL;
> 
>                                  val = SETFIELD(VAS_64K_MODE_MASK, val,
> true); val = SETFIELD(VAS_ACCEPT_PASTE_MASK, val, true); val =
> SETFIELD(VAS_ENABLE_WC_MMIO_BAR, val, true); val =
> SETFIELD(VAS_ENABLE_UWC_MMIO_BAR, val, true); val =
> SETFIELD(VAS_ENABLE_RMA_MMIO_BAR, val, true);
> 
>                                  return vas_scom_write(chip,
> VAS_MISC_N_CTL, val); }
> 
> I am copying Bulent Albali and Haren Myneni who have been working with
> VAS/NX for their thoughts/experience.

Thanks for this and for your input, by the way.

> 
> > Maybe ask Sukadev who did the implementation and is maintaining it ?
> > 
> > > I do have to say I'm quite satisfied with the results of the NX
> > > accelerator, though. Being able to shuffle data to a RaptorCS box
> > > over gigE and get compressed data back faster than most software
> > > gzip could ever hope to achieve is no small feat, let alone the
> > > instantaneous results locally.> > 
> > > :)
> > > 
> > > Cheers,
> > > Will Springer [she/her]
> > > 
> > > [1]:
> > > https://github.com/Skirmisher/void-packages/blob/vas-4k-pages/srcpkgs/linux5.9/patches/ppc-vas-on-4k.patch
> > Christophe

Will [she/her]

Re: CONFIG_PPC_VAS depends on 64k pages...?

[Carlos Eduardo de Paula]

[-- Attachment #1: Type: text/plain, Size: 6228 bytes --]

Hi all, any news on this matter?

Can a patch be submitted for evaluation?

Thanks,
Carlos

On Wed, Dec 2, 2020 at 4:19 AM Will Springer <skirmisher@protonmail.com>
wrote:

> On Tuesday, December 1, 2020 5:16:51 AM PST Bulent Abali wrote:
> > I don't know anything about VAS page size requirements in the kernel.  I
> > checked the user compression library and saw that we do a sysconf to
> > get the page size; so the library should be immune to page size by
> > design. But it wouldn't surprise me if a 64KB constant is inadvertently
> > hardcoded somewhere else in the library.  Giving heads up to Tulio and
> > Raphael who are owners of the github repo.
> >
> > https://github.com/libnxz/power-gzip/blob/master/lib/nx_zlib.c#L922
> >
> > If we got this wrong in the library it might manifest itself as an error
> > message of the sort "excessive page faults".  The library must touch
> > pages ahead to make them present in the memory; occasional page faults
> > is acceptable. It will retry.
>
> Hm, good to know. As I said I haven't noticed any problems so far, over a
> few different days of testing. My change is now in the Void Linux kernel
> package, and is working for others as well (including the Void maintainer
> Daniel/q66 who I CC'd initially).
>
> >
> > Bulent
> >
> >
> >
> >
> > From:        "Sukadev Bhattiprolu" <sukadev@linux.ibm.com>
> > To:        "Christophe Leroy" <christophe.leroy@csgroup.eu>
> > Cc:        "Will Springer" <skirmisher@protonmail.com>,
> > linuxppc-dev@lists.ozlabs.org, daniel@octaforge.org, Bulent
> > Abali/Watson/IBM@IBM, haren@linux.ibm.com Date:        12/01/2020 12:53
> > AM
> > Subject:        Re: CONFIG_PPC_VAS depends on 64k pages...?
> >
> > Christophe Leroy [christophe.leroy@csgroup.eu] wrote:
> > > Hi,
> > >
> > > Le 19/11/2020 à 11:58, Will Springer a écrit :
> > > > I learned about the POWER9 gzip accelerator a few months ago when
> > > > the
> > > > support hit upstream Linux 5.8. However, for some reason the Kconfig
> > > > dictates that VAS depends on a 64k page size, which is problematic
> > > > as I
> > > > run Void Linux, which uses a 4k-page kernel.
> > > >
> > > > Some early poking by others indicated there wasn't an obvious page
> > > > size
> > > > dependency in the code, and suggested I try modifying the config to
> > > > switch it on. I did so, but was stopped by a minor complaint of an
> > > > "unexpected DT configuration" by the VAS code. I wasn't equipped to
> > > > figure out exactly what this meant, even after finding the
> > > > offending condition, so after writing a very drawn-out forum post
> > > > asking for help, I dropped the subject.
> > > >
> > > > Fast forward to today, when I was reminded of the whole thing again,
> > > > and decided to debug a bit further. Apparently the VAS platform
> > > > device (derived from the DT node) has 5 resources on my 4k kernel,
> > > > instead of 4 (which evidently works for others who have had success
> > > > on 64k kernels). I have no idea what this means in practice (I
> > > > don't know how to introspect it), but after making a tiny patch[1],
> > > > everything came up smoothly and I was doing blazing-fast gzip
> > > > (de)compression in no time.
> > > >
> > > > Everything seems to work fine on 4k pages. So, what's up? Are there
> > > > pitfalls lurking around that I've yet to stumble over? More
> > > > reasonably,
> > > > I'm curious as to why the feature supposedly depends on 64k pages,
> > > > or if there's anything else I should be concerned about.
> >
> > Will,
> >
> > The reason I put in that config check is because we were only able to
> > test 64K pages at that point.
> >
> > It is interesting that it is working for you. Following code in skiboot
> > https://github.com/open-power/skiboot/blob/master/hw/vas.cshould
> > restrict it to 64K pages. IIRC there is also a corresponding change in
> > some NX registers that should also be configured to allow 4K pages.
>
> Huh, that is interesting indeed. As far as the kernel code, the only thing
> specific to 64k pages I could find was in [1], where
> VAS_XLATE_LPCR_PAGE_SIZE is set. There is also NX_PAGE_SIZE in drivers/
> crypto/nx/nx.h, which is set to 4096, but I don't know if that's related
> to
> kernel page size at all. Without a better idea of the code base, I didn't
> examine more thoroughly.
>
> [1]:
> https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/tree/arch/powerpc/platforms/powernv/vas-window.c#n293
>
> >                 static int init_north_ctl(struct proc_chip *chip)
> >                 {
> >                                  uint64_t val = 0ULL;
> >
> >                                  val = SETFIELD(VAS_64K_MODE_MASK, val,
> > true); val = SETFIELD(VAS_ACCEPT_PASTE_MASK, val, true); val =
> > SETFIELD(VAS_ENABLE_WC_MMIO_BAR, val, true); val =
> > SETFIELD(VAS_ENABLE_UWC_MMIO_BAR, val, true); val =
> > SETFIELD(VAS_ENABLE_RMA_MMIO_BAR, val, true);
> >
> >                                  return vas_scom_write(chip,
> > VAS_MISC_N_CTL, val); }
> >
> > I am copying Bulent Albali and Haren Myneni who have been working with
> > VAS/NX for their thoughts/experience.
>
> Thanks for this and for your input, by the way.
>
> >
> > > Maybe ask Sukadev who did the implementation and is maintaining it ?
> > >
> > > > I do have to say I'm quite satisfied with the results of the NX
> > > > accelerator, though. Being able to shuffle data to a RaptorCS box
> > > > over gigE and get compressed data back faster than most software
> > > > gzip could ever hope to achieve is no small feat, let alone the
> > > > instantaneous results locally.> >
> > > > :)
> > > >
> > > > Cheers,
> > > > Will Springer [she/her]
> > > >
> > > > [1]:
> > > >
> https://github.com/Skirmisher/void-packages/blob/vas-4k-pages/srcpkgs/linux5.9/patches/ppc-vas-on-4k.patch
> > > Christophe
>
> Will [she/her]
>
>
>
>
>

-- 
________________________________________
Carlos Eduardo de Paula
me@carlosedp.com
http://carlosedp.com
https://twitter.com/carlosedp
https://www.linkedin.com/in/carlosedp/
________________________________________

[-- Attachment #2: Type: text/html, Size: 9450 bytes --]

Re: CONFIG_PPC_VAS depends on 64k pages...?

[Carlos Eduardo de Paula]

[-- Attachment #1: Type: text/plain, Size: 4119 bytes --]

On Tue, Dec 1, 2020 at 2:54 AM Sukadev Bhattiprolu <sukadev@linux.ibm.com>
wrote:

>
> Christophe Leroy [christophe.leroy@csgroup.eu] wrote:
> > Hi,
> >
> > Le 19/11/2020 à 11:58, Will Springer a écrit :
> > > I learned about the POWER9 gzip accelerator a few months ago when the
> > > support hit upstream Linux 5.8. However, for some reason the Kconfig
> > > dictates that VAS depends on a 64k page size, which is problematic as I
> > > run Void Linux, which uses a 4k-page kernel.
> > >
> > > Some early poking by others indicated there wasn't an obvious page size
> > > dependency in the code, and suggested I try modifying the config to
> switch
> > > it on. I did so, but was stopped by a minor complaint of an
> "unexpected DT
> > > configuration" by the VAS code. I wasn't equipped to figure out
> exactly what
> > > this meant, even after finding the offending condition, so after
> writing a
> > > very drawn-out forum post asking for help, I dropped the subject.
> > >
> > > Fast forward to today, when I was reminded of the whole thing again,
> and
> > > decided to debug a bit further. Apparently the VAS platform device
> > > (derived from the DT node) has 5 resources on my 4k kernel, instead of
> 4
> > > (which evidently works for others who have had success on 64k
> kernels). I
> > > have no idea what this means in practice (I don't know how to
> introspect
> > > it), but after making a tiny patch[1], everything came up smoothly and
> I
> > > was doing blazing-fast gzip (de)compression in no time.
> > >
> > > Everything seems to work fine on 4k pages. So, what's up? Are there
> > > pitfalls lurking around that I've yet to stumble over? More reasonably,
> > > I'm curious as to why the feature supposedly depends on 64k pages, or
> if
> > > there's anything else I should be concerned about.
>
> Will,
>
> The reason I put in that config check is because we were only able to
> test 64K pages at that point.
>
> It is interesting that it is working for you. Following code in skiboot
> https://github.com/open-power/skiboot/blob/master/hw/vas.c should restrict
> it to 64K pages. IIRC there is also a corresponding change in some NX
> registers that should also be configured to allow 4K pages.
>
>
>         static int init_north_ctl(struct proc_chip *chip)
>         {
>                 uint64_t val = 0ULL;
>
>                 val = SETFIELD(VAS_64K_MODE_MASK, val, true);
>                 val = SETFIELD(VAS_ACCEPT_PASTE_MASK, val, true);
>                 val = SETFIELD(VAS_ENABLE_WC_MMIO_BAR, val, true);
>                 val = SETFIELD(VAS_ENABLE_UWC_MMIO_BAR, val, true);
>                 val = SETFIELD(VAS_ENABLE_RMA_MMIO_BAR, val, true);
>
>                 return vas_scom_write(chip, VAS_MISC_N_CTL, val);
>         }
>
> I am copying Bulent Albali and Haren Myneni who have been working with
> VAS/NX for their thoughts/experience.
>
> > >
> >
> > Maybe ask Sukadev who did the implementation and is maintaining it ?
> >
> > > I do have to say I'm quite satisfied with the results of the NX
> > > accelerator, though. Being able to shuffle data to a RaptorCS box over
> gigE
> > > and get compressed data back faster than most software gzip could ever
> > > hope to achieve is no small feat, let alone the instantaneous results
> locally.
> > > :)
> > >
> > > Cheers,
> > > Will Springer [she/her]
> > >
> > > [1]:
> https://github.com/Skirmisher/void-packages/blob/vas-4k-pages/srcpkgs/linux5.9/patches/ppc-vas-on-4k.patch
> > >
> >
> >
> > Christophe
>

Hi all, I'd like to report that with Will's patch, I'm using NX-Gzip
perfectly on Linux 5.9.10 built with 4K pages and no changes on firmware in
a Raptor Computing Blackbird workstation.

I'm using Debian 10 distro.

Ref. https://twitter.com/carlosedp/status/1328424799216021511

Carlos


-- 
________________________________________
Carlos Eduardo de Paula
me@carlosedp.com
http://carlosedp.com
https://twitter.com/carlosedp
https://www.linkedin.com/in/carlosedp/
________________________________________

[-- Attachment #2: Type: text/html, Size: 6384 bytes --]