User question about memory scrubbing

User question about memory scrubbing

[Anders Andersson]

Hi! I realize that this is more of a developer-to-developer list, but
I'm a hobbyist who recently bought my first system with ECC RAM
(Opteron 6386 SE) and I can't get memory scrubbing to work. It's hard
to find people who know anything about it.

Preliminary research led me to the EDAC documentation on
https://www.kernel.org/doc/Documentation/ABI/testing/sysfs-devices-edac
and in particular the "sdram_scrub_rate" file, but had no luck
manipulating it.

Before I'm getting too lost: Is that the right way to configure it?

I have amd64_edac_mod and edac_mce_amd loaded. I briefly looked at
amd64_edac.c and it appears to have the necessary code and matches the
documentation from AMD so there's something I'm not doing right. I did
post a more elaborate question on
https://unix.stackexchange.com/questions/593060/how-do-i-enable-and-verify-ecc-ram-scrubbing-in-linux
but I'm afraid it's too technical for most users (too technical for me
too apparently!)

Hints and pointers welcome,
Anders

Re: User question about memory scrubbing

[Borislav Petkov]

On Thu, Jun 18, 2020 at 06:49:45PM +0200, Anders Andersson wrote:
> Hi! I realize that this is more of a developer-to-developer list, but
> I'm a hobbyist who recently bought my first system with ECC RAM
> (Opteron 6386 SE) and I can't get memory scrubbing to work. It's hard
> to find people who know anything about it.
> 
> Preliminary research led me to the EDAC documentation on
> https://www.kernel.org/doc/Documentation/ABI/testing/sysfs-devices-edac
> and in particular the "sdram_scrub_rate" file, but had no luck
> manipulating it.

Oh, you're manipulating it alright but there's a bug in reporting it.
Wanna test a patch?

-- 
Regards/Gruss,
    Boris.

https://people.kernel.org/tglx/notes-about-netiquette

[PATCH] EDAC/amd64: Read back the scrub rate PCI register on F15h

[Borislav Petkov]

On Thu, Jun 18, 2020 at 07:56:46PM +0200, Borislav Petkov wrote:
> Oh, you're manipulating it alright but there's a bug in reporting it.
> Wanna test a patch?

Here it is:

---
From: Borislav Petkov <bp@suse.de>

Commit:

  da92110dfdfa ("EDAC, amd64_edac: Extend scrub rate support to F15hM60h")

added support for F15h, model 0x60 CPUs but in doing so, missed to read
back SCRCTRL PCI config register on F15h CPUs which are *not* model
0x60. Add that read so that doing

  $ cat /sys/devices/system/edac/mc/mc0/sdram_scrub_rate

can show the previously set DRAM scrub rate.

Fixes: da92110dfdfa ("EDAC, amd64_edac: Extend scrub rate support to F15hM60h")
Reported-by: Anders Andersson <pipatron@gmail.com>
Signed-off-by: Borislav Petkov <bp@suse.de>
Cc: <stable@vger.kernel.org> #v4.4..
Link: https://lkml.kernel.org/r/CAKkunMbNWppx_i6xSdDHLseA2QQmGJqj_crY=NF-GZML5np4Vw@mail.gmail.com
---
 drivers/edac/amd64_edac.c | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/drivers/edac/amd64_edac.c b/drivers/edac/amd64_edac.c
index ef90070a9194..6262f6370c5d 100644
--- a/drivers/edac/amd64_edac.c
+++ b/drivers/edac/amd64_edac.c
@@ -269,6 +269,8 @@ static int get_scrub_rate(struct mem_ctl_info *mci)
 
 		if (pvt->model == 0x60)
 			amd64_read_pci_cfg(pvt->F2, F15H_M60H_SCRCTRL, &scrubval);
+		else
+			amd64_read_pci_cfg(pvt->F3, SCRCTRL, &scrubval);
 	} else {
 		amd64_read_pci_cfg(pvt->F3, SCRCTRL, &scrubval);
 	}
-- 
2.21.0

-- 
Regards/Gruss,
    Boris.

https://people.kernel.org/tglx/notes-about-netiquette

Re: [PATCH] EDAC/amd64: Read back the scrub rate PCI register on F15h

[Borislav Petkov]

On Thu, Jun 18, 2020 at 08:40:41PM +0200, Borislav Petkov wrote:
> On Thu, Jun 18, 2020 at 07:56:46PM +0200, Borislav Petkov wrote:
> > Oh, you're manipulating it alright but there's a bug in reporting it.
> > Wanna test a patch?
> 
> Here it is:
> 
> ---
> From: Borislav Petkov <bp@suse.de>
> 
> Commit:
> 
>   da92110dfdfa ("EDAC, amd64_edac: Extend scrub rate support to F15hM60h")
> 
> added support for F15h, model 0x60 CPUs but in doing so, missed to read
> back SCRCTRL PCI config register on F15h CPUs which are *not* model
> 0x60. Add that read so that doing
> 
>   $ cat /sys/devices/system/edac/mc/mc0/sdram_scrub_rate
> 
> can show the previously set DRAM scrub rate.
> 
> Fixes: da92110dfdfa ("EDAC, amd64_edac: Extend scrub rate support to F15hM60h")
> Reported-by: Anders Andersson <pipatron@gmail.com>
> Signed-off-by: Borislav Petkov <bp@suse.de>
> Cc: <stable@vger.kernel.org> #v4.4..
> Link: https://lkml.kernel.org/r/CAKkunMbNWppx_i6xSdDHLseA2QQmGJqj_crY=NF-GZML5np4Vw@mail.gmail.com
> ---
>  drivers/edac/amd64_edac.c | 2 ++
>  1 file changed, 2 insertions(+)
> 
> diff --git a/drivers/edac/amd64_edac.c b/drivers/edac/amd64_edac.c
> index ef90070a9194..6262f6370c5d 100644
> --- a/drivers/edac/amd64_edac.c
> +++ b/drivers/edac/amd64_edac.c
> @@ -269,6 +269,8 @@ static int get_scrub_rate(struct mem_ctl_info *mci)
>  
>  		if (pvt->model == 0x60)
>  			amd64_read_pci_cfg(pvt->F2, F15H_M60H_SCRCTRL, &scrubval);
> +		else
> +			amd64_read_pci_cfg(pvt->F3, SCRCTRL, &scrubval);
>  	} else {
>  		amd64_read_pci_cfg(pvt->F3, SCRCTRL, &scrubval);
>  	}
> -- 

Queued into edac-urgent.

Thx.

-- 
Regards/Gruss,
    Boris.

https://people.kernel.org/tglx/notes-about-netiquette

Re: [PATCH] EDAC/amd64: Read back the scrub rate PCI register on F15h

[Anders Andersson]

On Mon, Jun 22, 2020 at 5:13 PM Borislav Petkov <bp@alien8.de> wrote:
>
> On Thu, Jun 18, 2020 at 08:40:41PM +0200, Borislav Petkov wrote:
> > On Thu, Jun 18, 2020 at 07:56:46PM +0200, Borislav Petkov wrote:
> > > Oh, you're manipulating it alright but there's a bug in reporting it.
> > > Wanna test a patch?
> >
> > Here it is:
> >
> > ---
> > From: Borislav Petkov <bp@suse.de>
> >
> > Commit:
> >
> >   da92110dfdfa ("EDAC, amd64_edac: Extend scrub rate support to F15hM60h")
> >
> > added support for F15h, model 0x60 CPUs but in doing so, missed to read
> > back SCRCTRL PCI config register on F15h CPUs which are *not* model
> > 0x60. Add that read so that doing
> >
> >   $ cat /sys/devices/system/edac/mc/mc0/sdram_scrub_rate
> >
> > can show the previously set DRAM scrub rate.
> >
> > Fixes: da92110dfdfa ("EDAC, amd64_edac: Extend scrub rate support to F15hM60h")
> > Reported-by: Anders Andersson <pipatron@gmail.com>
> > Signed-off-by: Borislav Petkov <bp@suse.de>
> > Cc: <stable@vger.kernel.org> #v4.4..
> > Link: https://lkml.kernel.org/r/CAKkunMbNWppx_i6xSdDHLseA2QQmGJqj_crY=NF-GZML5np4Vw@mail.gmail.com
> > ---
> >  drivers/edac/amd64_edac.c | 2 ++
> >  1 file changed, 2 insertions(+)
> >
> > diff --git a/drivers/edac/amd64_edac.c b/drivers/edac/amd64_edac.c
> > index ef90070a9194..6262f6370c5d 100644
> > --- a/drivers/edac/amd64_edac.c
> > +++ b/drivers/edac/amd64_edac.c
> > @@ -269,6 +269,8 @@ static int get_scrub_rate(struct mem_ctl_info *mci)
> >
> >               if (pvt->model == 0x60)
> >                       amd64_read_pci_cfg(pvt->F2, F15H_M60H_SCRCTRL, &scrubval);
> > +             else
> > +                     amd64_read_pci_cfg(pvt->F3, SCRCTRL, &scrubval);
> >       } else {
> >               amd64_read_pci_cfg(pvt->F3, SCRCTRL, &scrubval);
> >       }
> > --
>
> Queued into edac-urgent.
>
> Thx.
>
> --
> Regards/Gruss,
>     Boris.
>
> https://people.kernel.org/tglx/notes-about-netiquette


Ok, finally tested the patch on my machine, and (no surprise)
everything now works as expected, thanks!

// Anders

Re: [PATCH] EDAC/amd64: Read back the scrub rate PCI register on F15h

[Borislav Petkov]

On Tue, Jun 23, 2020 at 03:41:35AM +0200, Anders Andersson wrote:
> Ok, finally tested the patch on my machine, and (no surprise)
> everything now works as expected, thanks!

Thanks for testing, patch will appear upstream and in stable soon.

-- 
Regards/Gruss,
    Boris.

https://people.kernel.org/tglx/notes-about-netiquette

Re: User question about memory scrubbing

[Anders Andersson]

On Thu, Jun 18, 2020 at 7:56 PM Borislav Petkov <bp@alien8.de> wrote:
>
> On Thu, Jun 18, 2020 at 06:49:45PM +0200, Anders Andersson wrote:
> > Hi! I realize that this is more of a developer-to-developer list, but
> > I'm a hobbyist who recently bought my first system with ECC RAM
> > (Opteron 6386 SE) and I can't get memory scrubbing to work. It's hard
> > to find people who know anything about it.
> >
> > Preliminary research led me to the EDAC documentation on
> > https://www.kernel.org/doc/Documentation/ABI/testing/sysfs-devices-edac
> > and in particular the "sdram_scrub_rate" file, but had no luck
> > manipulating it.
>
> Oh, you're manipulating it alright but there's a bug in reporting it.
> Wanna test a patch?

Wow, nice to see that I'm not crazy, and thanks for the quick fix! I
saw the patch and I will try it out after I've re-learnt how to build
a kernel again. It's been well over 10 years since I last had to...

// Anders