[PATCH] badram filtering

[PATCH] badram filtering

[Robert Millan]

[-- Attachment #1: Type: text/plain, Size: 772 bytes --]


Hi,

This patch implements badram filtering in GRUB.  It's the same idea as in
http://rick.vanrein.org/linux/badram/ but applied to GRUB.

The badram module sits between loaders (or other users like lsmmap) and
filters the mmap entries, but only when user previously set the "badram"
variable (whose syntax is the same as the one in badram patch for Linux).

NOTE this won't affect loadees that get the memory map from BIOS instead
of from GRUB.  This means it currently works for Multiboot and for the
new/experimental Linux loader.

-- 
Robert Millan

  The DRM opt-in fallacy: "Your data belongs to us. We will decide when (and
  how) you may access your data; but nobody's threatening your freedom: we
  still allow you to remove your data and not access it at all."

[-- Attachment #2: badram.diff --]
[-- Type: text/x-diff, Size: 10630 bytes --]

2009-02-27  Robert Millan  <rmh@aybabtu.com>

	* conf/i386-pc.rmk (pkglib_MODULES): Add `badram.mod'.
	(badram_mod_SOURCES, badram_mod_CFLAGS)
	(badram_mod_LDFLAGS): New variables.
	* conf/i386-coreboot.rmk: Likewise.
	* conf/powerpc-ieee1275.rmk: Likewise.
	* conf/i386-ieee1275.rmk: Likewise.

	* lib/badram.c: New file.
	* include/grub/badram.h: New file.

	* commands/lsmmap.c (grub_cmd_lsmmap): Replace
	grub_machine_mmap_iterate() with grub_badram_mmap_iterate ().
	* loader/i386/linux.c (find_mmap_size, allocate_pages)
	(grub_linux32_boot): Likewise.
	* loader/i386/pc/multiboot.c (grub_get_multiboot_mmap_len)
	(grub_fill_multiboot_mmap): Likewise.

Index: conf/i386-pc.rmk
===================================================================
--- conf/i386-pc.rmk	(revision 2001)
+++ conf/i386-pc.rmk	(working copy)
@@ -370,6 +370,11 @@ lsmmap_mod_SOURCES = commands/lsmmap.c
 lsmmap_mod_CFLAGS = $(COMMON_CFLAGS)
 lsmmap_mod_LDFLAGS = $(COMMON_LDFLAGS)
 
+pkglib_MODULES += badram.mod
+badram_mod_SOURCES = lib/badram.c
+badram_mod_CFLAGS = $(COMMON_CFLAGS)
+badram_mod_LDFLAGS = $(COMMON_LDFLAGS)
+
 # For ata_pthru.mod.
 ata_pthru_mod_SOURCES = disk/ata_pthru.c
 ata_pthru_mod_CFLAGS = $(COMMON_CFLAGS)
Index: conf/i386-coreboot.rmk
===================================================================
--- conf/i386-coreboot.rmk	(revision 2001)
+++ conf/i386-coreboot.rmk	(working copy)
@@ -211,5 +211,10 @@ lsmmap_mod_SOURCES = commands/lsmmap.c
 lsmmap_mod_CFLAGS = $(COMMON_CFLAGS)
 lsmmap_mod_LDFLAGS = $(COMMON_LDFLAGS)
 
+pkglib_MODULES += badram.mod
+badram_mod_SOURCES = lib/badram.c
+badram_mod_CFLAGS = $(COMMON_CFLAGS)
+badram_mod_LDFLAGS = $(COMMON_LDFLAGS)
+
 include $(srcdir)/conf/i386.mk
 include $(srcdir)/conf/common.mk
Index: conf/powerpc-ieee1275.rmk
===================================================================
--- conf/powerpc-ieee1275.rmk	(revision 2001)
+++ conf/powerpc-ieee1275.rmk	(working copy)
@@ -181,5 +181,10 @@ lsmmap_mod_SOURCES = commands/lsmmap.c
 lsmmap_mod_CFLAGS = $(COMMON_CFLAGS)
 lsmmap_mod_LDFLAGS = $(COMMON_LDFLAGS)
 
+pkglib_MODULES += badram.mod
+badram_mod_SOURCES = lib/badram.c
+badram_mod_CFLAGS = $(COMMON_CFLAGS)
+badram_mod_LDFLAGS = $(COMMON_LDFLAGS)
+
 include $(srcdir)/conf/common.mk
 
Index: conf/i386-ieee1275.rmk
===================================================================
--- conf/i386-ieee1275.rmk	(revision 2001)
+++ conf/i386-ieee1275.rmk	(working copy)
@@ -210,5 +210,10 @@ lsmmap_mod_SOURCES = commands/lsmmap.c
 lsmmap_mod_CFLAGS = $(COMMON_CFLAGS)
 lsmmap_mod_LDFLAGS = $(COMMON_LDFLAGS)
 
+pkglib_MODULES += badram.mod
+badram_mod_SOURCES = lib/badram.c
+badram_mod_CFLAGS = $(COMMON_CFLAGS)
+badram_mod_LDFLAGS = $(COMMON_LDFLAGS)
+
 include $(srcdir)/conf/i386.mk
 include $(srcdir)/conf/common.mk
Index: lib/badram.c
===================================================================
--- lib/badram.c	(revision 0)
+++ lib/badram.c	(revision 0)
@@ -0,0 +1,127 @@
+/*
+ *  GRUB  --  GRand Unified Bootloader
+ *  Copyright (C) 2009  Free Software Foundation, Inc.
+ *
+ *  GRUB is free software: you can redistribute it and/or modify
+ *  it under the terms of the GNU General Public License as published by
+ *  the Free Software Foundation, either version 3 of the License, or
+ *  (at your option) any later version.
+ *
+ *  GRUB is distributed in the hope that it will be useful,
+ *  but WITHOUT ANY WARRANTY; without even the implied warranty of
+ *  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ *  GNU General Public License for more details.
+ *
+ *  You should have received a copy of the GNU General Public License
+ *  along with GRUB.  If not, see <http://www.gnu.org/licenses/>.
+ */
+
+#include <grub/err.h>
+#include <grub/types.h>
+#include <grub/badram.h>
+#include <grub/mm.h>
+#include <grub/env.h>
+#include <grub/misc.h>
+#include <grub/machine/memory.h>
+
+#define CHUNK_SIZE	0x400
+
+struct badness
+{
+  grub_uint64_t addr;
+  grub_uint64_t mask;
+  struct badness *next;
+};
+
+static struct badness *badness;
+
+#define xor(a,b)	(((a) & ~(b)) | (~(a) & (b)))
+
+static int
+is_bad (grub_uint64_t page)
+{
+  struct badness *pat;
+  for (pat = badness; pat != NULL; pat = pat->next)
+    if ((xor (page, pat->addr) & pat->mask) == 0)
+      return 1;
+  return 0;
+}
+
+grub_err_t
+grub_badram_mmap_iterate (int NESTED_FUNC_ATTR (*hook) (grub_uint64_t, grub_uint64_t, grub_uint32_t))
+{
+  char *str;
+  grub_err_t ret;
+
+  str = grub_env_get ("badram");
+  if (! str)
+    return grub_machine_mmap_iterate (hook);
+
+  while (1)
+    {
+      struct badness *new;
+      grub_uint64_t addr, mask;
+      
+      /* Parse address and mask.  */
+      addr = grub_strtoull (str, &str, 16);
+      if (*str == ',')
+	str++;
+      mask = grub_strtoull (str, &str, 16);
+      if (*str == ',')
+	str++;
+
+      if (grub_errno == GRUB_ERR_BAD_NUMBER)
+	{
+	  grub_errno = 0;
+	  break;
+	}
+
+      /* When part of a page is tainted, we discard the whole of it.  There's
+	 no point in providing sub-page chunks.  */
+      mask &= ~(CHUNK_SIZE - 1);
+
+      new = grub_malloc (sizeof (struct badness));
+      new->addr = addr;
+      new->mask = mask;
+
+      /* Insert.  */
+      new->next = badness;
+      badness = new;
+    }
+
+  auto int NESTED_FUNC_ATTR badram_hook (grub_uint64_t, grub_uint64_t, grub_uint32_t);
+  int NESTED_FUNC_ATTR badram_hook (grub_uint64_t addr, grub_uint64_t size, grub_uint32_t type)
+  {
+    /* These define the start and end of each chunk of good pages.  */
+    grub_uint64_t start, end;
+
+    for (start = addr; 1; start = end)
+      {
+	/* Find a good page.  */
+	for (; is_bad (start); start += CHUNK_SIZE);
+	
+	/* Nothing left to use.  */
+	if (start >= addr + size)
+	  return 0;
+	
+	/* Find the last good page in this chunk.  */
+	for (end = start + CHUNK_SIZE; end < addr + size && ! is_bad (end); end += CHUNK_SIZE);
+	
+	/* Process what we found.  */
+	if (hook (start, end - start, type))
+	  return 1;
+      }
+  }
+
+  ret = grub_machine_mmap_iterate (badram_hook);
+
+  while (badness)
+    {
+      struct badness *p;
+      p = badness->next;
+      grub_free (badness);
+      badness = p;
+    }
+
+  return ret;
+}
Index: include/grub/badram.h
===================================================================
--- include/grub/badram.h	(revision 0)
+++ include/grub/badram.h	(revision 0)
@@ -0,0 +1,29 @@
+/*
+ *  GRUB  --  GRand Unified Bootloader
+ *  Copyright (C) 2009  Free Software Foundation, Inc.
+ *
+ *  GRUB is free software: you can redistribute it and/or modify
+ *  it under the terms of the GNU General Public License as published by
+ *  the Free Software Foundation, either version 3 of the License, or
+ *  (at your option) any later version.
+ *
+ *  GRUB is distributed in the hope that it will be useful,
+ *  but WITHOUT ANY WARRANTY; without even the implied warranty of
+ *  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ *  GNU General Public License for more details.
+ *
+ *  You should have received a copy of the GNU General Public License
+ *  along with GRUB.  If not, see <http://www.gnu.org/licenses/>.
+ */
+
+#ifndef GRUB_BADRAM_HEADER
+#define GRUB_BADRAM_HEADER	1
+
+#include <grub/symbol.h>
+#include <grub/types.h>
+#include <grub/err.h>
+
+grub_err_t EXPORT_FUNC(grub_badram_mmap_iterate)
+     (int NESTED_FUNC_ATTR (*hook) (grub_uint64_t, grub_uint64_t, grub_uint32_t));
+
+#endif
Index: commands/lsmmap.c
===================================================================
--- commands/lsmmap.c	(revision 2001)
+++ commands/lsmmap.c	(working copy)
@@ -1,6 +1,6 @@
 /*
  *  GRUB  --  GRand Unified Bootloader
- *  Copyright (C) 2008  Free Software Foundation, Inc.
+ *  Copyright (C) 2008,2009  Free Software Foundation, Inc.
  *
  *  GRUB is free software: you can redistribute it and/or modify
  *  it under the terms of the GNU General Public License as published by
@@ -16,7 +16,7 @@
  *  along with GRUB.  If not, see <http://www.gnu.org/licenses/>.
  */
 
-#include <grub/machine/memory.h>
+#include <grub/badram.h>
 #include <grub/normal.h>
 #include <grub/dl.h>
 #include <grub/arg.h>
@@ -34,7 +34,7 @@ grub_cmd_lsmmap (struct grub_arg_list *s
 		   addr, size, type);
       return 0;
     }
-  grub_machine_mmap_iterate (hook);
+  grub_badram_mmap_iterate (hook);
 
   return 0;
 }
Index: loader/i386/linux.c
===================================================================
--- loader/i386/linux.c	(revision 2001)
+++ loader/i386/linux.c	(working copy)
@@ -1,6 +1,6 @@
 /*
  *  GRUB  --  GRand Unified Bootloader
- *  Copyright (C) 2006,2007,2008  Free Software Foundation, Inc.
+ *  Copyright (C) 2006,2007,2008,2009  Free Software Foundation, Inc.
  *
  *  GRUB is free software: you can redistribute it and/or modify
  *  it under the terms of the GNU General Public License as published by
@@ -20,6 +20,7 @@
 #include <grub/machine/machine.h>
 #include <grub/machine/memory.h>
 #include <grub/machine/loader.h>
+#include <grub/badram.h>
 #include <grub/file.h>
 #include <grub/disk.h>
 #include <grub/err.h>
@@ -102,7 +103,7 @@ find_mmap_size (void)
       return 0;
     }
   
-  grub_machine_mmap_iterate (hook);
+  grub_badram_mmap_iterate (hook);
   
   mmap_size = count * sizeof (struct grub_e820_mmap);
 
@@ -173,7 +174,7 @@ allocate_pages (grub_size_t prot_size)
 
       return 0;
     }
-  grub_machine_mmap_iterate (hook);
+  grub_badram_mmap_iterate (hook);
   if (! real_mode_mem)
     {
       grub_error (GRUB_ERR_OUT_OF_MEMORY, "cannot allocate real mode pages");
@@ -258,7 +259,7 @@ grub_linux32_boot (void)
     }
 
   e820_num = 0;
-  grub_machine_mmap_iterate (hook);
+  grub_badram_mmap_iterate (hook);
   params->mmap_size = e820_num;
 
   /* Hardware interrupts are not safe any longer.  */
Index: loader/i386/pc/multiboot.c
===================================================================
--- loader/i386/pc/multiboot.c	(revision 2001)
+++ loader/i386/pc/multiboot.c	(working copy)
@@ -32,6 +32,7 @@
 #include <grub/multiboot.h>
 #include <grub/machine/init.h>
 #include <grub/machine/memory.h>
+#include <grub/badram.h>
 #include <grub/elf.h>
 #include <grub/aout.h>
 #include <grub/file.h>
@@ -98,7 +99,7 @@ grub_get_multiboot_mmap_len (void)
       return 0;
     }
   
-  grub_machine_mmap_iterate (hook);
+  grub_badram_mmap_iterate (hook);
   
   return count * sizeof (struct grub_multiboot_mmap_entry);
 }
@@ -121,7 +122,7 @@ grub_fill_multiboot_mmap (struct grub_mu
       return 0;
     }
 
-  grub_machine_mmap_iterate (hook);
+  grub_badram_mmap_iterate (hook);
 }
 
 /* Check if BUFFER contains ELF32.  */

Re: [PATCH] badram filtering

[phcoder]

Very interesting. I have following suggestions:
1) define platform independent way to enumerate memory map. This 
function wouldn't mark types with GRUB_MACHINE_* but with one of the 
following types:
GRUB_MMAP_HOLE - means that at this position there is no RAM at all. 
Should this be iterated at all?
GRUB_MMAP_IO - usable only by I/O drivers
GRUB_MMAP_FREE - usable by ACPI and firmware-unaware OS. But may be 
unusable by grub itself. memory allocator should be used for requesting 
memory in grub
GRUB_MMAP_ACPI - usable by ACPI-aware OS after loading tables
GRUB_MMAP_PLATFORM_FIRST...GRUB_MMAP_PLATFORM_LAST - platform specific 
types which can't be used except by firmware-aware code and OS
2) when badram variable is set corresponding memory chunks should be 
removed from memory manager
3) add a method to specify badram in init.c so that grub2 itself has no 
risk of using any of this memory. I propose to embed initial environment 
in core.img. Then prefix and root can be changed to use this mechanism. 
It would also be able to override this initial environment through 
multiboot command line when grub2 is loaded by multiboot loader


Robert Millan wrote:
> Hi,
> 
> This patch implements badram filtering in GRUB.  It's the same idea as in
> http://rick.vanrein.org/linux/badram/ but applied to GRUB.
> 
> The badram module sits between loaders (or other users like lsmmap) and
> filters the mmap entries, but only when user previously set the "badram"
> variable (whose syntax is the same as the one in badram patch for Linux).
> 
> NOTE this won't affect loadees that get the memory map from BIOS instead
> of from GRUB.  This means it currently works for Multiboot and for the
> new/experimental Linux loader.
> 
> 
> 
> ------------------------------------------------------------------------
> 
> _______________________________________________
> Grub-devel mailing list
> Grub-devel@gnu.org
> http://lists.gnu.org/mailman/listinfo/grub-devel


-- 

Regards
Vladimir 'phcoder' Serbinenko

Re: [PATCH] badram filtering

[phcoder]

I've looked in depth in memory map and have seen that (1) is already 
done so except for EFI systems. And except that memory types are defined 
on per-machine basis. IMO generic memory types should be shared at the 
output of grub_machine_memory_iterate but it's not critical. I will code 
a similar function for EFI and adapt multiboot loader for EFI systems
phcoder wrote:
> Very interesting. I have following suggestions:
> 1) define platform independent way to enumerate memory map. This 
> function wouldn't mark types with GRUB_MACHINE_* but with one of the 
> following types:
> GRUB_MMAP_HOLE - means that at this position there is no RAM at all. 
> Should this be iterated at all?
> GRUB_MMAP_IO - usable only by I/O drivers
> GRUB_MMAP_FREE - usable by ACPI and firmware-unaware OS. But may be 
> unusable by grub itself. memory allocator should be used for requesting 
> memory in grub
> GRUB_MMAP_ACPI - usable by ACPI-aware OS after loading tables
> GRUB_MMAP_PLATFORM_FIRST...GRUB_MMAP_PLATFORM_LAST - platform specific 
> types which can't be used except by firmware-aware code and OS
> 2) when badram variable is set corresponding memory chunks should be 
> removed from memory manager
> 3) add a method to specify badram in init.c so that grub2 itself has no 
> risk of using any of this memory. I propose to embed initial environment 
> in core.img. Then prefix and root can be changed to use this mechanism. 
> It would also be able to override this initial environment through 
> multiboot command line when grub2 is loaded by multiboot loader
> 
> 
> Robert Millan wrote:
>> Hi,
>>
>> This patch implements badram filtering in GRUB.  It's the same idea as in
>> http://rick.vanrein.org/linux/badram/ but applied to GRUB.
>>
>> The badram module sits between loaders (or other users like lsmmap) and
>> filters the mmap entries, but only when user previously set the "badram"
>> variable (whose syntax is the same as the one in badram patch for Linux).
>>
>> NOTE this won't affect loadees that get the memory map from BIOS instead
>> of from GRUB.  This means it currently works for Multiboot and for the
>> new/experimental Linux loader.
>>
>>
>>
>> ------------------------------------------------------------------------
>>
>> _______________________________________________
>> Grub-devel mailing list
>> Grub-devel@gnu.org
>> http://lists.gnu.org/mailman/listinfo/grub-devel
> 
> 


-- 

Regards
Vladimir 'phcoder' Serbinenko

Re: [PATCH] badram filtering

[phcoder]

Hello. I'm now writing the code to make multiboot and bsd work on EFI 
based on my grub_machine_memory_iterate for EFI. MY code will be the 
same for all i386 branches. It's quite complete except that I had no 
time yet to test it. I'l submit once it works
I was thinking and found another utility for badram: BIOS hooks. They 
are actually allocating take some memory and then removing it from mmap. 
If bios mmap hook and kernels get their mmap from the same source we can 
ensure that bios hooks won't be overwritten by the kernel independently 
how it acquired memory map. So I think badram patch should be extended 
to allow arbitrary additions to mmap.
P.S. sorry if this e-mail isn't clear enough I've just had a long day of 
studies
phcoder wrote:
>> 2) when badram variable is set corresponding memory chunks should be 
>> removed from memory manager
>> 3) add a method to specify badram in init.c so that grub2 itself has 
>> no risk of using any of this memory. I propose to embed initial 
>> environment in core.img. Then prefix and root can be changed to use 
>> this mechanism. It would also be able to override this initial 
>> environment through multiboot command line when grub2 is loaded by 
>> multiboot loader
>>
>>
>> Robert Millan wrote:
>>> Hi,
>>>
>>> This patch implements badram filtering in GRUB.  It's the same idea 
>>> as in
>>> http://rick.vanrein.org/linux/badram/ but applied to GRUB.
>>>
>>> The badram module sits between loaders (or other users like lsmmap) and
>>> filters the mmap entries, but only when user previously set the "badram"
>>> variable (whose syntax is the same as the one in badram patch for 
>>> Linux).
>>>
>>> NOTE this won't affect loadees that get the memory map from BIOS instead
>>> of from GRUB.  This means it currently works for Multiboot and for the
>>> new/experimental Linux loader.
>>>
>>>
>>>
>>> ------------------------------------------------------------------------
>>>
>>> _______________________________________________
>>> Grub-devel mailing list
>>> Grub-devel@gnu.org
>>> http://lists.gnu.org/mailman/listinfo/grub-devel
>>
>>
> 
> 


-- 

Regards
Vladimir 'phcoder' Serbinenko