RFC: enhanced memory protection support

RFC: enhanced memory protection support

[Bean]

Hi,

While testing network function in efi mode, I've found several memory
leak related to fragmentation, but there is still some memory problem
that's very tricky to locate. For example, you can run testspeed on a
large file several times in a row and it could show the memory error.
Since network condition are very difficult to reproduce, I have to
look at the source of this issue, memory allocation. Currently it has
mm_debug option that could print out file and line number of each
memory call, but it's quite useless since we can't find the relevant
informaton with full screen of prints.

Here are some of my ideas for enhanced memory protection support:

1, when we allocate memory, we append some information at the end of
the buffer, which include filename, lineno of caller, and tag to
indicate what is used for and some padding to detect memory overwrite
problem.

2. add a command to print the current memory list with extended
information, this can be used to find memory leaks.

3. it's also a good idea to run the memory check in automated test to
locate potential issue.

-- 
Best wishes
Bean

Re: RFC: enhanced memory protection support

[Vladimir 'φ-coder/phcoder' Serbinenko]

[-- Attachment #1: Type: text/plain, Size: 1776 bytes --]

On 30.04.2012 17:26, Bean wrote:
> Hi,
>
> While testing network function in efi mode, I've found several memory
> leak related to fragmentation, but there is still some memory problem
> that's very tricky to locate. For example, you can run testspeed on a
> large file several times in a row and it could show the memory error.
> Since network condition are very difficult to reproduce, I have to
> look at the source of this issue, memory allocation. Currently it has
> mm_debug option that could print out file and line number of each
> memory call, but it's quite useless since we can't find the relevant
> informaton with full screen of prints.
> Here are some of my ideas for enhanced memory protection support:
>
> 1, when we allocate memory, we append some information at the end of
> the buffer, which include filename, lineno of caller, and tag to
> indicate what is used for and some padding to detect memory overwrite
> problem.
>
> 2. add a command to print the current memory list with extended
> information, this can be used to find memory leaks.
>
> 3. it's also a good idea to run the memory check in automated test to
> locate potential issue.
This is pretty easy to do leveraging some of the code I did for POSIX
support. But:
- Due to additional time and space required it should be done only when
mm-debug is enabled.
- The additional data has to be stored before rather than after the
range since we store all the info before.
- Integrating with automated tests isn't that easy since some memory is
intentionally never freed i.a. disk cache or otherwise in use (i.a.
terminal). We need exceptions for those.
I have half-working patch, just needs few fixes.


-- 
Regards
Vladimir 'φ-coder/phcoder' Serbinenko



[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 294 bytes --]

Re: RFC: enhanced memory protection support

[Bean]

On Tue, May 1, 2012 at 1:24 AM, Vladimir 'φ-coder/phcoder' Serbinenko
<phcoder@gmail.com> wrote:
> On 30.04.2012 17:26, Bean wrote:
>> Hi,
>>
>> While testing network function in efi mode, I've found several memory
>> leak related to fragmentation, but there is still some memory problem
>> that's very tricky to locate. For example, you can run testspeed on a
>> large file several times in a row and it could show the memory error.
>> Since network condition are very difficult to reproduce, I have to
>> look at the source of this issue, memory allocation. Currently it has
>> mm_debug option that could print out file and line number of each
>> memory call, but it's quite useless since we can't find the relevant
>> informaton with full screen of prints.
>> Here are some of my ideas for enhanced memory protection support:
>>
>> 1, when we allocate memory, we append some information at the end of
>> the buffer, which include filename, lineno of caller, and tag to
>> indicate what is used for and some padding to detect memory overwrite
>> problem.
>>
>> 2. add a command to print the current memory list with extended
>> information, this can be used to find memory leaks.
>>
>> 3. it's also a good idea to run the memory check in automated test to
>> locate potential issue.
> This is pretty easy to do leveraging some of the code I did for POSIX
> support. But:
> - Due to additional time and space required it should be done only when
> mm-debug is enabled.
> - The additional data has to be stored before rather than after the
> range since we store all the info before.
> - Integrating with automated tests isn't that easy since some memory is
> intentionally never freed i.a. disk cache or otherwise in use (i.a.
> terminal). We need exceptions for those.
> I have half-working patch, just needs few fixes.

Hi,

Some thoughts about this:

We can assign a sequence id for each allocated memory, then we can
search memory allocated from point A to B. This can be used to skip
memory allocated during initialization stage.

We can use special tag for memory that would stick around, for
example, the *prefix: *cache, *fs, *term, etc, then we can skip them
in auto tests.

-- 
Best wishes
Bean

Re: RFC: enhanced memory protection support

[Vladimir 'φ-coder/phcoder' Serbinenko]

[-- Attachment #1: Type: text/plain, Size: 2532 bytes --]

On 30.04.2012 21:55, Bean wrote:
> On Tue, May 1, 2012 at 1:24 AM, Vladimir 'φ-coder/phcoder' Serbinenko
> <phcoder@gmail.com> wrote:
>> On 30.04.2012 17:26, Bean wrote:
>>> Hi,
>>>
>>> While testing network function in efi mode, I've found several memory
>>> leak related to fragmentation, but there is still some memory problem
>>> that's very tricky to locate. For example, you can run testspeed on a
>>> large file several times in a row and it could show the memory error.
>>> Since network condition are very difficult to reproduce, I have to
>>> look at the source of this issue, memory allocation. Currently it has
>>> mm_debug option that could print out file and line number of each
>>> memory call, but it's quite useless since we can't find the relevant
>>> informaton with full screen of prints.
>>> Here are some of my ideas for enhanced memory protection support:
>>>
>>> 1, when we allocate memory, we append some information at the end of
>>> the buffer, which include filename, lineno of caller, and tag to
>>> indicate what is used for and some padding to detect memory overwrite
>>> problem.
>>>
>>> 2. add a command to print the current memory list with extended
>>> information, this can be used to find memory leaks.
>>>
>>> 3. it's also a good idea to run the memory check in automated test to
>>> locate potential issue.
>> This is pretty easy to do leveraging some of the code I did for POSIX
>> support. But:
>> - Due to additional time and space required it should be done only when
>> mm-debug is enabled.
>> - The additional data has to be stored before rather than after the
>> range since we store all the info before.
>> - Integrating with automated tests isn't that easy since some memory is
>> intentionally never freed i.a. disk cache or otherwise in use (i.a.
>> terminal). We need exceptions for those.
>> I have half-working patch, just needs few fixes.
> Hi,
>
> Some thoughts about this:
>
> We can assign a sequence id for each allocated memory, then we can
> search memory allocated from point A to B. This can be used to skip
> memory allocated during initialization stage.
>
> We can use special tag for memory that would stick around, for
> example, the *prefix: *cache, *fs, *term, etc, then we can skip them
> in auto tests.
That's overengineering since at the end we need just one information: do
we track this memory or don't we? So we just need grub_*alloc_notrack
>


-- 
Regards
Vladimir 'φ-coder/phcoder' Serbinenko



[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 294 bytes --]

Re: RFC: enhanced memory protection support

[Vladimir 'φ-coder/phcoder' Serbinenko]

[-- Attachment #1.1: Type: text/plain, Size: 2116 bytes --]

On 30.04.2012 19:24, Vladimir 'φ-coder/phcoder' Serbinenko wrote:
> On 30.04.2012 17:26, Bean wrote:
>> Hi,
>>
>> While testing network function in efi mode, I've found several memory
>> leak related to fragmentation, but there is still some memory problem
>> that's very tricky to locate. For example, you can run testspeed on a
>> large file several times in a row and it could show the memory error.
>> Since network condition are very difficult to reproduce, I have to
>> look at the source of this issue, memory allocation. Currently it has
>> mm_debug option that could print out file and line number of each
>> memory call, but it's quite useless since we can't find the relevant
>> informaton with full screen of prints.
>> Here are some of my ideas for enhanced memory protection support:
>>
>> 1, when we allocate memory, we append some information at the end of
>> the buffer, which include filename, lineno of caller, and tag to
>> indicate what is used for and some padding to detect memory overwrite
>> problem.
>>
>> 2. add a command to print the current memory list with extended
>> information, this can be used to find memory leaks.
>>
>> 3. it's also a good idea to run the memory check in automated test to
>> locate potential issue.
> This is pretty easy to do leveraging some of the code I did for POSIX
> support. But:
> - Due to additional time and space required it should be done only when
> mm-debug is enabled.
> - The additional data has to be stored before rather than after the
> range since we store all the info before.
> - Integrating with automated tests isn't that easy since some memory is
> intentionally never freed i.a. disk cache or otherwise in use (i.a.
> terminal). We need exceptions for those.
> I have half-working patch, just needs few fixes.
>
>
Patch attached. It still has a bug since printing the report to terminal
may change allocations and so iterator may become invalid. That's
another reason why terminal should be excluded altogether from memory
tracking.


-- 
Regards
Vladimir 'φ-coder/phcoder' Serbinenko


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1.2: mm1.diff --]
[-- Type: text/x-diff; name="mm1.diff", Size: 19546 bytes --]

=== modified file 'config.h.in'
--- config.h.in	2012-02-04 20:47:29 +0000
+++ config.h.in	2012-05-01 09:53:02 +0000
@@ -8,7 +8,11 @@
 #if defined (GRUB_UTIL) || !defined (GRUB_MACHINE)
 #include <config-util.h>
 #define NESTED_FUNC_ATTR
+#define MM_DEBUG 0
 #else
+
+/* Define to 1 if you enable memory manager debugging. */
+#define MM_DEBUG @MM_DEBUG@
 /* Define if C symbols get an underscore after compilation. */
 #define HAVE_ASM_USCORE @HAVE_ASM_USCORE@
 /* Define it to \"addr32\" or \"addr32;\" to make GAS happy.  */

=== modified file 'configure.ac'
--- configure.ac	2012-04-29 16:15:24 +0000
+++ configure.ac	2012-05-01 09:53:58 +0000
@@ -759,9 +759,14 @@
 # Memory manager debugging.
 AC_ARG_ENABLE([mm-debug],
 	      AS_HELP_STRING([--enable-mm-debug],
-                             [include memory manager debugging]),
-              [AC_DEFINE([MM_DEBUG], [1],
-                         [Define to 1 if you enable memory manager debugging.])])
+                             [include memory manager debugging]))
+if test x$enable_mm_debug = xyes; then
+  MM_DEBUG=1
+else
+  MM_DEBUG=0
+fi
+
+AC_SUBST([MM_DEBUG])
 
 AC_ARG_ENABLE([cache-stats],
 	      AS_HELP_STRING([--enable-cache-stats],
@@ -786,6 +791,10 @@
 	      [AS_HELP_STRING([--enable-grub-emu-pci],
                              [build and install the `grub-emu' debugging utility with PCI support (potentially dangerous) (default=no)])])
 
+if test "$platform" = emu && test "$MM_DEBUG" = 1; then
+   AC_MSG_ERROR([grub-emu doesn't support mm-debug])
+fi
+
 if test "$platform" = emu; then
   missing_ncurses=
 [# Check for curses libraries.]
@@ -1132,6 +1141,7 @@
 AM_CONDITIONAL([COND_APPLE_CC], [test x$TARGET_APPLE_CC = x1])
 AM_CONDITIONAL([COND_ENABLE_EFIEMU], [test x$enable_efiemu = xyes])
 AM_CONDITIONAL([COND_ENABLE_CACHE_STATS], [test x$DISK_CACHE_STATS = x1])
+AM_CONDITIONAL([COND_ENABLE_MM_DEBUG], [test x$MM_DEBUG = x1])
 
 AM_CONDITIONAL([COND_HAVE_ASM_USCORE], [test x$HAVE_ASM_USCORE = x1])
 AM_CONDITIONAL([COND_CYGWIN], [test x$host_os = xcygwin])

=== modified file 'grub-core/Makefile.core.def'
--- grub-core/Makefile.core.def	2012-04-01 19:35:18 +0000
+++ grub-core/Makefile.core.def	2012-04-30 18:22:33 +0000
@@ -1816,6 +1816,13 @@
 };
 
 module = {
+  name = printmem;
+  common = commands/printmem.c;
+  condition = COND_ENABLE_MM_DEBUG;
+  enable = noemu;
+};
+
+module = {
   name = adler32;
   common = lib/adler32.c;
 };

=== added file 'grub-core/commands/printmem.c'
--- grub-core/commands/printmem.c	1970-01-01 00:00:00 +0000
+++ grub-core/commands/printmem.c	2012-05-01 10:20:29 +0000
@@ -0,0 +1,75 @@
+/*
+ *  GRUB  --  GRand Unified Bootloader
+ *  Copyright (C) 2012  Free Software Foundation, Inc.
+ *
+ *  GRUB is free software: you can redistribute it and/or modify
+ *  it under the terms of the GNU General Public License as published by
+ *  the Free Software Foundation, either version 3 of the License, or
+ *  (at your option) any later version.
+ *
+ *  GRUB is distributed in the hope that it will be useful,
+ *  but WITHOUT ANY WARRANTY; without even the implied warranty of
+ *  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ *  GNU General Public License for more details.
+ *
+ *  You should have received a copy of the GNU General Public License
+ *  along with GRUB.  If not, see <http://www.gnu.org/licenses/>.
+ */
+
+#include <grub/types.h>
+#include <grub/misc.h>
+#include <grub/mm.h>
+#include <grub/err.h>
+#include <grub/dl.h>
+#include <grub/command.h>
+#include <grub/i18n.h>
+#include <grub/mm_private.h>
+
+GRUB_MOD_LICENSE ("GPLv3+");
+
+static grub_err_t
+grub_cmd_print_mem (grub_command_t ctxt __attribute__ ((unused)),
+		    int argc __attribute__ ((unused)),
+		    char **args __attribute__ ((unused)))
+{
+  grub_mm_header_t cur;
+  for (cur = grub_mm_allocated; cur; cur = cur->next)
+    {
+      struct grub_mm_alloc_info *info = (struct grub_mm_alloc_info *) (cur + 1);
+      if (grub_strcmp (info->file, "kern/dl.c") == 0
+	  || grub_strcmp (info->file, "kern/env.c") == 0
+	  || grub_strcmp (info->file, "kern/disk.c") == 0
+	  || grub_strcmp (info->file, "kern/command.c") == 0
+	  || grub_strcmp (info->file, "kern/parser.c") == 0
+	  || grub_strcmp (info->file, "normal/dyncmd.c") == 0
+	  || grub_strcmp (info->file, "normal/autofs.c") == 0
+	  || grub_strcmp (info->file, "normal/crypto.c") == 0
+	  || grub_strcmp (info->file, "normal/term.c") == 0
+	  || grub_strcmp (info->file, "normal/color.c") == 0
+	  || grub_strcmp (info->file, "script/script.c") == 0
+	  || grub_strcmp (info->file, "script/argv.c") == 0
+	  || grub_strcmp (info->file, "commands/extcmd.c") == 0)
+	continue;
+      /* TRANSLATORS: This line lists currently allocated memory.
+	 First two fiels are filename and line number.
+	 It's followed by the pointer and the size.  */
+      grub_printf_ (N_("%s:%d %p %lu B\n"), info->file, info->lineno, cur + 1,
+		    (unsigned long)
+		    ((cur->size << GRUB_MM_ALIGN_LOG2) - sizeof (*cur)
+		     - sizeof (*info)));
+    }
+  return 0;
+}
+
+static grub_command_t cmd;
+
+GRUB_MOD_INIT(printmem)
+{
+  cmd = grub_register_command ("printmem", grub_cmd_print_mem, 0,
+			       N_("Print currently used memory."));
+}
+
+GRUB_MOD_FINI(printmem)
+{
+  grub_unregister_command (cmd);
+}

=== modified file 'grub-core/kern/misc.c'
--- grub-core/kern/misc.c	2012-02-29 18:35:19 +0000
+++ grub-core/kern/misc.c	2012-05-01 10:08:34 +0000
@@ -436,22 +436,37 @@
   return num;
 }
 
+#if MM_DEBUG
+char *
+grub_debug_strdup (const char *file, int line, const char *s)
+#else
 char *
 grub_strdup (const char *s)
+#endif
 {
   grub_size_t len;
   char *p;
 
   len = grub_strlen (s) + 1;
-  p = (char *) grub_malloc (len);
+#if MM_DEBUG
+  p = (char *) grub_debug_malloc (file, line, len);
+#else
+  p = (char *) grub_malloc (len);  
+#endif
   if (! p)
     return 0;
 
   return grub_memcpy (p, s, len);
 }
 
+
+#if MM_DEBUG
+char *
+grub_debug_strndup (const char *file, int line, const char *s, grub_size_t n)
+#else
 char *
 grub_strndup (const char *s, grub_size_t n)
+#endif
 {
   grub_size_t len;
   char *p;
@@ -459,7 +474,11 @@
   len = grub_strlen (s);
   if (len > n)
     len = n;
+#if MM_DEBUG
+  p = (char *) grub_debug_malloc (file, line, len + 1);
+#else
   p = (char *) grub_malloc (len + 1);
+#endif
   if (! p)
     return 0;
 
@@ -1051,8 +1070,13 @@
   return ret;
 }
 
+#if MM_DEBUG
+char *
+grub_debug_xvasprintf (const char *file, int line, const char *fmt, va_list ap)
+#else
 char *
 grub_xvasprintf (const char *fmt, va_list ap)
+#endif
 {
   grub_size_t s, as = PREALLOC_SIZE;
   char *ret;
@@ -1060,7 +1084,11 @@
   while (1)
     {
       va_list ap2;
+#if MM_DEBUG
+      ret = grub_debug_malloc (file, line, as + 1);
+#else
       ret = grub_malloc (as + 1);
+#endif
       if (!ret)
 	return NULL;
 
@@ -1078,14 +1106,23 @@
     }
 }
 
+#if MM_DEBUG
+char *
+grub_debug_xasprintf (const char *file, int line, const char *fmt, ...)
+#else
 char *
 grub_xasprintf (const char *fmt, ...)
+#endif
 {
   va_list ap;
   char *ret;
 
   va_start (ap, fmt);
+#if MM_DEBUG
+  ret = grub_debug_xvasprintf (file, line, fmt, ap);
+#else
   ret = grub_xvasprintf (fmt, ap);
+#endif
   va_end (ap);
 
   return ret;

=== modified file 'grub-core/kern/mm.c'
--- grub-core/kern/mm.c	2012-02-03 13:20:31 +0000
+++ grub-core/kern/mm.c	2012-05-01 09:50:18 +0000
@@ -68,7 +68,7 @@
 #include <grub/i18n.h>
 #include <grub/mm_private.h>
 
-#ifdef MM_DEBUG
+#if MM_DEBUG
 # undef grub_malloc
 # undef grub_zalloc
 # undef grub_realloc
@@ -84,10 +84,13 @@
    to the header and a pointer to its region, respectively. PTR must
    be allocated.  */
 static void
-get_header_from_pointer (void *ptr, grub_mm_header_t *p, grub_mm_region_t *r)
+get_header_from_pointer (void *ptr_in, grub_mm_header_t *p, grub_mm_region_t *r)
 {
-  if ((grub_addr_t) ptr & (GRUB_MM_ALIGN - 1))
-    grub_fatal ("unaligned pointer %p", ptr);
+  void *ptr;
+  if ((grub_addr_t) ptr_in & (GRUB_MM_ALIGN - 1))
+    grub_fatal ("unaligned pointer %p", ptr_in);
+
+  ptr = (grub_mm_header_t) ptr_in - (GRUB_MM_CELLS_METADATA - 1);
 
   for (*r = grub_mm_base; *r; *r = (*r)->next)
     if ((grub_addr_t) ptr > (grub_addr_t) ((*r) + 1)
@@ -95,11 +98,11 @@
       break;
 
   if (! *r)
-    grub_fatal ("out of range pointer %p", ptr);
+    grub_fatal ("out of range pointer %p", ptr_in);
 
   *p = (grub_mm_header_t) ptr - 1;
   if ((*p)->magic != GRUB_MM_ALLOC_MAGIC)
-    grub_fatal ("alloc magic is broken at %p", *p);
+    grub_fatal ("alloc magic is broken at %p: 0x%x", *p, (*p)->magic);
 }
 
 /* Initialize a region starting from ADDR and whose size is SIZE,
@@ -150,6 +153,9 @@
 {
   grub_mm_header_t p, q;
 
+  COMPILE_TIME_ASSERT (sizeof (struct grub_mm_header) == GRUB_MM_ALIGN);
+  COMPILE_TIME_ASSERT (sizeof (struct grub_mm_region) == GRUB_MM_ALIGN);
+
   /* When everything is allocated side effect is that *first will have alloc
      magic marked, meaning that there is no room in this region.  */
   if ((*first)->magic == GRUB_MM_ALLOC_MAGIC)
@@ -160,7 +166,8 @@
     {
       grub_off_t extra;
 
-      extra = ((grub_addr_t) (p + 1) >> GRUB_MM_ALIGN_LOG2) % align;
+      extra = ((grub_addr_t) (p + GRUB_MM_CELLS_METADATA)
+	       >> GRUB_MM_ALIGN_LOG2) & (align - 1);
       if (extra)
 	extra = align - extra;
 
@@ -266,7 +273,7 @@
 	     pieces before this will be un-used.  */
 	  *first = q;
 
-	  return p + 1;
+	  return p + GRUB_MM_CELLS_METADATA;
 	}
 
       /* Search was completed without result.  */
@@ -282,7 +289,8 @@
 grub_memalign (grub_size_t align, grub_size_t size)
 {
   grub_mm_region_t r;
-  grub_size_t n = ((size + GRUB_MM_ALIGN - 1) >> GRUB_MM_ALIGN_LOG2) + 1;
+  grub_size_t n = ((size + GRUB_MM_ALIGN - 1) >> GRUB_MM_ALIGN_LOG2)
+    + GRUB_MM_CELLS_METADATA;
   int count = 0;
 
   if (!grub_mm_base)
@@ -435,7 +443,8 @@
     }
 
   /* FIXME: Not optimal.  */
-  n = ((size + GRUB_MM_ALIGN - 1) >> GRUB_MM_ALIGN_LOG2) + 1;
+  n = ((size + GRUB_MM_ALIGN - 1) >> GRUB_MM_ALIGN_LOG2)
+    + GRUB_MM_CELLS_METADATA;
   get_header_from_pointer (ptr, &p, &r);
 
   if (p->size >= n)
@@ -450,7 +459,7 @@
   return q;
 }
 
-#ifdef MM_DEBUG
+#if MM_DEBUG
 int grub_mm_debug = 0;
 
 void
@@ -510,6 +519,50 @@
   grub_printf ("\n");
 }
 
+grub_mm_header_t grub_mm_allocated;
+
+static void
+save_ref (void *ptr, int line, const char *file)
+{
+  grub_mm_header_t head;
+  struct grub_mm_alloc_info *info;
+
+  COMPILE_TIME_ASSERT ((sizeof (struct grub_mm_alloc_info)
+			& (GRUB_MM_ALIGN - 1)) == 0);
+
+  if (!ptr)
+    return;
+
+  head = ptr;
+  head -= GRUB_MM_CELLS_METADATA;
+  info = (struct grub_mm_alloc_info *) (head + 1);
+  head->next = grub_mm_allocated;
+  if (head->next)
+    head->next->prev = head;
+  head->prev = 0;
+  info->lineno = line;
+  grub_strncpy (info->file, file, sizeof (info->file));
+  grub_mm_allocated = head;
+}
+
+static void
+delete_ref (void *ptr)
+{
+  grub_mm_header_t head;
+
+  if (!ptr)
+    return;
+
+  head = ptr;
+  head -= GRUB_MM_CELLS_METADATA;
+  if (head->next)
+    head->next->prev = head->prev;
+  if (head->prev)
+    head->prev->next = head->next;
+  else if (grub_mm_allocated == head)
+    grub_mm_allocated = head->next;
+}
+
 void *
 grub_debug_malloc (const char *file, int line, grub_size_t size)
 {
@@ -518,6 +571,7 @@
   if (grub_mm_debug)
     grub_printf ("%s:%d: malloc (0x%" PRIxGRUB_SIZE ") = ", file, line, size);
   ptr = grub_malloc (size);
+  save_ref (ptr, line, file);
   if (grub_mm_debug)
     grub_printf ("%p\n", ptr);
   return ptr;
@@ -531,6 +585,7 @@
   if (grub_mm_debug)
     grub_printf ("%s:%d: zalloc (0x%" PRIxGRUB_SIZE ") = ", file, line, size);
   ptr = grub_zalloc (size);
+  save_ref (ptr, line, file);
   if (grub_mm_debug)
     grub_printf ("%p\n", ptr);
   return ptr;
@@ -539,6 +594,7 @@
 void
 grub_debug_free (const char *file, int line, void *ptr)
 {
+  delete_ref (ptr);
   if (grub_mm_debug)
     grub_printf ("%s:%d: free (%p)\n", file, line, ptr);
   grub_free (ptr);
@@ -547,9 +603,11 @@
 void *
 grub_debug_realloc (const char *file, int line, void *ptr, grub_size_t size)
 {
+  delete_ref (ptr);
   if (grub_mm_debug)
     grub_printf ("%s:%d: realloc (%p, 0x%" PRIxGRUB_SIZE ") = ", file, line, ptr, size);
   ptr = grub_realloc (ptr, size);
+  save_ref (ptr, line, file);
   if (grub_mm_debug)
     grub_printf ("%p\n", ptr);
   return ptr;

=== modified file 'grub-core/lib/relocator.c'
--- grub-core/lib/relocator.c	2012-04-13 14:55:20 +0000
+++ grub-core/lib/relocator.c	2012-05-01 09:34:38 +0000
@@ -358,7 +358,9 @@
 	    r2->first = r1->first;
 	    hl->next = r2->first;
 	    *rp = (*rp)->next;
-	    grub_free (g + 1);
+	    g->next = 0;
+	    g->prev = 0;
+	    grub_free (g + GRUB_MM_CELLS_METADATA);
 	  }
 	break;
       }
@@ -368,10 +370,11 @@
 							    GRUB_MM_ALIGN);
 	h->size
 	  = ((subchu->start + subchu->size + GRUB_MM_ALIGN - 1) / GRUB_MM_ALIGN)
-	  - (subchu->start / GRUB_MM_ALIGN) - 1;
-	h->next = h;
+	  - (subchu->start / GRUB_MM_ALIGN);
+	h->next = 0;
+	h->prev = 0;
 	h->magic = GRUB_MM_ALLOC_MAGIC;
-	grub_free (h + 1);
+	grub_free (h + GRUB_MM_CELLS_METADATA);
 	break;
       }
 #if GRUB_RELOCATOR_HAVE_FIRMWARE_REQUESTS
@@ -428,6 +431,9 @@
   unsigned j, N = 0;
   grub_addr_t target = 0;
 
+  COMPILE_TIME_ASSERT (sizeof (struct grub_mm_header) == GRUB_MM_ALIGN);
+  COMPILE_TIME_ASSERT (sizeof (struct grub_mm_region) == GRUB_MM_ALIGN);
+
   grub_dprintf ("relocator",
 		"trying to allocate in 0x%lx-0x%lx aligned 0x%lx size 0x%lx\n",
 		(unsigned long) start, (unsigned long) end,

=== modified file 'grub-core/normal/main.c'
--- grub-core/normal/main.c	2012-03-11 13:43:18 +0000
+++ grub-core/normal/main.c	2012-05-01 10:16:23 +0000
@@ -41,8 +41,13 @@
 int grub_normal_exit_level = 0;
 
 /* Read a line from the file FILE.  */
+#if MM_DEBUG
+char *
+grub_file_debug_getline (const char *cfile, int line, grub_file_t file)
+#else
 char *
 grub_file_getline (grub_file_t file)
+#endif
 {
   char c;
   grub_size_t pos = 0;
@@ -51,7 +56,11 @@
   grub_size_t max_len = 64;
 
   /* Initially locate some space.  */
-  cmdline = grub_malloc (max_len);
+#if MM_DEBUG
+  cmdline = grub_debug_malloc (cfile, line, max_len);
+#else
+  cmdline = grub_debug_malloc (max_len);
+#endif
   if (! cmdline)
     return 0;
 
@@ -69,7 +78,11 @@
 	{
 	  char *old_cmdline = cmdline;
 	  max_len = max_len * 2;
+#if MM_DEBUG
+	  cmdline = grub_debug_realloc (cfile, line, cmdline, max_len);
+#else
 	  cmdline = grub_realloc (cmdline, max_len);
+#endif
 	  if (! cmdline)
 	    {
 	      grub_free (old_cmdline);

=== modified file 'include/grub/misc.h'
--- include/grub/misc.h	2012-03-05 00:17:55 +0000
+++ include/grub/misc.h	2012-05-01 10:07:51 +0000
@@ -314,8 +314,16 @@
     }
 }
 
+#if MM_DEBUG
+char *EXPORT_FUNC(grub_debug_strdup) (const char *file, int line, const char *s) __attribute__ ((warn_unused_result));
+char *EXPORT_FUNC(grub_debug_strndup) (const char *file, int line, const char *s, grub_size_t n) __attribute__ ((warn_unused_result));
+
+#define grub_strdup(s) grub_debug_strdup (GRUB_FILE, __LINE__, s)
+#define grub_strndup(s, n) grub_debug_strndup (GRUB_FILE, __LINE__, s, n)
+#else
 char *EXPORT_FUNC(grub_strdup) (const char *s) __attribute__ ((warn_unused_result));
 char *EXPORT_FUNC(grub_strndup) (const char *s, grub_size_t n) __attribute__ ((warn_unused_result));
+#endif
 void *EXPORT_FUNC(grub_memset) (void *s, int c, grub_size_t n);
 grub_size_t EXPORT_FUNC(grub_strlen) (const char *s) __attribute__ ((warn_unused_result));
 int EXPORT_FUNC(grub_printf) (const char *fmt, ...) __attribute__ ((format (printf, 1, 2)));
@@ -363,9 +371,17 @@
      __attribute__ ((format (printf, 3, 4)));
 int EXPORT_FUNC(grub_vsnprintf) (char *str, grub_size_t n, const char *fmt,
 				 va_list args);
+#if MM_DEBUG
+char *EXPORT_FUNC(grub_debug_xasprintf) (const char *file, int line, const char *fmt, ...)
+     __attribute__ ((format (printf, 3, 4))) __attribute__ ((warn_unused_result));
+char *EXPORT_FUNC(grub_debug_xvasprintf) (const char *file, int line, const char *fmt, va_list args) __attribute__ ((warn_unused_result));
+#define grub_xasprintf(fmt, args...) grub_debug_xasprintf (GRUB_FILE, __LINE__, fmt, ## args)
+#define grub_xvasprintf(fmt, args) grub_debug_xvasprintf (GRUB_FILE, __LINE__, fmt, args)
+#else
 char *EXPORT_FUNC(grub_xasprintf) (const char *fmt, ...)
      __attribute__ ((format (printf, 1, 2))) __attribute__ ((warn_unused_result));
 char *EXPORT_FUNC(grub_xvasprintf) (const char *fmt, va_list args) __attribute__ ((warn_unused_result));
+#endif
 void EXPORT_FUNC(grub_exit) (void) __attribute__ ((noreturn));
 void EXPORT_FUNC(grub_abort) (void) __attribute__ ((noreturn));
 grub_uint64_t EXPORT_FUNC(grub_divmod64) (grub_uint64_t n,

=== modified file 'include/grub/mm.h'
--- include/grub/mm.h	2011-11-13 11:48:39 +0000
+++ include/grub/mm.h	2012-05-01 09:53:09 +0000
@@ -39,7 +39,7 @@
 #define grub_mm_check() grub_mm_check_real (GRUB_FILE, __LINE__);
 
 /* For debugging.  */
-#if defined(MM_DEBUG) && !defined(GRUB_UTIL) && !defined (GRUB_MACHINE_EMU)
+#if MM_DEBUG
 /* Set this variable to 1 when you want to trace all memory function calls.  */
 extern int EXPORT_VAR(grub_mm_debug);
 

=== modified file 'include/grub/mm_private.h'
--- include/grub/mm_private.h	2010-08-28 12:52:25 +0000
+++ include/grub/mm_private.h	2012-05-01 09:46:58 +0000
@@ -30,13 +30,7 @@
   struct grub_mm_header *next;
   grub_size_t size;
   grub_size_t magic;
-#if GRUB_CPU_SIZEOF_VOID_P == 4
-  char padding[4];
-#elif GRUB_CPU_SIZEOF_VOID_P == 8
-  char padding[8];
-#else
-# error "unknown word size"
-#endif
+  struct grub_mm_header *prev;
 }
 *grub_mm_header_t;
 
@@ -48,6 +42,17 @@
 
 #define GRUB_MM_ALIGN	(1 << GRUB_MM_ALIGN_LOG2)
 
+#if MM_DEBUG
+struct grub_mm_alloc_info
+{
+  int lineno;
+  char file[128 - sizeof (int)];
+};
+static const grub_size_t GRUB_MM_CELLS_METADATA = ((sizeof (struct grub_mm_alloc_info) >> GRUB_MM_ALIGN_LOG2) + 1);
+#else
+#define GRUB_MM_CELLS_METADATA 1
+#endif
+
 typedef struct grub_mm_region
 {
   struct grub_mm_header *first;
@@ -61,4 +66,8 @@
 extern grub_mm_region_t EXPORT_VAR (grub_mm_base);
 #endif
 
+#if MM_DEBUG
+extern grub_mm_header_t EXPORT_VAR(grub_mm_allocated);
+#endif
+
 #endif

=== modified file 'include/grub/normal.h'
--- include/grub/normal.h	2012-03-10 22:25:34 +0000
+++ include/grub/normal.h	2012-05-01 10:15:01 +0000
@@ -51,7 +51,14 @@
 void grub_menu_init_page (int nested, int edit, int *num_entries,
 			  struct grub_term_output *term);
 void grub_normal_init_page (struct grub_term_output *term);
+#if MM_DEBUG
+char *grub_file_debug_getline (const char *call_file,
+			       int line, grub_file_t file);
+#define grub_file_getline(file)	\
+  grub_file_debug_getline (GRUB_FILE, __LINE__, file)
+#else
 char *grub_file_getline (grub_file_t file);
+#endif
 void grub_cmdline_run (int nested);
 
 /* Defined in `cmdline.c'.  */


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 294 bytes --]

Re: RFC: enhanced memory protection support

[Vladimir 'φ-coder/phcoder' Serbinenko]

[-- Attachment #1.1: Type: text/plain, Size: 2236 bytes --]

On 01.05.2012 12:29, Vladimir 'φ-coder/phcoder' Serbinenko wrote:
> On 30.04.2012 19:24, Vladimir 'φ-coder/phcoder' Serbinenko wrote:
>> On 30.04.2012 17:26, Bean wrote:
>>> Hi,
>>>
>>> While testing network function in efi mode, I've found several memory
>>> leak related to fragmentation, but there is still some memory problem
>>> that's very tricky to locate. For example, you can run testspeed on a
>>> large file several times in a row and it could show the memory error.
>>> Since network condition are very difficult to reproduce, I have to
>>> look at the source of this issue, memory allocation. Currently it has
>>> mm_debug option that could print out file and line number of each
>>> memory call, but it's quite useless since we can't find the relevant
>>> informaton with full screen of prints.
>>> Here are some of my ideas for enhanced memory protection support:
>>>
>>> 1, when we allocate memory, we append some information at the end of
>>> the buffer, which include filename, lineno of caller, and tag to
>>> indicate what is used for and some padding to detect memory overwrite
>>> problem.
>>>
>>> 2. add a command to print the current memory list with extended
>>> information, this can be used to find memory leaks.
>>>
>>> 3. it's also a good idea to run the memory check in automated test to
>>> locate potential issue.
>> This is pretty easy to do leveraging some of the code I did for POSIX
>> support. But:
>> - Due to additional time and space required it should be done only when
>> mm-debug is enabled.
>> - The additional data has to be stored before rather than after the
>> range since we store all the info before.
>> - Integrating with automated tests isn't that easy since some memory is
>> intentionally never freed i.a. disk cache or otherwise in use (i.a.
>> terminal). We need exceptions for those.
>> I have half-working patch, just needs few fixes.
>>
>>
> Patch attached. It still has a bug since printing the report to terminal
> may change allocations and so iterator may become invalid. That's
> another reason why terminal should be excluded altogether from memory
> tracking.
>
>


-- 
Regards
Vladimir 'φ-coder/phcoder' Serbinenko


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1.2: mm2.diff --]
[-- Type: text/x-diff; name="mm2.diff", Size: 44199 bytes --]

=== modified file 'config.h.in'
--- config.h.in	2012-02-04 20:47:29 +0000
+++ config.h.in	2012-05-01 09:53:02 +0000
@@ -8,7 +8,11 @@
 #if defined (GRUB_UTIL) || !defined (GRUB_MACHINE)
 #include <config-util.h>
 #define NESTED_FUNC_ATTR
+#define MM_DEBUG 0
 #else
+
+/* Define to 1 if you enable memory manager debugging. */
+#define MM_DEBUG @MM_DEBUG@
 /* Define if C symbols get an underscore after compilation. */
 #define HAVE_ASM_USCORE @HAVE_ASM_USCORE@
 /* Define it to \"addr32\" or \"addr32;\" to make GAS happy.  */

=== modified file 'configure.ac'
--- configure.ac	2012-04-29 16:15:24 +0000
+++ configure.ac	2012-05-01 09:53:58 +0000
@@ -759,9 +759,14 @@
 # Memory manager debugging.
 AC_ARG_ENABLE([mm-debug],
 	      AS_HELP_STRING([--enable-mm-debug],
-                             [include memory manager debugging]),
-              [AC_DEFINE([MM_DEBUG], [1],
-                         [Define to 1 if you enable memory manager debugging.])])
+                             [include memory manager debugging]))
+if test x$enable_mm_debug = xyes; then
+  MM_DEBUG=1
+else
+  MM_DEBUG=0
+fi
+
+AC_SUBST([MM_DEBUG])
 
 AC_ARG_ENABLE([cache-stats],
 	      AS_HELP_STRING([--enable-cache-stats],
@@ -786,6 +791,10 @@
 	      [AS_HELP_STRING([--enable-grub-emu-pci],
                              [build and install the `grub-emu' debugging utility with PCI support (potentially dangerous) (default=no)])])
 
+if test "$platform" = emu && test "$MM_DEBUG" = 1; then
+   AC_MSG_ERROR([grub-emu doesn't support mm-debug])
+fi
+
 if test "$platform" = emu; then
   missing_ncurses=
 [# Check for curses libraries.]
@@ -1132,6 +1141,7 @@
 AM_CONDITIONAL([COND_APPLE_CC], [test x$TARGET_APPLE_CC = x1])
 AM_CONDITIONAL([COND_ENABLE_EFIEMU], [test x$enable_efiemu = xyes])
 AM_CONDITIONAL([COND_ENABLE_CACHE_STATS], [test x$DISK_CACHE_STATS = x1])
+AM_CONDITIONAL([COND_ENABLE_MM_DEBUG], [test x$MM_DEBUG = x1])
 
 AM_CONDITIONAL([COND_HAVE_ASM_USCORE], [test x$HAVE_ASM_USCORE = x1])
 AM_CONDITIONAL([COND_CYGWIN], [test x$host_os = xcygwin])

=== modified file 'grub-core/Makefile.core.def'
--- grub-core/Makefile.core.def	2012-04-01 19:35:18 +0000
+++ grub-core/Makefile.core.def	2012-04-30 18:22:33 +0000
@@ -1816,6 +1816,13 @@
 };
 
 module = {
+  name = printmem;
+  common = commands/printmem.c;
+  condition = COND_ENABLE_MM_DEBUG;
+  enable = noemu;
+};
+
+module = {
   name = adler32;
   common = lib/adler32.c;
 };

=== modified file 'grub-core/commands/boot.c'
--- grub-core/commands/boot.c	2012-04-11 20:32:31 +0000
+++ grub-core/commands/boot.c	2012-05-01 11:29:30 +0000
@@ -62,7 +62,7 @@
     return 0;
 
   new_preboot = (struct grub_preboot *)
-    grub_malloc (sizeof (struct grub_preboot));
+    grub_malloc_notrack (sizeof (struct grub_preboot));
   if (! new_preboot)
     return 0;
 

=== modified file 'grub-core/commands/extcmd.c'
--- grub-core/commands/extcmd.c	2011-04-11 21:01:51 +0000
+++ grub-core/commands/extcmd.c	2012-05-01 11:02:53 +0000
@@ -77,7 +77,7 @@
   grub_extcmd_t ext;
   grub_command_t cmd;
 
-  ext = (grub_extcmd_t) grub_malloc (sizeof (*ext));
+  ext = (grub_extcmd_t) grub_malloc_notrack (sizeof (*ext));
   if (! ext)
     return 0;
 

=== added file 'grub-core/commands/printmem.c'
--- grub-core/commands/printmem.c	1970-01-01 00:00:00 +0000
+++ grub-core/commands/printmem.c	2012-05-01 10:57:02 +0000
@@ -0,0 +1,61 @@
+/*
+ *  GRUB  --  GRand Unified Bootloader
+ *  Copyright (C) 2012  Free Software Foundation, Inc.
+ *
+ *  GRUB is free software: you can redistribute it and/or modify
+ *  it under the terms of the GNU General Public License as published by
+ *  the Free Software Foundation, either version 3 of the License, or
+ *  (at your option) any later version.
+ *
+ *  GRUB is distributed in the hope that it will be useful,
+ *  but WITHOUT ANY WARRANTY; without even the implied warranty of
+ *  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ *  GNU General Public License for more details.
+ *
+ *  You should have received a copy of the GNU General Public License
+ *  along with GRUB.  If not, see <http://www.gnu.org/licenses/>.
+ */
+
+#include <grub/types.h>
+#include <grub/misc.h>
+#include <grub/mm.h>
+#include <grub/err.h>
+#include <grub/dl.h>
+#include <grub/command.h>
+#include <grub/i18n.h>
+#include <grub/mm_private.h>
+
+GRUB_MOD_LICENSE ("GPLv3+");
+
+static grub_err_t
+grub_cmd_print_mem (grub_command_t ctxt __attribute__ ((unused)),
+		    int argc __attribute__ ((unused)),
+		    char **args __attribute__ ((unused)))
+{
+  grub_mm_header_t cur;
+  for (cur = grub_mm_allocated; cur; cur = cur->next)
+    {
+      struct grub_mm_alloc_info *info = (struct grub_mm_alloc_info *) (cur + 1);
+      /* TRANSLATORS: This line lists currently allocated memory.
+	 First two fiels are filename and line number.
+	 It's followed by the pointer and the size.  */
+      grub_printf_ (N_("%s:%d %p %lu B\n"), info->file, info->lineno, cur + 1,
+		    (unsigned long)
+		    ((cur->size << GRUB_MM_ALIGN_LOG2) - sizeof (*cur)
+		     - sizeof (*info)));
+    }
+  return 0;
+}
+
+static grub_command_t cmd;
+
+GRUB_MOD_INIT(printmem)
+{
+  cmd = grub_register_command ("printmem", grub_cmd_print_mem, 0,
+			       N_("Print currently used memory."));
+}
+
+GRUB_MOD_FINI(printmem)
+{
+  grub_unregister_command (cmd);
+}

=== modified file 'grub-core/font/font.c'
--- grub-core/font/font.c	2012-04-01 19:27:17 +0000
+++ grub-core/font/font.c	2012-05-01 12:02:33 +0000
@@ -146,7 +146,8 @@
       for (current = 0; current < 0x80; current++)
 	{
 	  ascii_font_glyph[current] =
-	    grub_malloc (sizeof (struct grub_font_glyph) + ASCII_BITMAP_SIZE);
+	    grub_malloc_notrack (sizeof (struct grub_font_glyph)
+				 + ASCII_BITMAP_SIZE);
 
 	  ascii_font_glyph[current]->width = 8;
 	  ascii_font_glyph[current]->height = 16;
@@ -178,8 +179,8 @@
     return;
 
   /* Make glyph for unknown glyph.  */
-  unknown_glyph = grub_malloc (sizeof (struct grub_font_glyph)
-			       + sizeof (unknown_glyph_bitmap));
+  unknown_glyph = grub_malloc_notrack (sizeof (struct grub_font_glyph)
+				       + sizeof (unknown_glyph_bitmap));
   if (!unknown_glyph)
     return;
 
@@ -310,11 +311,11 @@
   font->num_chars = sect_length / FONT_CHAR_INDEX_ENTRY_SIZE;
 
   /* Allocate the character index array.  */
-  font->char_index = grub_malloc (font->num_chars
-				  * sizeof (struct char_index_entry));
+  font->char_index = grub_malloc_notrack (font->num_chars
+					  * sizeof (struct char_index_entry));
   if (!font->char_index)
     return 1;
-  font->bmp_idx = grub_malloc (0x10000 * sizeof (grub_uint16_t));
+  font->bmp_idx = grub_malloc_notrack (0x10000 * sizeof (grub_uint16_t));
   if (!font->bmp_idx)
     return 1;
   grub_memset (font->bmp_idx, 0xff, 0x10000 * sizeof (grub_uint16_t));
@@ -380,7 +381,7 @@
   char *str;
   grub_ssize_t ret;
 
-  str = grub_malloc (section->length + 1);
+  str = grub_malloc_notrack (section->length + 1);
   if (!str)
     return 0;
 
@@ -514,7 +515,7 @@
 #endif
 
   /* Allocate the font object.  */
-  font = (grub_font_t) grub_zalloc (sizeof (struct grub_font));
+  font = (grub_font_t) grub_zalloc_notrack (sizeof (struct grub_font));
   if (!font)
     goto fail;
 
@@ -774,7 +775,7 @@
 	}
 
       len = (width * height + 7) / 8;
-      glyph = grub_malloc (sizeof (struct grub_font_glyph) + len);
+      glyph = grub_malloc_notrack (sizeof (struct grub_font_glyph) + len);
       if (!glyph)
 	{
 	  remove_font (font);
@@ -837,7 +838,7 @@
 {
   struct grub_font_node *node = 0;
 
-  node = grub_malloc (sizeof (struct grub_font_node));
+  node = grub_malloc_notrack (sizeof (struct grub_font_node));
   if (!node)
     return 1;
 

=== modified file 'grub-core/fs/fshelp.c'
--- grub-core/fs/fshelp.c	2012-02-08 18:26:01 +0000
+++ grub-core/fs/fshelp.c	2012-05-01 11:39:01 +0000
@@ -136,6 +136,7 @@
 	  found = iterate_dir (currnode, iterate);
 	  if (! found)
 	    {
+	      free_node (currnode);
 	      if (grub_errno)
 		return grub_errno;
 

=== modified file 'grub-core/gettext/gettext.c'
--- grub-core/gettext/gettext.c	2012-04-04 12:41:55 +0000
+++ grub-core/gettext/gettext.c	2012-05-01 12:06:28 +0000
@@ -109,7 +109,7 @@
   length = grub_cpu_to_le32 (desc.length);
   offset = grub_cpu_to_le32 (desc.offset);
 
-  translation = grub_malloc (length + 1);
+  translation = grub_malloc_notrack (length + 1);
   if (!translation)
     return NULL;
 
@@ -323,8 +323,8 @@
   for (ctx->grub_gettext_max_log = 0; ctx->grub_gettext_max >> ctx->grub_gettext_max_log;
        ctx->grub_gettext_max_log++);
 
-  ctx->grub_gettext_msg_list = grub_zalloc (ctx->grub_gettext_max
-					    * sizeof (ctx->grub_gettext_msg_list[0]));
+  ctx->grub_gettext_msg_list = grub_zalloc_notrack (ctx->grub_gettext_max
+						    * sizeof (ctx->grub_gettext_msg_list[0]));
   if (!ctx->grub_gettext_msg_list)
     {
       grub_file_close (fd);
@@ -430,7 +430,7 @@
   if (err)
     grub_print_error ();
 
-  return grub_strdup (val);
+  return grub_strdup_notrack (val);
 }
 
 void

=== modified file 'grub-core/kern/command.c'
--- grub-core/kern/command.c	2012-02-12 02:52:17 +0000
+++ grub-core/kern/command.c	2012-05-01 11:01:46 +0000
@@ -34,7 +34,7 @@
 
   grub_command_t *p, q;
 
-  cmd = (grub_command_t) grub_zalloc (sizeof (*cmd));
+  cmd = (grub_command_t) grub_zalloc_notrack (sizeof (*cmd));
   if (! cmd)
     return 0;
 

=== modified file 'grub-core/kern/disk.c'
--- grub-core/kern/disk.c	2012-03-03 11:59:28 +0000
+++ grub-core/kern/disk.c	2012-05-01 10:56:51 +0000
@@ -163,7 +163,8 @@
   cache->data = 0;
   cache->lock = 0;
 
-  cache->data = grub_malloc (GRUB_DISK_SECTOR_SIZE << GRUB_DISK_CACHE_BITS);
+  cache->data = grub_malloc_notrack (GRUB_DISK_SECTOR_SIZE
+				     << GRUB_DISK_CACHE_BITS);
   if (! cache->data)
     return grub_errno;
 

=== modified file 'grub-core/kern/dl.c'
--- grub-core/kern/dl.c	2012-03-19 10:10:11 +0000
+++ grub-core/kern/dl.c	2012-05-01 11:01:29 +0000
@@ -141,13 +141,13 @@
   grub_symbol_t sym;
   unsigned k;
 
-  sym = (grub_symbol_t) grub_malloc (sizeof (*sym));
+  sym = (grub_symbol_t) grub_malloc_notrack (sizeof (*sym));
   if (! sym)
     return grub_errno;
 
   if (mod)
     {
-      sym->name = grub_strdup (name);
+      sym->name = grub_strdup_notrack (name);
       if (! sym->name)
 	{
 	  grub_free (sym);
@@ -276,7 +276,7 @@
   tsize = ALIGN_UP (tsize, 8192 * 16);
 #endif
 
-  mod->base = grub_memalign (talign, tsize);
+  mod->base = grub_memalign_notrack (talign, tsize);
   if (!mod->base)
     return grub_errno;
   mod->sz = tsize;
@@ -294,7 +294,7 @@
 	{
 	  grub_dl_segment_t seg;
 
-	  seg = (grub_dl_segment_t) grub_malloc (sizeof (*seg));
+	  seg = (grub_dl_segment_t) grub_malloc_notrack (sizeof (*seg));
 	  if (! seg)
 	    return grub_errno;
 
@@ -499,7 +499,7 @@
        i++, s = (Elf_Shdr *) ((char *) s + e->e_shentsize))
     if (grub_strcmp (str + s->sh_name, ".modname") == 0)
       {
-	mod->name = grub_strdup ((char *) e + s->sh_offset);
+	mod->name = grub_strdup_notrack ((char *) e + s->sh_offset);
 	if (! mod->name)
 	  return grub_errno;
 	break;
@@ -540,7 +540,7 @@
 
 	    grub_dl_ref (m);
 
-	    dep = (grub_dl_dep_t) grub_malloc (sizeof (*dep));
+	    dep = (grub_dl_dep_t) grub_malloc_notrack (sizeof (*dep));
 	    if (! dep)
 	      return grub_errno;
 
@@ -617,7 +617,7 @@
       return 0;
     }
 
-  mod = (grub_dl_t) grub_zalloc (sizeof (*mod));
+  mod = (grub_dl_t) grub_zalloc_notrack (sizeof (*mod));
   if (! mod)
     return 0;
 

=== modified file 'grub-core/kern/env.c'
--- grub-core/kern/env.c	2012-02-26 22:53:21 +0000
+++ grub-core/kern/env.c	2012-05-01 11:23:52 +0000
@@ -92,7 +92,7 @@
       if (var->write_hook)
 	var->value = var->write_hook (var, val);
       else
-	var->value = grub_strdup (val);
+	var->value = grub_strdup_notrack (val);
 
       if (! var->value)
 	{
@@ -105,15 +105,15 @@
     }
 
   /* The variable does not exist, so create a new one.  */
-  var = grub_zalloc (sizeof (*var));
+  var = grub_zalloc_notrack (sizeof (*var));
   if (! var)
     return grub_errno;
 
-  var->name = grub_strdup (name);
+  var->name = grub_strdup_notrack (name);
   if (! var->name)
     goto fail;
 
-  var->value = grub_strdup (val);
+  var->value = grub_strdup_notrack (val);
   if (! var->value)
     goto fail;
 

=== modified file 'grub-core/kern/main.c'
--- grub-core/kern/main.c	2011-12-13 14:13:51 +0000
+++ grub-core/kern/main.c	2012-05-01 11:10:06 +0000
@@ -95,7 +95,7 @@
   if (val[0] == '(' && val[len - 1] == ')')
     return grub_strndup (val + 1, len - 2);
 
-  return grub_strdup (val);
+  return grub_strdup_notrack (val);
 }
 
 static void

=== modified file 'grub-core/kern/misc.c'
--- grub-core/kern/misc.c	2012-02-29 18:35:19 +0000
+++ grub-core/kern/misc.c	2012-05-01 10:54:23 +0000
@@ -437,21 +437,42 @@
 }
 
 char *
-grub_strdup (const char *s)
-{
-  grub_size_t len;
-  char *p;
-
-  len = grub_strlen (s) + 1;
-  p = (char *) grub_malloc (len);
-  if (! p)
-    return 0;
-
-  return grub_memcpy (p, s, len);
-}
-
+grub_strdup_notrack (const char *s)
+{
+  grub_size_t len;
+  char *p;
+
+  len = grub_strlen (s) + 1;
+  p = (char *) grub_malloc_notrack (len);  
+  if (! p)
+    return 0;
+
+  return grub_memcpy (p, s, len);
+}
+
+#if MM_DEBUG
+char *
+grub_debug_strdup (const char *file, int line, const char *s)
+{
+  grub_size_t len;
+  char *p;
+
+  len = grub_strlen (s) + 1;
+  p = (char *) grub_debug_malloc (file, line, len);
+  if (! p)
+    return 0;
+
+  return grub_memcpy (p, s, len);
+}
+#endif
+
+#if MM_DEBUG
+char *
+grub_debug_strndup (const char *file, int line, const char *s, grub_size_t n)
+#else
 char *
 grub_strndup (const char *s, grub_size_t n)
+#endif
 {
   grub_size_t len;
   char *p;
@@ -459,7 +480,11 @@
   len = grub_strlen (s);
   if (len > n)
     len = n;
+#if MM_DEBUG
+  p = (char *) grub_debug_malloc (file, line, len + 1);
+#else
   p = (char *) grub_malloc (len + 1);
+#endif
   if (! p)
     return 0;
 
@@ -1051,8 +1076,13 @@
   return ret;
 }
 
+#if MM_DEBUG
+char *
+grub_debug_xvasprintf (const char *file, int line, const char *fmt, va_list ap)
+#else
 char *
 grub_xvasprintf (const char *fmt, va_list ap)
+#endif
 {
   grub_size_t s, as = PREALLOC_SIZE;
   char *ret;
@@ -1060,7 +1090,11 @@
   while (1)
     {
       va_list ap2;
+#if MM_DEBUG
+      ret = grub_debug_malloc (file, line, as + 1);
+#else
       ret = grub_malloc (as + 1);
+#endif
       if (!ret)
 	return NULL;
 
@@ -1078,14 +1112,23 @@
     }
 }
 
+#if MM_DEBUG
+char *
+grub_debug_xasprintf (const char *file, int line, const char *fmt, ...)
+#else
 char *
 grub_xasprintf (const char *fmt, ...)
+#endif
 {
   va_list ap;
   char *ret;
 
   va_start (ap, fmt);
+#if MM_DEBUG
+  ret = grub_debug_xvasprintf (file, line, fmt, ap);
+#else
   ret = grub_xvasprintf (fmt, ap);
+#endif
   va_end (ap);
 
   return ret;

=== modified file 'grub-core/kern/mm.c'
--- grub-core/kern/mm.c	2012-02-03 13:20:31 +0000
+++ grub-core/kern/mm.c	2012-05-01 11:08:22 +0000
@@ -68,7 +68,7 @@
 #include <grub/i18n.h>
 #include <grub/mm_private.h>
 
-#ifdef MM_DEBUG
+#if MM_DEBUG
 # undef grub_malloc
 # undef grub_zalloc
 # undef grub_realloc
@@ -84,10 +84,13 @@
    to the header and a pointer to its region, respectively. PTR must
    be allocated.  */
 static void
-get_header_from_pointer (void *ptr, grub_mm_header_t *p, grub_mm_region_t *r)
+get_header_from_pointer (void *ptr_in, grub_mm_header_t *p, grub_mm_region_t *r)
 {
-  if ((grub_addr_t) ptr & (GRUB_MM_ALIGN - 1))
-    grub_fatal ("unaligned pointer %p", ptr);
+  void *ptr;
+  if ((grub_addr_t) ptr_in & (GRUB_MM_ALIGN - 1))
+    grub_fatal ("unaligned pointer %p", ptr_in);
+
+  ptr = (grub_mm_header_t) ptr_in - (GRUB_MM_CELLS_METADATA - 1);
 
   for (*r = grub_mm_base; *r; *r = (*r)->next)
     if ((grub_addr_t) ptr > (grub_addr_t) ((*r) + 1)
@@ -95,11 +98,11 @@
       break;
 
   if (! *r)
-    grub_fatal ("out of range pointer %p", ptr);
+    grub_fatal ("out of range pointer %p", ptr_in);
 
   *p = (grub_mm_header_t) ptr - 1;
   if ((*p)->magic != GRUB_MM_ALLOC_MAGIC)
-    grub_fatal ("alloc magic is broken at %p", *p);
+    grub_fatal ("alloc magic is broken at %p: 0x%x", *p, (*p)->magic);
 }
 
 /* Initialize a region starting from ADDR and whose size is SIZE,
@@ -150,9 +153,12 @@
 {
   grub_mm_header_t p, q;
 
+  COMPILE_TIME_ASSERT (sizeof (struct grub_mm_header) == GRUB_MM_ALIGN);
+  COMPILE_TIME_ASSERT (sizeof (struct grub_mm_region) == GRUB_MM_ALIGN);
+
   /* When everything is allocated side effect is that *first will have alloc
      magic marked, meaning that there is no room in this region.  */
-  if ((*first)->magic == GRUB_MM_ALLOC_MAGIC)
+  if (!*first)
     return 0;
 
   /* Try to search free slot for allocation in this memory region.  */
@@ -160,7 +166,8 @@
     {
       grub_off_t extra;
 
-      extra = ((grub_addr_t) (p + 1) >> GRUB_MM_ALIGN_LOG2) % align;
+      extra = ((grub_addr_t) (p + GRUB_MM_CELLS_METADATA)
+	       >> GRUB_MM_ALIGN_LOG2) & (align - 1);
       if (extra)
 	extra = align - extra;
 
@@ -260,13 +267,18 @@
 
 	  p->magic = GRUB_MM_ALLOC_MAGIC;
 	  p->size = n;
+	  p->next = 0;
+	  p->prev = 0;
 
 	  /* Mark find as a start marker for next allocation to fasten it.
 	     This will have side effect of fragmenting memory as small
 	     pieces before this will be un-used.  */
-	  *first = q;
+	  if (q->magic == GRUB_MM_ALLOC_MAGIC)
+	    *first = 0;
+	  else
+	    *first = q;
 
-	  return p + 1;
+	  return p + GRUB_MM_CELLS_METADATA;
 	}
 
       /* Search was completed without result.  */
@@ -279,10 +291,11 @@
 
 /* Allocate SIZE bytes with the alignment ALIGN and return the pointer.  */
 void *
-grub_memalign (grub_size_t align, grub_size_t size)
+grub_memalign_notrack (grub_size_t align, grub_size_t size)
 {
   grub_mm_region_t r;
-  grub_size_t n = ((size + GRUB_MM_ALIGN - 1) >> GRUB_MM_ALIGN_LOG2) + 1;
+  grub_size_t n = ((size + GRUB_MM_ALIGN - 1) >> GRUB_MM_ALIGN_LOG2)
+    + GRUB_MM_CELLS_METADATA;
   int count = 0;
 
   if (!grub_mm_base)
@@ -331,18 +344,18 @@
 
 /* Allocate SIZE bytes and return the pointer.  */
 void *
-grub_malloc (grub_size_t size)
+grub_malloc_notrack (grub_size_t size)
 {
-  return grub_memalign (0, size);
+  return grub_memalign_notrack (0, size);
 }
 
 /* Allocate SIZE bytes, clear them and return the pointer.  */
 void *
-grub_zalloc (grub_size_t size)
+grub_zalloc_notrack (grub_size_t size)
 {
   void *ret;
 
-  ret = grub_memalign (0, size);
+  ret = grub_memalign_notrack (0, size);
   if (ret)
     grub_memset (ret, 0, size);
 
@@ -361,7 +374,7 @@
 
   get_header_from_pointer (ptr, &p, &r);
 
-  if (r->first->magic == GRUB_MM_ALLOC_MAGIC)
+  if (!r->first)
     {
       p->magic = GRUB_MM_FREE_MAGIC;
       r->first = p->next = p;
@@ -418,7 +431,7 @@
 /* Reallocate SIZE bytes and return the pointer. The contents will be
    the same as that of PTR.  */
 void *
-grub_realloc (void *ptr, grub_size_t size)
+grub_realloc_notrack (void *ptr, grub_size_t size)
 {
   grub_mm_header_t p;
   grub_mm_region_t r;
@@ -426,7 +439,7 @@
   grub_size_t n;
 
   if (! ptr)
-    return grub_malloc (size);
+    return grub_malloc_notrack (size);
 
   if (! size)
     {
@@ -435,13 +448,14 @@
     }
 
   /* FIXME: Not optimal.  */
-  n = ((size + GRUB_MM_ALIGN - 1) >> GRUB_MM_ALIGN_LOG2) + 1;
+  n = ((size + GRUB_MM_ALIGN - 1) >> GRUB_MM_ALIGN_LOG2)
+    + GRUB_MM_CELLS_METADATA;
   get_header_from_pointer (ptr, &p, &r);
 
   if (p->size >= n)
     return ptr;
 
-  q = grub_malloc (size);
+  q = grub_malloc_notrack (size);
   if (! q)
     return q;
 
@@ -450,7 +464,7 @@
   return q;
 }
 
-#ifdef MM_DEBUG
+#if MM_DEBUG
 int grub_mm_debug = 0;
 
 void
@@ -510,6 +524,50 @@
   grub_printf ("\n");
 }
 
+grub_mm_header_t grub_mm_allocated;
+
+static void
+save_ref (void *ptr, int line, const char *file)
+{
+  grub_mm_header_t head;
+  struct grub_mm_alloc_info *info;
+
+  COMPILE_TIME_ASSERT ((sizeof (struct grub_mm_alloc_info)
+			& (GRUB_MM_ALIGN - 1)) == 0);
+
+  if (!ptr)
+    return;
+
+  head = ptr;
+  head -= GRUB_MM_CELLS_METADATA;
+  info = (struct grub_mm_alloc_info *) (head + 1);
+  head->next = grub_mm_allocated;
+  if (head->next)
+    head->next->prev = head;
+  head->prev = 0;
+  info->lineno = line;
+  grub_strncpy (info->file, file, sizeof (info->file));
+  grub_mm_allocated = head;
+}
+
+static void
+delete_ref (void *ptr)
+{
+  grub_mm_header_t head;
+
+  if (!ptr)
+    return;
+
+  head = ptr;
+  head -= GRUB_MM_CELLS_METADATA;
+  if (head->next)
+    head->next->prev = head->prev;
+  if (head->prev)
+    head->prev->next = head->next;
+  else if (grub_mm_allocated == head)
+    grub_mm_allocated = head->next;
+}
+
 void *
 grub_debug_malloc (const char *file, int line, grub_size_t size)
 {
@@ -517,7 +575,8 @@
 
   if (grub_mm_debug)
     grub_printf ("%s:%d: malloc (0x%" PRIxGRUB_SIZE ") = ", file, line, size);
-  ptr = grub_malloc (size);
+  ptr = grub_malloc_notrack (size);
+  save_ref (ptr, line, file);
   if (grub_mm_debug)
     grub_printf ("%p\n", ptr);
   return ptr;
@@ -530,7 +589,8 @@
 
   if (grub_mm_debug)
     grub_printf ("%s:%d: zalloc (0x%" PRIxGRUB_SIZE ") = ", file, line, size);
-  ptr = grub_zalloc (size);
+  ptr = grub_zalloc_notrack (size);
+  save_ref (ptr, line, file);
   if (grub_mm_debug)
     grub_printf ("%p\n", ptr);
   return ptr;
@@ -539,6 +599,7 @@
 void
 grub_debug_free (const char *file, int line, void *ptr)
 {
+  delete_ref (ptr);
   if (grub_mm_debug)
     grub_printf ("%s:%d: free (%p)\n", file, line, ptr);
   grub_free (ptr);
@@ -547,9 +608,11 @@
 void *
 grub_debug_realloc (const char *file, int line, void *ptr, grub_size_t size)
 {
+  delete_ref (ptr);
   if (grub_mm_debug)
     grub_printf ("%s:%d: realloc (%p, 0x%" PRIxGRUB_SIZE ") = ", file, line, ptr, size);
-  ptr = grub_realloc (ptr, size);
+  ptr = grub_realloc_notrack (ptr, size);
+  save_ref (ptr, line, file);
   if (grub_mm_debug)
     grub_printf ("%p\n", ptr);
   return ptr;
@@ -564,7 +627,8 @@
   if (grub_mm_debug)
     grub_printf ("%s:%d: memalign (0x%" PRIxGRUB_SIZE  ", 0x%" PRIxGRUB_SIZE  
 		 ") = ", file, line, align, size);
-  ptr = grub_memalign (align, size);
+  ptr = grub_memalign_notrack (align, size);
+  save_ref (ptr, line, file);
   if (grub_mm_debug)
     grub_printf ("%p\n", ptr);
   return ptr;

=== modified file 'grub-core/kern/parser.c'
--- grub-core/kern/parser.c	2011-11-11 19:34:37 +0000
+++ grub-core/kern/parser.c	2012-05-01 11:07:30 +0000
@@ -204,12 +204,12 @@
     }
 
   /* Reserve memory for the return values.  */
-  args = grub_malloc (bp - buffer);
+  args = grub_malloc_notrack (bp - buffer);
   if (!args)
     return grub_errno;
   grub_memcpy (args, buffer, bp - buffer);
 
-  *argv = grub_malloc (sizeof (char *) * (*argc + 1));
+  *argv = grub_malloc_notrack (sizeof (char *) * (*argc + 1));
   if (!*argv)
     {
       grub_free (args);

=== modified file 'grub-core/lib/relocator.c'
--- grub-core/lib/relocator.c	2012-04-13 14:55:20 +0000
+++ grub-core/lib/relocator.c	2012-05-01 09:34:38 +0000
@@ -358,7 +358,9 @@
 	    r2->first = r1->first;
 	    hl->next = r2->first;
 	    *rp = (*rp)->next;
-	    grub_free (g + 1);
+	    g->next = 0;
+	    g->prev = 0;
+	    grub_free (g + GRUB_MM_CELLS_METADATA);
 	  }
 	break;
       }
@@ -368,10 +370,11 @@
 							    GRUB_MM_ALIGN);
 	h->size
 	  = ((subchu->start + subchu->size + GRUB_MM_ALIGN - 1) / GRUB_MM_ALIGN)
-	  - (subchu->start / GRUB_MM_ALIGN) - 1;
-	h->next = h;
+	  - (subchu->start / GRUB_MM_ALIGN);
+	h->next = 0;
+	h->prev = 0;
 	h->magic = GRUB_MM_ALLOC_MAGIC;
-	grub_free (h + 1);
+	grub_free (h + GRUB_MM_CELLS_METADATA);
 	break;
       }
 #if GRUB_RELOCATOR_HAVE_FIRMWARE_REQUESTS
@@ -428,6 +431,9 @@
   unsigned j, N = 0;
   grub_addr_t target = 0;
 
+  COMPILE_TIME_ASSERT (sizeof (struct grub_mm_header) == GRUB_MM_ALIGN);
+  COMPILE_TIME_ASSERT (sizeof (struct grub_mm_region) == GRUB_MM_ALIGN);
+
   grub_dprintf ("relocator",
 		"trying to allocate in 0x%lx-0x%lx aligned 0x%lx size 0x%lx\n",
 		(unsigned long) start, (unsigned long) end,

=== modified file 'grub-core/normal/autofs.c'
--- grub-core/normal/autofs.c	2012-02-04 20:47:29 +0000
+++ grub-core/normal/autofs.c	2012-05-01 11:04:40 +0000
@@ -104,13 +104,20 @@
 
 		  /* If the line is empty, skip it.  */
 		  if (p >= q)
-		    continue;
+		    {
+		      grub_free (buf);
+		      continue;
+		    }
 
-		  fs_mod = grub_malloc (sizeof (*fs_mod));
+		  fs_mod = grub_malloc_notrack (sizeof (*fs_mod));
 		  if (! fs_mod)
-		    continue;
+		    {
+		      grub_free (buf);
+		      continue;
+		    }
 
-		  fs_mod->name = grub_strdup (p);
+		  fs_mod->name = grub_strdup_notrack (p);
+		  grub_free (buf);
 		  if (! fs_mod->name)
 		    {
 		      grub_free (fs_mod);

=== modified file 'grub-core/normal/charset.c'
--- grub-core/normal/charset.c	2012-04-07 18:11:04 +0000
+++ grub-core/normal/charset.c	2012-05-01 11:44:22 +0000
@@ -160,19 +160,37 @@
 
 /* Convert UCS-4 to UTF-8.  */
 char *
-grub_ucs4_to_utf8_alloc (const grub_uint32_t *src, grub_size_t size)
-{
-  grub_uint8_t *ret;
-  grub_size_t cnt = grub_get_num_of_utf8_bytes (src, size) + 1;
-
-  ret = grub_malloc (cnt);
-  if (!ret)
-    return 0;
-
-  grub_ucs4_to_utf8 (src, size, ret, cnt);
-
-  return (char *) ret;
-}
+grub_ucs4_to_utf8_alloc_notrack (const grub_uint32_t *src, grub_size_t size)
+{
+  grub_uint8_t *ret;
+  grub_size_t cnt = grub_get_num_of_utf8_bytes (src, size) + 1;
+
+  ret = grub_malloc_notrack (cnt);
+  if (!ret)
+    return 0;
+
+  grub_ucs4_to_utf8 (src, size, ret, cnt);
+
+  return (char *) ret;
+}
+
+#if MM_DEBUG
+char *
+grub_ucs4_to_utf8_alloc_debug (const char *file, int line,
+			       const grub_uint32_t *src, grub_size_t size)
+{
+  grub_uint8_t *ret;
+  grub_size_t cnt = grub_get_num_of_utf8_bytes (src, size) + 1;
+
+  ret = grub_debug_malloc (file, line, cnt);
+  if (!ret)
+    return 0;
+
+  grub_ucs4_to_utf8 (src, size, ret, cnt);
+
+  return (char *) ret;
+}
+#endif
 
 int
 grub_is_valid_utf8 (const grub_uint8_t *src, grub_size_t srcsize)
@@ -269,7 +287,7 @@
   unsigned i;
   struct grub_unicode_compact_range *cur;
 
-  join_types = grub_zalloc (GRUB_UNICODE_MAX_CACHED_CHAR);
+  join_types = grub_zalloc_notrack (GRUB_UNICODE_MAX_CACHED_CHAR);
   if (!join_types)
     {
       grub_errno = GRUB_ERR_NONE;
@@ -289,7 +307,7 @@
   unsigned i;
   struct grub_unicode_compact_range *cur;
 
-  bidi_types = grub_zalloc (GRUB_UNICODE_MAX_CACHED_CHAR);
+  bidi_types = grub_zalloc_notrack (GRUB_UNICODE_MAX_CACHED_CHAR);
   if (!bidi_types)
     {
       grub_errno = GRUB_ERR_NONE;
@@ -367,7 +385,7 @@
   if (!comb_types)
     {
       unsigned i;
-      comb_types = grub_zalloc (GRUB_UNICODE_MAX_CACHED_CHAR);
+      comb_types = grub_zalloc_notrack (GRUB_UNICODE_MAX_CACHED_CHAR);
       if (comb_types)
 	for (cur = grub_unicode_compact; cur->len; cur++)
 	  for (i = cur->start; i < cur->start + (unsigned) cur->len

=== modified file 'grub-core/normal/cmdline.c'
--- grub-core/normal/cmdline.c	2012-04-01 19:25:17 +0000
+++ grub-core/normal/cmdline.c	2012-05-01 11:21:01 +0000
@@ -41,7 +41,7 @@
 grub_set_history (int newsize)
 {
   grub_uint32_t **old_hist_lines = hist_lines;
-  hist_lines = grub_malloc (sizeof (grub_uint32_t *) * newsize);
+  hist_lines = grub_malloc_notrack (sizeof (grub_uint32_t *) * newsize);
 
   /* Copy the old lines into the new buffer.  */
   if (old_hist_lines)
@@ -114,7 +114,7 @@
 grub_history_set (int pos, grub_uint32_t *s, grub_size_t len)
 {
   grub_free (hist_lines[pos]);
-  hist_lines[pos] = grub_malloc ((len + 1) * sizeof (grub_uint32_t));
+  hist_lines[pos] = grub_malloc_notrack ((len + 1) * sizeof (grub_uint32_t));
   if (!hist_lines[pos])
     {
       grub_print_error ();
@@ -656,7 +656,7 @@
       grub_history_add (empty, 0);
     }
 
-  ret = grub_ucs4_to_utf8_alloc (buf, llen + 1);
+  ret = grub_ucs4_to_utf8_alloc_notrack (buf, llen + 1);
   grub_free (buf);
   grub_free (cl_terms);
   return ret;

=== modified file 'grub-core/normal/color.c'
--- grub-core/normal/color.c	2011-11-30 15:20:13 +0000
+++ grub-core/normal/color.c	2012-05-01 11:07:03 +0000
@@ -133,7 +133,7 @@
 
   set_colors ();
 
-  return grub_strdup (val);
+  return grub_strdup_notrack (val);
 }
 
 /* Replace default `highlight' colors with the ones specified by user (if any).  */
@@ -146,5 +146,5 @@
 
   set_colors ();
 
-  return grub_strdup (val);
+  return grub_strdup_notrack (val);
 }

=== modified file 'grub-core/normal/crypto.c'
--- grub-core/normal/crypto.c	2012-03-11 13:43:18 +0000
+++ grub-core/normal/crypto.c	2012-05-01 10:55:20 +0000
@@ -128,14 +128,14 @@
       while (*p == ' ' || *p == '\t')
 	p++;
 
-      cur = grub_malloc (sizeof (*cur));
+      cur = grub_malloc_notrack (sizeof (*cur));
       if (!cur)
 	{
 	  grub_errno = GRUB_ERR_NONE;
 	  continue;
 	}
       
-      cur->name = grub_strdup (name);
+      cur->name = grub_strdup_notrack (name);
       if (! name)
 	{
 	  grub_errno = GRUB_ERR_NONE;
@@ -143,7 +143,7 @@
 	  continue;
 	}
 	
-      cur->modname = grub_strdup (p);
+      cur->modname = grub_strdup_notrack (p);
       if (! cur->modname)
 	{
 	  grub_errno = GRUB_ERR_NONE;

=== modified file 'grub-core/normal/dyncmd.c'
--- grub-core/normal/dyncmd.c	2012-03-11 13:43:18 +0000
+++ grub-core/normal/dyncmd.c	2012-05-01 10:56:14 +0000
@@ -168,11 +168,11 @@
 		  if (grub_dl_get (p))
 		    continue;
 
-		  name = grub_strdup (name);
+		  name = grub_strdup_notrack (name);
 		  if (! name)
 		    continue;
 
-		  modname = grub_strdup (p);
+		  modname = grub_strdup_notrack (p);
 		  if (! modname)
 		    {
 		      grub_free (name);

=== modified file 'grub-core/normal/main.c'
--- grub-core/normal/main.c	2012-03-11 13:43:18 +0000
+++ grub-core/normal/main.c	2012-05-01 11:13:07 +0000
@@ -41,8 +41,13 @@
 int grub_normal_exit_level = 0;
 
 /* Read a line from the file FILE.  */
+#if MM_DEBUG
+char *
+grub_file_debug_getline (const char *cfile, int line, grub_file_t file)
+#else
 char *
 grub_file_getline (grub_file_t file)
+#endif
 {
   char c;
   grub_size_t pos = 0;
@@ -51,7 +56,11 @@
   grub_size_t max_len = 64;
 
   /* Initially locate some space.  */
-  cmdline = grub_malloc (max_len);
+#if MM_DEBUG
+  cmdline = grub_debug_malloc (cfile, line, max_len);
+#else
+  cmdline = grub_debug_malloc (max_len);
+#endif
   if (! cmdline)
     return 0;
 
@@ -69,7 +78,11 @@
 	{
 	  char *old_cmdline = cmdline;
 	  max_len = max_len * 2;
+#if MM_DEBUG
+	  cmdline = grub_debug_realloc (cfile, line, cmdline, max_len);
+#else
 	  cmdline = grub_realloc (cmdline, max_len);
+#endif
 	  if (! cmdline)
 	    {
 	      grub_free (old_cmdline);
@@ -164,7 +177,7 @@
   newmenu = grub_env_get_menu ();
   if (! newmenu)
     {
-      newmenu = grub_zalloc (sizeof (*newmenu));
+      newmenu = grub_zalloc_notrack (sizeof (*newmenu));
       if (! newmenu)
 	return 0;
 
@@ -314,9 +327,13 @@
       prefix = grub_env_get ("prefix");
       if (prefix)
 	{
-	  config = grub_xasprintf ("%s/grub.cfg", prefix);
+	  char *ptr;
+	  config = grub_malloc_notrack (grub_strlen (prefix)
+					+ sizeof ("/grub.cfg"));
 	  if (! config)
 	    goto quit;
+	  ptr = grub_stpcpy (config, prefix);
+	  grub_memcpy (ptr, "/grub.cfg", sizeof ("/grub.cfg"));
 
 	  grub_enter_normal_mode (config);
 	  grub_free (config);

=== modified file 'grub-core/normal/term.c'
--- grub-core/normal/term.c	2012-03-27 15:07:26 +0000
+++ grub-core/normal/term.c	2012-05-01 10:53:34 +0000
@@ -373,14 +373,14 @@
       while (*p == ' ' || *p == '\t')
 	p++;
 
-      cur = grub_malloc (sizeof (*cur));
+      cur = grub_malloc_notrack (sizeof (*cur));
       if (!cur)
 	{
 	  grub_errno = GRUB_ERR_NONE;
 	  continue;
 	}
       
-      cur->name = grub_strdup (name);
+      cur->name = grub_strdup_notrack (name);
       if (! name)
 	{
 	  grub_errno = GRUB_ERR_NONE;
@@ -388,7 +388,7 @@
 	  continue;
 	}
 	
-      cur->modname = grub_strdup (p);
+      cur->modname = grub_strdup_notrack (p);
       if (! cur->modname)
 	{
 	  grub_errno = GRUB_ERR_NONE;

=== modified file 'grub-core/script/argv.c'
--- grub-core/script/argv.c	2012-03-19 12:29:43 +0000
+++ grub-core/script/argv.c	2012-05-01 11:06:18 +0000
@@ -85,7 +85,8 @@
   if (argv->args && argv->argc && argv->args[argv->argc - 1] == 0)
     return 0;
 
-  p = grub_realloc (p, round_up_exp ((argv->argc + 2) * sizeof (char *)));
+  p = grub_realloc_notrack (p,
+			    round_up_exp ((argv->argc + 2) * sizeof (char *)));
   if (! p)
     return 1;
 
@@ -111,7 +112,7 @@
 
   a = p ? grub_strlen (p) : 0;
 
-  p = grub_realloc (p, round_up_exp ((a + slen + 1) * sizeof (char)));
+  p = grub_realloc_notrack (p, round_up_exp ((a + slen + 1) * sizeof (char)));
   if (! p)
     return 1;
 

=== modified file 'grub-core/script/script.c'
--- grub-core/script/script.c	2010-11-25 13:26:20 +0000
+++ grub-core/script/script.c	2012-05-01 11:05:25 +0000
@@ -42,8 +42,8 @@
 grub_script_malloc (struct grub_parser_param *state, grub_size_t size)
 {
   struct grub_script_mem *mem;
-  mem = (struct grub_script_mem *) grub_malloc (size + sizeof (*mem)
-						- sizeof (char));
+  mem = (struct grub_script_mem *) grub_malloc_notrack (size + sizeof (*mem)
+							- sizeof (char));
   if (!mem)
     return 0;
 
@@ -324,7 +324,7 @@
 {
   struct grub_script *parsed;
 
-  parsed = grub_malloc (sizeof (*parsed));
+  parsed = grub_malloc_notrack (sizeof (*parsed));
   if (! parsed)
     return 0;
 

=== modified file 'grub-core/term/gfxterm.c'
--- grub-core/term/gfxterm.c	2012-03-11 22:36:35 +0000
+++ grub-core/term/gfxterm.c	2012-05-01 11:45:31 +0000
@@ -232,9 +232,9 @@
 
   /* Allocate memory for text buffer.  */
   virtual_screen.text_buffer =
-    (struct grub_colored_char *) grub_malloc (virtual_screen.columns
-                                              * virtual_screen.rows
-                                              * sizeof (*virtual_screen.text_buffer));
+    (struct grub_colored_char *) grub_malloc_notrack (virtual_screen.columns
+						      * virtual_screen.rows
+						      * sizeof (*virtual_screen.text_buffer));
   if (grub_errno != GRUB_ERR_NONE)
     return grub_errno;
 

=== modified file 'grub-core/video/fb/video_fb.c'
--- grub-core/video/fb/video_fb.c	2012-02-08 18:26:01 +0000
+++ grub-core/video/fb/video_fb.c	2012-05-01 11:45:19 +0000
@@ -128,9 +128,9 @@
   if (start + count > framebuffer.palette_size)
     {
       framebuffer.palette_size = start + count;
-      framebuffer.palette = grub_realloc (framebuffer.palette,
-					  sizeof (framebuffer.palette[0])
-					  * framebuffer.palette_size);
+      framebuffer.palette = grub_realloc_notrack (framebuffer.palette,
+						  sizeof (framebuffer.palette[0])
+						  * framebuffer.palette_size);
       if (!framebuffer.palette)
 	{
 	  grub_video_fb_fini ();
@@ -1134,7 +1134,7 @@
                        "invalid argument given");
 
   /* Allocate memory for render target.  */
-  target = grub_malloc (sizeof (struct grub_video_fbrender_target));
+  target = grub_malloc_notrack (sizeof (struct grub_video_fbrender_target));
   if (! target)
     return grub_errno;
 
@@ -1173,7 +1173,7 @@
   /* Calculate size needed for the data.  */
   size = (width * target->mode_info.bytes_per_pixel) * height;
 
-  target->data = grub_malloc (size);
+  target->data = grub_malloc_notrack (size);
   if (! target->data)
     {
       grub_free (target);
@@ -1209,7 +1209,7 @@
 #endif
 
   /* Allocate memory for render target.  */
-  target = grub_malloc (sizeof (struct grub_video_fbrender_target));
+  target = grub_malloc_notrack (sizeof (struct grub_video_fbrender_target));
   if (! target)
     return grub_errno;
 

=== modified file 'include/grub/charset.h'
--- include/grub/charset.h	2012-04-07 18:11:04 +0000
+++ include/grub/charset.h	2012-05-01 11:20:01 +0000
@@ -273,7 +273,18 @@
 }
 
 /* Convert UCS-4 to UTF-8.  */
-char *grub_ucs4_to_utf8_alloc (const grub_uint32_t *src, grub_size_t size);
+#if MM_DEBUG
+char *
+grub_ucs4_to_utf8_alloc_debug (const char *file, int line,
+			       const grub_uint32_t *src, grub_size_t size);
+#define grub_ucs4_to_utf8_alloc(src, size) grub_ucs4_to_utf8_alloc_debug (GRUB_FILE, __LINE__, src, size)
+#else
+#define grub_ucs4_to_utf8_alloc_notrack grub_ucs4_to_utf8_alloc
+#endif
+
+char *grub_ucs4_to_utf8_alloc_notrack (const grub_uint32_t *src,
+				       grub_size_t size);
+
 
 int
 grub_is_valid_utf8 (const grub_uint8_t *src, grub_size_t srcsize);

=== modified file 'include/grub/misc.h'
--- include/grub/misc.h	2012-03-05 00:17:55 +0000
+++ include/grub/misc.h	2012-05-01 10:54:59 +0000
@@ -314,8 +314,17 @@
     }
 }
 
-char *EXPORT_FUNC(grub_strdup) (const char *s) __attribute__ ((warn_unused_result));
+#if MM_DEBUG
+char *EXPORT_FUNC(grub_debug_strdup) (const char *file, int line, const char *s) __attribute__ ((warn_unused_result));
+char *EXPORT_FUNC(grub_debug_strndup) (const char *file, int line, const char *s, grub_size_t n) __attribute__ ((warn_unused_result));
+
+#define grub_strdup(s) grub_debug_strdup (GRUB_FILE, __LINE__, s)
+#define grub_strndup(s, n) grub_debug_strndup (GRUB_FILE, __LINE__, s, n)
+#else
 char *EXPORT_FUNC(grub_strndup) (const char *s, grub_size_t n) __attribute__ ((warn_unused_result));
+#define grub_strdup_notrack grub_strdup
+#endif
+char *EXPORT_FUNC(grub_strdup_notrack) (const char *s) __attribute__ ((warn_unused_result));
 void *EXPORT_FUNC(grub_memset) (void *s, int c, grub_size_t n);
 grub_size_t EXPORT_FUNC(grub_strlen) (const char *s) __attribute__ ((warn_unused_result));
 int EXPORT_FUNC(grub_printf) (const char *fmt, ...) __attribute__ ((format (printf, 1, 2)));
@@ -363,9 +372,17 @@
      __attribute__ ((format (printf, 3, 4)));
 int EXPORT_FUNC(grub_vsnprintf) (char *str, grub_size_t n, const char *fmt,
 				 va_list args);
+#if MM_DEBUG
+char *EXPORT_FUNC(grub_debug_xasprintf) (const char *file, int line, const char *fmt, ...)
+     __attribute__ ((format (printf, 3, 4))) __attribute__ ((warn_unused_result));
+char *EXPORT_FUNC(grub_debug_xvasprintf) (const char *file, int line, const char *fmt, va_list args) __attribute__ ((warn_unused_result));
+#define grub_xasprintf(fmt, args...) grub_debug_xasprintf (GRUB_FILE, __LINE__, fmt, ## args)
+#define grub_xvasprintf(fmt, args) grub_debug_xvasprintf (GRUB_FILE, __LINE__, fmt, args)
+#else
 char *EXPORT_FUNC(grub_xasprintf) (const char *fmt, ...)
      __attribute__ ((format (printf, 1, 2))) __attribute__ ((warn_unused_result));
 char *EXPORT_FUNC(grub_xvasprintf) (const char *fmt, va_list args) __attribute__ ((warn_unused_result));
+#endif
 void EXPORT_FUNC(grub_exit) (void) __attribute__ ((noreturn));
 void EXPORT_FUNC(grub_abort) (void) __attribute__ ((noreturn));
 grub_uint64_t EXPORT_FUNC(grub_divmod64) (grub_uint64_t n,

=== modified file 'include/grub/mm.h'
--- include/grub/mm.h	2011-11-13 11:48:39 +0000
+++ include/grub/mm.h	2012-05-01 11:59:32 +0000
@@ -28,18 +28,25 @@
 # define NULL	((void *) 0)
 #endif
 
+#if !MM_DEBUG
+#define grub_malloc_notrack grub_malloc
+#define grub_realloc_notrack grub_realloc
+#define grub_zalloc_notrack grub_zalloc
+#define grub_memalign_notrack grub_memealign
+#endif
+
 void grub_mm_init_region (void *addr, grub_size_t size);
-void *EXPORT_FUNC(grub_malloc) (grub_size_t size);
-void *EXPORT_FUNC(grub_zalloc) (grub_size_t size);
+void *EXPORT_FUNC(grub_malloc_notrack) (grub_size_t size);
+void *EXPORT_FUNC(grub_zalloc_notrack) (grub_size_t size);
 void EXPORT_FUNC(grub_free) (void *ptr);
-void *EXPORT_FUNC(grub_realloc) (void *ptr, grub_size_t size);
-void *EXPORT_FUNC(grub_memalign) (grub_size_t align, grub_size_t size);
+void *EXPORT_FUNC(grub_realloc_notrack) (void *ptr, grub_size_t size);
+void *EXPORT_FUNC(grub_memalign_notrack) (grub_size_t align, grub_size_t size);
 
 void grub_mm_check_real (char *file, int line);
 #define grub_mm_check() grub_mm_check_real (GRUB_FILE, __LINE__);
 
 /* For debugging.  */
-#if defined(MM_DEBUG) && !defined(GRUB_UTIL) && !defined (GRUB_MACHINE_EMU)
+#if MM_DEBUG
 /* Set this variable to 1 when you want to trace all memory function calls.  */
 extern int EXPORT_VAR(grub_mm_debug);
 

=== modified file 'include/grub/mm_private.h'
--- include/grub/mm_private.h	2010-08-28 12:52:25 +0000
+++ include/grub/mm_private.h	2012-05-01 11:09:26 +0000
@@ -30,13 +30,7 @@
   struct grub_mm_header *next;
   grub_size_t size;
   grub_size_t magic;
-#if GRUB_CPU_SIZEOF_VOID_P == 4
-  char padding[4];
-#elif GRUB_CPU_SIZEOF_VOID_P == 8
-  char padding[8];
-#else
-# error "unknown word size"
-#endif
+  struct grub_mm_header *prev;
 }
 *grub_mm_header_t;
 
@@ -48,6 +42,17 @@
 
 #define GRUB_MM_ALIGN	(1 << GRUB_MM_ALIGN_LOG2)
 
+#if MM_DEBUG
+struct grub_mm_alloc_info
+{
+  int lineno;
+  char file[64 - sizeof (int)];
+};
+static const grub_size_t GRUB_MM_CELLS_METADATA = ((sizeof (struct grub_mm_alloc_info) >> GRUB_MM_ALIGN_LOG2) + 1);
+#else
+#define GRUB_MM_CELLS_METADATA 1
+#endif
+
 typedef struct grub_mm_region
 {
   struct grub_mm_header *first;
@@ -61,4 +66,8 @@
 extern grub_mm_region_t EXPORT_VAR (grub_mm_base);
 #endif
 
+#if MM_DEBUG
+extern grub_mm_header_t EXPORT_VAR(grub_mm_allocated);
+#endif
+
 #endif

=== modified file 'include/grub/normal.h'
--- include/grub/normal.h	2012-03-10 22:25:34 +0000
+++ include/grub/normal.h	2012-05-01 10:15:01 +0000
@@ -51,7 +51,14 @@
 void grub_menu_init_page (int nested, int edit, int *num_entries,
 			  struct grub_term_output *term);
 void grub_normal_init_page (struct grub_term_output *term);
+#if MM_DEBUG
+char *grub_file_debug_getline (const char *call_file,
+			       int line, grub_file_t file);
+#define grub_file_getline(file)	\
+  grub_file_debug_getline (GRUB_FILE, __LINE__, file)
+#else
 char *grub_file_getline (grub_file_t file);
+#endif
 void grub_cmdline_run (int nested);
 
 /* Defined in `cmdline.c'.  */

=== modified file 'include/grub/unicode.h'
--- include/grub/unicode.h	2012-03-27 15:07:26 +0000
+++ include/grub/unicode.h	2012-05-01 11:44:04 +0000
@@ -237,13 +237,14 @@
 static inline struct grub_unicode_glyph *
 grub_unicode_glyph_dup (const struct grub_unicode_glyph *in)
 {
-  struct grub_unicode_glyph *out = grub_malloc (sizeof (*out));
+  struct grub_unicode_glyph *out = grub_malloc_notrack (sizeof (*out));
   if (!out)
     return NULL;
   grub_memcpy (out, in, sizeof (*in));
   if (in->combining)
     {
-      out->combining = grub_malloc (in->ncomb * sizeof (out->combining[0]));
+      out->combining = grub_malloc_notrack (in->ncomb
+					    * sizeof (out->combining[0]));
       if (!out->combining)
 	{
 	  grub_free (out);
@@ -259,7 +260,7 @@
 grub_unicode_glyph_from_code (grub_uint32_t code)
 {
   struct grub_unicode_glyph *ret;
-  ret = grub_zalloc (sizeof (*ret));
+  ret = grub_zalloc_notrack (sizeof (*ret));
   if (!ret)
     return NULL;
 


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 294 bytes --]

Re: RFC: enhanced memory protection support

[Seth Goldberg]

   I like this idea very much.

  --S

Quoting Bean, who wrote the following on Mon, 30 Apr 2012:

> Hi,
>
> While testing network function in efi mode, I've found several memory
> leak related to fragmentation, but there is still some memory problem
> that's very tricky to locate. For example, you can run testspeed on a
> large file several times in a row and it could show the memory error.
> Since network condition are very difficult to reproduce, I have to
> look at the source of this issue, memory allocation. Currently it has
> mm_debug option that could print out file and line number of each
> memory call, but it's quite useless since we can't find the relevant
> informaton with full screen of prints.
>
> Here are some of my ideas for enhanced memory protection support:
>
> 1, when we allocate memory, we append some information at the end of
> the buffer, which include filename, lineno of caller, and tag to
> indicate what is used for and some padding to detect memory overwrite
> problem.
>
> 2. add a command to print the current memory list with extended
> information, this can be used to find memory leaks.
>
> 3. it's also a good idea to run the memory check in automated test to
> locate potential issue.
>
> -- 
> Best wishes
> Bean
>
> _______________________________________________
> Grub-devel mailing list
> Grub-devel@gnu.org
> https://lists.gnu.org/mailman/listinfo/grub-devel
>