[PATCH] Use stddef.h in Mini-OS to define size_t

[PATCH] Use stddef.h in Mini-OS to define size_t

[John D. Ramsdell]

Please patch Mini-OS so that it uses stddef.h to define size_t and
NULL. This problem fixes errors that occur when linking Mini-OS with
ANSI standard code that uses stddef.h.

John

diff -ur oxen-3.0-testing/extras/mini-os/include/lib.h nxen-3.0-testing/extras/mini-os/include/lib.h
--- oxen-3.0-testing/extras/mini-os/include/lib.h	2006-04-14 22:21:55.000000000 -0400
+++ nxen-3.0-testing/extras/mini-os/include/lib.h	2006-04-28 13:37:44.000000000 -0400
@@ -56,6 +56,7 @@
 #define _LIB_H_
 
 #include <stdarg.h>
+#include <stddef.h>
 #include <console.h>
 
 /* printing */
diff -ur oxen-3.0-testing/extras/mini-os/include/os.h nxen-3.0-testing/extras/mini-os/include/os.h
--- oxen-3.0-testing/extras/mini-os/include/os.h	2006-04-14 22:21:55.000000000 -0400
+++ nxen-3.0-testing/extras/mini-os/include/os.h	2006-04-28 13:39:13.000000000 -0400
@@ -7,9 +7,6 @@
 #ifndef _OS_H_
 #define _OS_H_
 
-#define NULL 0
-
-
 #if __GNUC__ == 2 && __GNUC_MINOR__ < 96
 #define __builtin_expect(x, expected_value) (x)
 #endif
diff -ur oxen-3.0-testing/extras/mini-os/include/types.h nxen-3.0-testing/extras/mini-os/include/types.h
--- oxen-3.0-testing/extras/mini-os/include/types.h	2006-04-14 22:21:55.000000000 -0400
+++ nxen-3.0-testing/extras/mini-os/include/types.h	2006-04-28 13:39:17.000000000 -0400
@@ -34,8 +34,6 @@
 typedef unsigned long       u64;
 #endif
 
-typedef unsigned int        size_t;
-
 /* FreeBSD compat types */
 typedef unsigned char       u_char;
 typedef unsigned int        u_int;

Re: [PATCH] Use stddef.h in Mini-OS to define size_t

[Keir Fraser]

On 1 May 2006, at 16:10, John D. Ramsdell wrote:

> Please patch Mini-OS so that it uses stddef.h to define size_t and
> NULL. This problem fixes errors that occur when linking Mini-OS with
> ANSI standard code that uses stddef.h.

Linux goes to considerable pain to ensure that stdarg.h is the only 
system header that it needs to include. stddef.h looks like it ought to 
be safe in a kernel environment, but would it not be just as easy to 
modify the files that you are adding to the build? They're not being 
built in anything like a standard C environment: I would have thought 
that stddef.h would be the tip of the iceberg.

  -- Keir

Re: [PATCH] Use stddef.h in Mini-OS to define size_t

[John D. Ramsdell]

Keir Fraser <Keir.Fraser@cl.cam.ac.uk> writes:

> Linux goes to considerable pain to ensure that stdarg.h is the only
> system header that it needs to include. stddef.h looks like it ought
> to be safe in a kernel environment, but would it not be just as easy
> to modify the files that you are adding to the build?

I'm trying to insert a Lua interpreter into the Mini-OS kernel.
Before I attempt to do something useful with the interpreter, I
decided to do a quick test to see if there are any unforeseen issues
with incorporating Lua into the kernel.  My quick check involved
modifying the console program so that it uses Lua to evaluate console
input, and then prints the results on the terminal.

Keir, it would not be easy to modify the files that I'm adding.
Instead, I modify the Lua interpreter so that it no longer performs
floating point operations, but is otherwise a standard ANSI C program.
The Lua interpreter and Mini-OS are compiled and linked using a
cross-compiler combined with newlib for C library support.  Making
extensive modifications to both newlib and the Lua interpreter is
unappealing, to say the least.

I have enclosed the patch for the console and the Makefile.  My quick
test has uncovered an unforeseen problem.  The interpreter reads,
evaluates, and prints a few times, and then a reference to a strange
address causes a page fault.  I've just begun looking into the
problem, however, in the past, I've noted that typically kernels
allocate a small amount of space to its stack, and if it's too small,
something like this could happen.  I suppose a bug in xmalloc would do
it too, but I just guessing at this stage.  Off to studying x86_32.S.
Unfortunately, my assemble language programming skills were mostly
developed and used on a PDP-11.  They may be a little rusty.

John

Xen Minimal OS!
start_info:   c0035000
  nr_pages:   8192  shared_inf: 00bbd000
  pt_base:    c0038000  mod_start:  0x0
  mod_len:    0
  flags:      0x0
  cmd_line:
MM: Init
  _text:        c0000000
  _etext:       c001f500
  _edata:       c0023090
  stack start:  c002aa60
  _end:         c002cec8
  start_pfn:    3d
  max_pfn:      2000
Mapping memory range 0xc0400000 - 0xc2000000
MM: Initialise page allocator for c0044000(44000)-c2000000(2000000)
MM: done
Initialising timer interface
Initialising console ... Initializing Lua ... done.
Initialising scheduler, idle_thread 00000000
Thread "Idle": pointer: 0xc0045b40, stack: 0xc0046000
Thread "init_xs": pointer: 0xc0045b68, stack: 0xc0048000
Thread "xenstore": pointer: 0xc0045b90, stack: 0xc004a000
print (3)
Lua     3
print (20
Lua     Page fault at linear address 0000e021, regs c0047734, code 2

------------------------------------ PATCH ---------------------------------

diff -ur oxen-3.0-testing/extras/mini-os/console/console.c xen-3.0-testing/extras/mini-os/console/console.c
--- oxen-3.0-testing/extras/mini-os/console/console.c	2006-04-14 22:21:55.000000000 -0400
+++ xen-3.0-testing/extras/mini-os/console/console.c	2006-04-28 15:56:50.000000000 -0400
@@ -44,6 +44,98 @@
 #include <xenbus.h>
 #include <xen/io/console.h>
 
+#if defined LUA_CONSOLE
+
+#include <lua.h>
+#include <lauxlib.h>
+#include <lualib.h>
+#include <xmalloc.h>
+
+static void *luaxen_alloc(void *ud, void *ptr, size_t osize, size_t nsize)
+{
+  if (nsize == 0) {
+    free(ptr);
+    return NULL;
+  }
+  else if (ptr == NULL) 
+    return malloc(nsize);
+  else if (osize >= nsize)
+    return ptr;
+  else {
+    void *p = malloc(nsize);
+    if (p == NULL)
+      return p;
+    else
+      return memcpy(p, ptr, osize);
+  }
+}
+
+/* Adapted from min.c in the Lua distribution. */
+
+static int luaxen_print(lua_State *L)
+{
+  int n = lua_gettop(L);
+  int i;
+  for (i = 1; i <= n; i++) {
+    if (i > 1) printk("\t");
+    if (lua_isstring(L, i))
+      printk("%s", lua_tostring(L, i));
+    else if (lua_isnil(L, i) == 2)
+      printk("%s", "nil");
+    else if (lua_isboolean(L, i))
+      printk("%s", lua_toboolean(L, i) ? "true" : "false");
+    else
+      printk("%s:%p", luaL_typename(L, i), lua_topointer(L, i));
+  }
+  return 0;
+}
+
+static lua_State *L;
+#define LUAXEN_BUFSIZ 256
+static char luaxen_buf[LUAXEN_BUFSIZ];
+static size_t luaxen_len;
+
+static void luaxen_init(void) 
+{
+  L = lua_newstate(luaxen_alloc, NULL);
+  luaxen_len = 0;
+  lua_pushcfunction(L, luaxen_print);
+  lua_setglobal(L, "print");
+}
+  
+static void luaxen_read_eval_print(const char *data, size_t len) 
+{
+  size_t i;
+  size_t n = len;
+  if (n + luaxen_len > LUAXEN_BUFSIZ)
+    n = LUAXEN_BUFSIZ - luaxen_len;
+  for (i = 0; i < n; i++)
+    luaxen_buf[luaxen_len++] = data[i];
+  if (data[len - 1] != '\r')
+    return;
+  printk("\nLua\t");
+  if (luaL_loadbuffer(L, luaxen_buf, luaxen_len, "console")
+      || lua_pcall(L, 0, 0, 0)) {
+    printk("\nLua error\t%s", lua_tostring(L, -1));
+    lua_pop(L, 1);
+  }
+  printk("\n");
+  luaxen_len = 0;
+}
+
+void _exit(int status) 
+{
+  do_exit();
+}
+
+void *sbrk(int inc)
+{
+  printk("Function sbrk called in Mini-OS kernel\n");
+  do_exit();
+  return NULL;
+}
+
+#endif
 
 /* Low level functions defined in xencons_ring.c */
 extern int xencons_ring_init(void);
@@ -64,8 +156,12 @@
         buf[len] = '\0';
         printk("%s", buf);
         
+#if defined LUA_CONSOLE
+        luaxen_read_eval_print(buf, len);
+#else
         if(buf[len-1] == '\r')
             printk("\nNo console input handler.\n");
+#endif
     }
 }
 
@@ -144,6 +240,10 @@
     printk("Initialising console ... ");
     xencons_ring_init();    
     console_initialised = 1;
+#if defined LUA_CONSOLE
+    printk("Initializing Lua ... ");
+    luaxen_init();
+#endif
     /* This is also required to notify the daemon */
     printk("done.\n");
 }
diff -ur oxen-3.0-testing/extras/mini-os/Makefile xen-3.0-testing/extras/mini-os/Makefile
--- oxen-3.0-testing/extras/mini-os/Makefile	2006-04-14 22:21:55.000000000 -0400
+++ xen-3.0-testing/extras/mini-os/Makefile	2006-04-28 13:45:14.000000000 -0400
@@ -20,6 +20,10 @@
 LDFLAGS := -m elf_x86_64
 endif
 
+LUA_INCLUDES=$(HOME)/src/lua-5.1-long/src
+LUA_LIBS=$(HOME)/src/lua-5.1-long/src/liblua.a -lc -lnosys
+CFLAGS += -I$(LUA_INCLUDES) -DLUA_CONSOLE -DLUA_ANSI
+
 ifeq ($(debug),y)
 CFLAGS += -g
 else
@@ -33,7 +37,7 @@
 OBJS += $(patsubst %.c,%.o,$(wildcard lib/*.c))
 OBJS += $(patsubst %.c,%.o,$(wildcard xenbus/*.c))
 OBJS += $(patsubst %.c,%.o,$(wildcard console/*.c))
-										   
+
 HDRS := $(wildcard include/*.h)
 HDRS += $(wildcard include/xen/*.h)
 
@@ -43,7 +47,7 @@
 	[ -e include/xen ] || ln -sf ../../../xen/include/public include/xen
 
 $(TARGET): links $(OBJS)
-	$(LD) -N -T minios-$(TARGET_ARCH).lds $(OBJS) -o $@.elf
+	$(LD) -N -T minios-$(TARGET_ARCH).lds $(OBJS) $(LUA_LIBS) -o $@.elf
 	gzip -f -9 -c $@.elf >$@.gz
 
 clean:

Re: [PATCH] Use stddef.h in Mini-OS to define size_t

[Keir Fraser]

On 1 May 2006, at 21:52, John D. Ramsdell wrote:

> Keir, it would not be easy to modify the files that I'm adding.
> Instead, I modify the Lua interpreter so that it no longer performs
> floating point operations, but is otherwise a standard ANSI C program.
> The Lua interpreter and Mini-OS are compiled and linked using a
> cross-compiler combined with newlib for C library support.  Making
> extensive modifications to both newlib and the Lua interpreter is
> unappealing, to say the least.

Don't you get horrible namespace collisions when linking with a C 
library replacement? Not just stddef.h, but what about the definitions 
in minios's include/lib.h?

  -- Keir

Re: [PATCH] Use stddef.h in Mini-OS to define size_t

[John D. Ramsdell]

Keir Fraser <Keir.Fraser@cl.cam.ac.uk> writes:

> Don't you get horrible namespace collisions when linking with a C
> library replacement? Not just stddef.h, but what about the
> definitions in minios's include/lib.h?

I received no compiler warnings other than the reported problem with
size_t and NULL.  The console source file includes lib.h before it
includes Lua headers, so the other definitions must fit with Lua's
definitions.  The functions in lib.c take precedence over the ones in
the C library, so they better be good.  The definition of gettimeofday
also overrides that which is provided by the C library, which is good
because the C library simply provides a stub routine.  By the way, the
Lua allocator in the modified version of console.c does not free
enough.  I did that to look for malloc bugs, but forgot to remove the
change when it made no difference.

John

Re: [PATCH] Use stddef.h in Mini-OS to define size_t

[Keir Fraser]

On 2 May 2006, at 10:41, John D. Ramsdell wrote:

>> Don't you get horrible namespace collisions when linking with a C
>> library replacement? Not just stddef.h, but what about the
>> definitions in minios's include/lib.h?
>
> I received no compiler warnings other than the reported problem with
> size_t and NULL.  The console source file includes lib.h before it
> includes Lua headers, so the other definitions must fit with Lua's
> definitions.  The functions in lib.c take precedence over the ones in
> the C library, so they better be good.  The definition of gettimeofday
> also overrides that which is provided by the C library, which is good
> because the C library simply provides a stub routine.  By the way, the
> Lua allocator in the modified version of console.c does not free
> enough.  I did that to look for malloc bugs, but forgot to remove the
> change when it made no difference.

Okay, well ultimately it's up to Grzegorz whether to take the patch as 
he's currently de facto maintainer of mini-os. However, I don't see any 
obvious problem in using stddef.h from within minios.

  -- Keir

Re: [PATCH] Use stddef.h in Mini-OS to define size_t

[Grzegorz Milos]

> Okay, well ultimately it's up to Grzegorz whether to take the patch as 
> he's currently de facto maintainer of mini-os. However, I don't see any 
> obvious problem in using stddef.h from within minios.
> 
>  -- Keir

As you mention Keir stddef.h looks like it should be safe to use in 
kernel environment. If that helps to compile standard ANSI code using 
John's approach I see no reason not to apply the patch. Ultimately we 
would like to make the job of porting apps to Mini-OS as easy as possible.

Could you apply the patch Keir?

Additionally, as I've already told John in a private email, the page 
fault trap handler seems to be broken (the faulting linear address is 
bogus). We are working with Aravindh to fix that. Afterwards it will be 
much easier to pinpoint to cause of the pagefault.

Cheers
Gregor