Transparent decompression with file system filter

Transparent decompression with file system filter

[Bean]

Currently, grub2 support gziped file with the gzio module. To open a
gziped file,
you have to use a special function grub_gzfile_open, I think it could
be better if
grub_file_open could handle compressed file transparently.

The implementation is actually quite simple. We define a file system
filter interface,
modules can use it to register its filter. Then, as the last step of
opening a file, we
can the filters. If a filter recognized the file format, it will
create a new file object
to represent the file. If none of the filter want to handle this file,
the orginal file
object is used.

We can add a new function grub_file_rawopen, which will always skip
the filters. We can
also use variable to control behavior, for example, if nocomp=1, no
filter is called.

The advantage of this scheme is that it's extensible. For example. if
we want to support
bzip2 format, we just need to add a new decompression filter.

-- 
Bean

Re: Transparent decompression with file system filter

[Marco Gerards]

Bean <bean123ch@gmail.com> writes:

> Currently, grub2 support gziped file with the gzio module. To open a
> gziped file,
> you have to use a special function grub_gzfile_open, I think it could
> be better if
> grub_file_open could handle compressed file transparently.

Are you still interested in working on this?

--
Marco

Re: Transparent decompression with file system filter

[Bean]

Hi,

> Are you still interested in working on this?

Yes, if you think this is an useful feature, I can start working on it.

-- 
Bean

Re: Transparent decompression with file system filter

[Vesa Jääskeläinen]

Bean wrote:
> Hi,
> 
>> Are you still interested in working on this?
> 
> Yes, if you think this is an useful feature, I can start working on it.
> 

I am just wondering is there need to read compressed files without being
compressed?

I like the idea that opening compressed file is in a separate function,
but perhaps there should be some new generic open() function that can
handle both compressed (gz, bz2?, others with modules?) and in case
compression cannot be detected then use traditional file access.

Re: Transparent decompression with file system filter

[Jan C. Kleinsorge]

Vesa Jääskeläinen wrote:
> Bean wrote:
>> Hi,
>>
>>> Are you still interested in working on this?
>> Yes, if you think this is an useful feature, I can start working on it.
>>
> 
> I am just wondering is there need to read compressed files without being
> compressed?
> 
> I like the idea that opening compressed file is in a separate function,
> but perhaps there should be some new generic open() function that can
> handle both compressed (gz, bz2?, others with modules?) and in case
> compression cannot be detected then use traditional file access.
> 

It would be nice if an archive is handled just as a filesystem, like
with a loopback device. That would make the support both optional and
transparent.

> 
> 
> _______________________________________________
> Grub-devel mailing list
> Grub-devel@gnu.org
> http://lists.gnu.org/mailman/listinfo/grub-devel
> 

Re: Transparent decompression with file system filter

[Marco Gerards]

Vesa Jääskeläinen <chaac@nic.fi> writes:

> Bean wrote:
>> Hi,
>> 
>>> Are you still interested in working on this?
>> 
>> Yes, if you think this is an useful feature, I can start working on it.
>> 
>
> I am just wondering is there need to read compressed files without being
> compressed?
>
> I like the idea that opening compressed file is in a separate function,
> but perhaps there should be some new generic open() function that can
> handle both compressed (gz, bz2?, others with modules?) and in case
> compression cannot be detected then use traditional file access.

That sounds like a good idea.

--
Marco

Re: Transparent decompression with file system filter

[Bean]

[-- Attachment #1: Type: text/plain, Size: 498 bytes --]

Hi,

I've just written a patch for this feature. In order to enable gzip
support, you just need to load the gzio module, then grub_file_open
will handle gz file just like they're not compressed. If you don't
want auto decompression, use grub_file_open_raw instead of
grub_file_open. You can also disable auto decompression by setting
nocomp=1.

btw, the current cvs code seems to have problem, i can't access
partition from hd! but i'm able to test this feature using (host) in
grub-emu.

-- 
Bean

[-- Attachment #2: grub2-fshook.diff --]
[-- Type: application/octet-stream, Size: 6479 bytes --]

2007-11-19  Bean  <bean123ch@gmail.com>

	* include/grub/file.h (grub_fshook): New structure.
	(grub_fshook_register): New function.
	(grub_fshook_unregister): New function.
	(grub_open_file_raw): New function.

	* kern/file.c (grub_fshook_list): New variable.
	(grub_fshook_register): New function.
	(grub_fshook_unregister): New function.
	(grub_open_file_raw): The original grub_open_file.
	(grub_open_file): New function.

	* io/gzio.c (grub_fshook_open): New function.
	(grub_fshook_gzio): New variable.
	
	* commands/cat.c (grub_cmd_cat): Replace grub_gzfile_open with grub_file_open.

	* commands/cmp.c (grub_cmd_cmp): The same.

	* commands/hexdump.c (grub_cmd_hexdump): The same.


Index: commands/cat.c
===================================================================
RCS file: /sources/grub/grub2/commands/cat.c,v
retrieving revision 1.6
diff -u -p -r1.6 cat.c
--- commands/cat.c	21 Jul 2007 23:32:19 -0000	1.6
+++ commands/cat.c	19 Nov 2007 07:38:53 -0000
@@ -24,7 +24,6 @@
 #include <grub/disk.h>
 #include <grub/term.h>
 #include <grub/misc.h>
-#include <grub/gzio.h>
 
 static grub_err_t
 grub_cmd_cat (struct grub_arg_list *state __attribute__ ((unused)),
@@ -38,7 +37,7 @@ grub_cmd_cat (struct grub_arg_list *stat
   if (argc != 1)
     return grub_error (GRUB_ERR_BAD_ARGUMENT, "file name required");
 
-  file = grub_gzfile_open (args[0], 1);
+  file = grub_file_open (args[0]);
   if (! file)
     return 0;
   
Index: commands/cmp.c
===================================================================
RCS file: /sources/grub/grub2/commands/cmp.c,v
retrieving revision 1.8
diff -u -p -r1.8 cmp.c
--- commands/cmp.c	21 Jul 2007 23:32:19 -0000	1.8
+++ commands/cmp.c	19 Nov 2007 07:38:54 -0000
@@ -23,7 +23,6 @@
 #include <grub/misc.h>
 #include <grub/file.h>
 #include <grub/mm.h>
-#include <grub/gzio.h>
 
 #define BUFFER_SIZE 512
 
@@ -44,8 +43,8 @@ grub_cmd_cmp (struct grub_arg_list *stat
   grub_printf ("Compare `%s' and `%s':\n", args[0],
 	       args[1]);
 
-  file1 = grub_gzfile_open (args[0], 1);
-  file2 = grub_gzfile_open (args[1], 1);
+  file1 = grub_file_open (args[0]);
+  file2 = grub_file_open (args[1]);
   if (! file1 || ! file2)
     goto cleanup;
 
Index: commands/hexdump.c
===================================================================
RCS file: /sources/grub/grub2/commands/hexdump.c,v
retrieving revision 1.1
diff -u -p -r1.1 hexdump.c
--- commands/hexdump.c	18 Nov 2007 06:41:45 -0000	1.1
+++ commands/hexdump.c	19 Nov 2007 07:38:54 -0000
@@ -23,7 +23,6 @@
 #include <grub/file.h>
 #include <grub/disk.h>
 #include <grub/misc.h>
-#include <grub/gzio.h>
 #include <grub/hexdump.h>
 
 static const struct grub_arg_option options[] = {
@@ -100,7 +99,7 @@ grub_cmd_hexdump (struct grub_arg_list *
 
   if (is_file)
     {
-      file = grub_gzfile_open (args[0], 1);
+      file = grub_file_open (args[0]);
       if (!file)
 	return 0;
 
Index: include/grub/file.h
===================================================================
RCS file: /sources/grub/grub2/include/grub/file.h,v
retrieving revision 1.6
diff -u -p -r1.6 file.h
--- include/grub/file.h	2 Aug 2007 18:40:36 -0000	1.6
+++ include/grub/file.h	19 Nov 2007 07:38:55 -0000
@@ -48,9 +48,20 @@ struct grub_file
 };
 typedef struct grub_file *grub_file_t;
 
+struct grub_fshook
+{
+  grub_file_t (*open_func)(grub_file_t file);
+  struct grub_fshook *next;
+};
+typedef struct grub_fshook *grub_fshook_t;
+
+void EXPORT_FUNC(grub_fshook_register) (grub_fshook_t fshook);
+void EXPORT_FUNC(grub_fshook_unregister) (grub_fshook_t fshook);
+
 /* Get a device name from NAME.  */
 char *EXPORT_FUNC(grub_file_get_device_name) (const char *name);
 
+grub_file_t EXPORT_FUNC(grub_file_open_raw) (const char *name);
 grub_file_t EXPORT_FUNC(grub_file_open) (const char *name);
 grub_ssize_t EXPORT_FUNC(grub_file_read) (grub_file_t file, char *buf,
 					  grub_size_t len);
Index: io/gzio.c
===================================================================
RCS file: /sources/grub/grub2/io/gzio.c,v
retrieving revision 1.5
diff -u -p -r1.5 gzio.c
--- io/gzio.c	21 Jul 2007 23:32:25 -0000	1.5
+++ io/gzio.c	19 Nov 2007 07:39:03 -0000
@@ -36,6 +36,7 @@
 
 #include <grub/err.h>
 #include <grub/types.h>
+#include <grub/dl.h>
 #include <grub/mm.h>
 #include <grub/misc.h>
 #include <grub/fs.h>
@@ -1241,3 +1242,25 @@ static struct grub_fs grub_gzio_fs =
     .label = 0,
     .next = 0
   };
+
+static grub_file_t grub_fshook_open(grub_file_t file)
+{
+  return grub_gzio_open (file, 1);
+}
+
+static struct grub_fshook grub_fshook_gzio =
+  {
+    .open_func = grub_fshook_open,
+    .next = 0
+  };
+
+GRUB_MOD_INIT(gzio)
+{
+  (void) mod;			/* To stop warning. */
+  grub_fshook_register(&grub_fshook_gzio);
+}
+
+GRUB_MOD_FINI(gzio)
+{
+  grub_fshook_unregister(&grub_fshook_gzio);
+}
Index: kern/file.c
===================================================================
RCS file: /sources/grub/grub2/kern/file.c,v
retrieving revision 1.7
diff -u -p -r1.7 file.c
--- kern/file.c	21 Jul 2007 23:32:26 -0000	1.7
+++ kern/file.c	19 Nov 2007 07:39:07 -0000
@@ -23,6 +23,30 @@
 #include <grub/mm.h>
 #include <grub/fs.h>
 #include <grub/device.h>
+#include <grub/env.h>
+
+static grub_fshook_t grub_fshook_list;
+
+void
+grub_fshook_register (grub_fshook_t fshook)
+{
+  fshook->next = grub_fshook_list;
+  grub_fshook_list = fshook;
+}
+
+void
+grub_fshook_unregister (grub_fshook_t fshook)
+{
+  grub_fshook_t *p, q;
+
+  for (p = &grub_fshook_list, q = *p; q; p = &(q->next), q = q->next)
+    if (q == fshook)
+      {
+	*p = q->next;
+	break;
+      }
+}
+
 
 /* Get the device part of the filename NAME. It is enclosed by parentheses.  */
 char *
@@ -52,7 +76,7 @@ grub_file_get_device_name (const char *n
 }
 
 grub_file_t
-grub_file_open (const char *name)
+grub_file_open_raw (const char *name)
 {
   grub_device_t device;
   grub_file_t file = 0;
@@ -110,6 +134,34 @@ grub_file_open (const char *name)
   return 0;
 }
 
+grub_file_t
+grub_file_open (const char *name)
+{
+  grub_file_t file;
+  grub_fshook_t p;
+  char *val;
+
+  file=grub_file_open_raw (name);
+
+  if (! file)
+    return file;
+
+  val= grub_env_get ("nocomp");
+  if ((val) && (val[0]=='1'))
+    return file;
+
+  for (p=grub_fshook_list;p;p=p->next)
+    {
+      grub_file_t new_file;
+
+      new_file = (*p->open_func) (file);
+      if (new_file)
+        return new_file;
+    }
+
+  return file;
+}
+
 grub_ssize_t
 grub_file_read (grub_file_t file, char *buf, grub_size_t len)
 {

Re: Transparent decompression with file system filter

[Bean]

[-- Attachment #1: Type: text/plain, Size: 710 bytes --]

Hi,

On Nov 19, 2007 4:02 PM, Bean <bean123ch@gmail.com> wrote:
> btw, the current cvs code seems to have problem, i can't access
> partition from hd! but i'm able to test this feature using (host) in
> grub-emu.

Ok, i figure it out now, i need to add the biosdisk module to
core.img, then it can access hard disk just as before.

I find more grub_gzfile_open to change in the elf and multiboot
loader, I also change the name of controlling variable from "nocomp"
to "nofshook" as it seems more natural, please check out the new
patch.

I test the auto decompression is various situation, it work out fine,
for example, you can now use loopback on a compressed file:

loopback lo1 (hd0,1)/aa.img.gz

-- 
Bean

[-- Attachment #2: grub2-fshook2.diff --]
[-- Type: application/octet-stream, Size: 9982 bytes --]

2007-11-19  Bean  <bean123ch@gmail.com>

	* include/grub/file.h (grub_fshook): New structure.
	(grub_fshook_register): New function.
	(grub_fshook_unregister): New function.
	(grub_open_file_raw): New function.

	* kern/file.c (grub_fshook_list): New variable.
	(grub_fshook_register): New function.
	(grub_fshook_unregister): New function.
	(grub_open_file_raw): The original grub_open_file.
	(grub_open_file): New function.

	* io/gzio.c (grub_fshook_open): New function.
	(grub_fshook_gzio): New variable.
	
	* commands/cat.c (grub_cmd_cat): Replace grub_gzfile_open with grub_file_open.

	* commands/cmp.c (grub_cmd_cmp): The same.

	* commands/hexdump.c (grub_cmd_hexdump): The same.

	* kern/elf.c (grub_elf_open): The same.

	* loader/multiboot2.c (grub_multiboot2): The same.
	(grub_module2): The same.

	* loader/multiboot_loader.c (grub_rescue_cmd_multiboot_loader): The same.

	* loader/i386/pc/multiboot.c (grub_multiboot): The same.
	(grub_module): The same.


Index: commands/cat.c
===================================================================
RCS file: /sources/grub/grub2/commands/cat.c,v
retrieving revision 1.6
diff -u -p -r1.6 cat.c
--- commands/cat.c	21 Jul 2007 23:32:19 -0000	1.6
+++ commands/cat.c	19 Nov 2007 09:51:12 -0000
@@ -24,7 +24,6 @@
 #include <grub/disk.h>
 #include <grub/term.h>
 #include <grub/misc.h>
-#include <grub/gzio.h>
 
 static grub_err_t
 grub_cmd_cat (struct grub_arg_list *state __attribute__ ((unused)),
@@ -38,7 +37,7 @@ grub_cmd_cat (struct grub_arg_list *stat
   if (argc != 1)
     return grub_error (GRUB_ERR_BAD_ARGUMENT, "file name required");
 
-  file = grub_gzfile_open (args[0], 1);
+  file = grub_file_open (args[0]);
   if (! file)
     return 0;
   
Index: commands/cmp.c
===================================================================
RCS file: /sources/grub/grub2/commands/cmp.c,v
retrieving revision 1.8
diff -u -p -r1.8 cmp.c
--- commands/cmp.c	21 Jul 2007 23:32:19 -0000	1.8
+++ commands/cmp.c	19 Nov 2007 09:51:13 -0000
@@ -23,7 +23,6 @@
 #include <grub/misc.h>
 #include <grub/file.h>
 #include <grub/mm.h>
-#include <grub/gzio.h>
 
 #define BUFFER_SIZE 512
 
@@ -44,8 +43,8 @@ grub_cmd_cmp (struct grub_arg_list *stat
   grub_printf ("Compare `%s' and `%s':\n", args[0],
 	       args[1]);
 
-  file1 = grub_gzfile_open (args[0], 1);
-  file2 = grub_gzfile_open (args[1], 1);
+  file1 = grub_file_open (args[0]);
+  file2 = grub_file_open (args[1]);
   if (! file1 || ! file2)
     goto cleanup;
 
Index: commands/hexdump.c
===================================================================
RCS file: /sources/grub/grub2/commands/hexdump.c,v
retrieving revision 1.1
diff -u -p -r1.1 hexdump.c
--- commands/hexdump.c	18 Nov 2007 06:41:45 -0000	1.1
+++ commands/hexdump.c	19 Nov 2007 09:51:13 -0000
@@ -23,7 +23,6 @@
 #include <grub/file.h>
 #include <grub/disk.h>
 #include <grub/misc.h>
-#include <grub/gzio.h>
 #include <grub/hexdump.h>
 
 static const struct grub_arg_option options[] = {
@@ -100,7 +99,7 @@ grub_cmd_hexdump (struct grub_arg_list *
 
   if (is_file)
     {
-      file = grub_gzfile_open (args[0], 1);
+      file = grub_file_open (args[0]);
       if (!file)
 	return 0;
 
Index: include/grub/file.h
===================================================================
RCS file: /sources/grub/grub2/include/grub/file.h,v
retrieving revision 1.6
diff -u -p -r1.6 file.h
--- include/grub/file.h	2 Aug 2007 18:40:36 -0000	1.6
+++ include/grub/file.h	19 Nov 2007 09:51:13 -0000
@@ -48,9 +48,20 @@ struct grub_file
 };
 typedef struct grub_file *grub_file_t;
 
+struct grub_fshook
+{
+  grub_file_t (*open_func)(grub_file_t file);
+  struct grub_fshook *next;
+};
+typedef struct grub_fshook *grub_fshook_t;
+
+void EXPORT_FUNC(grub_fshook_register) (grub_fshook_t fshook);
+void EXPORT_FUNC(grub_fshook_unregister) (grub_fshook_t fshook);
+
 /* Get a device name from NAME.  */
 char *EXPORT_FUNC(grub_file_get_device_name) (const char *name);
 
+grub_file_t EXPORT_FUNC(grub_file_open_raw) (const char *name);
 grub_file_t EXPORT_FUNC(grub_file_open) (const char *name);
 grub_ssize_t EXPORT_FUNC(grub_file_read) (grub_file_t file, char *buf,
 					  grub_size_t len);
Index: io/gzio.c
===================================================================
RCS file: /sources/grub/grub2/io/gzio.c,v
retrieving revision 1.5
diff -u -p -r1.5 gzio.c
--- io/gzio.c	21 Jul 2007 23:32:25 -0000	1.5
+++ io/gzio.c	19 Nov 2007 09:51:18 -0000
@@ -36,6 +36,7 @@
 
 #include <grub/err.h>
 #include <grub/types.h>
+#include <grub/dl.h>
 #include <grub/mm.h>
 #include <grub/misc.h>
 #include <grub/fs.h>
@@ -1241,3 +1242,25 @@ static struct grub_fs grub_gzio_fs =
     .label = 0,
     .next = 0
   };
+
+static grub_file_t grub_fshook_open(grub_file_t file)
+{
+  return grub_gzio_open (file, 1);
+}
+
+static struct grub_fshook grub_fshook_gzio =
+  {
+    .open_func = grub_fshook_open,
+    .next = 0
+  };
+
+GRUB_MOD_INIT(gzio)
+{
+  (void) mod;			/* To stop warning. */
+  grub_fshook_register(&grub_fshook_gzio);
+}
+
+GRUB_MOD_FINI(gzio)
+{
+  grub_fshook_unregister(&grub_fshook_gzio);
+}
Index: kern/elf.c
===================================================================
RCS file: /sources/grub/grub2/kern/elf.c,v
retrieving revision 1.7
diff -u -p -r1.7 elf.c
--- kern/elf.c	21 Jul 2007 23:32:26 -0000	1.7
+++ kern/elf.c	19 Nov 2007 09:51:19 -0000
@@ -21,7 +21,6 @@
 #include <grub/elf.h>
 #include <grub/elfload.h>
 #include <grub/file.h>
-#include <grub/gzio.h>
 #include <grub/misc.h>
 #include <grub/mm.h>
 
@@ -96,7 +95,7 @@ grub_elf_open (const char *name)
 {
   grub_file_t file;
 
-  file = grub_gzfile_open (name, 1);
+  file = grub_file_open (name);
   if (! file)
     return 0;
 
Index: kern/file.c
===================================================================
RCS file: /sources/grub/grub2/kern/file.c,v
retrieving revision 1.7
diff -u -p -r1.7 file.c
--- kern/file.c	21 Jul 2007 23:32:26 -0000	1.7
+++ kern/file.c	19 Nov 2007 09:51:20 -0000
@@ -23,6 +23,30 @@
 #include <grub/mm.h>
 #include <grub/fs.h>
 #include <grub/device.h>
+#include <grub/env.h>
+
+static grub_fshook_t grub_fshook_list;
+
+void
+grub_fshook_register (grub_fshook_t fshook)
+{
+  fshook->next = grub_fshook_list;
+  grub_fshook_list = fshook;
+}
+
+void
+grub_fshook_unregister (grub_fshook_t fshook)
+{
+  grub_fshook_t *p, q;
+
+  for (p = &grub_fshook_list, q = *p; q; p = &(q->next), q = q->next)
+    if (q == fshook)
+      {
+	*p = q->next;
+	break;
+      }
+}
+
 
 /* Get the device part of the filename NAME. It is enclosed by parentheses.  */
 char *
@@ -52,7 +76,7 @@ grub_file_get_device_name (const char *n
 }
 
 grub_file_t
-grub_file_open (const char *name)
+grub_file_open_raw (const char *name)
 {
   grub_device_t device;
   grub_file_t file = 0;
@@ -110,6 +134,34 @@ grub_file_open (const char *name)
   return 0;
 }
 
+grub_file_t
+grub_file_open (const char *name)
+{
+  grub_file_t file;
+  grub_fshook_t p;
+  char *val;
+
+  file=grub_file_open_raw (name);
+
+  if (! file)
+    return file;
+
+  val= grub_env_get ("nofshook");
+  if ((val) && (val[0]=='1'))
+    return file;
+
+  for (p=grub_fshook_list;p;p=p->next)
+    {
+      grub_file_t new_file;
+
+      new_file = (*p->open_func) (file);
+      if (new_file)
+        return new_file;
+    }
+
+  return file;
+}
+
 grub_ssize_t
 grub_file_read (grub_file_t file, char *buf, grub_size_t len)
 {
Index: loader/multiboot2.c
===================================================================
RCS file: /sources/grub/grub2/loader/multiboot2.c,v
retrieving revision 1.1
diff -u -p -r1.1 multiboot2.c
--- loader/multiboot2.c	25 Jul 2007 00:44:02 -0000	1.1
+++ loader/multiboot2.c	19 Nov 2007 09:51:21 -0000
@@ -28,7 +28,6 @@
 #include <grub/dl.h>
 #include <grub/mm.h>
 #include <grub/misc.h>
-#include <grub/gzio.h>
 
 static grub_addr_t entry;
 extern grub_dl_t my_mod;
@@ -323,7 +322,7 @@ grub_multiboot2 (int argc, char *argv[])
       goto fail;
     }
 
-  file = grub_gzfile_open (argv[0], 1);
+  file = grub_file_open (argv[0]);
   if (! file)
     {
       grub_error (GRUB_ERR_BAD_ARGUMENT, "Couldn't open file");
@@ -423,7 +422,7 @@ grub_module2 (int argc, char *argv[])
     }
 
   /* Load module data.  */
-  file = grub_gzfile_open (argv[0], 1);
+  file = grub_file_open (argv[0]);
   if (! file)
     goto out;
 
Index: loader/multiboot_loader.c
===================================================================
RCS file: /sources/grub/grub2/loader/multiboot_loader.c,v
retrieving revision 1.1
diff -u -p -r1.1 multiboot_loader.c
--- loader/multiboot_loader.c	25 Jul 2007 00:44:02 -0000	1.1
+++ loader/multiboot_loader.c	19 Nov 2007 09:51:23 -0000
@@ -28,7 +28,6 @@
 #include <grub/dl.h>
 #include <grub/mm.h>
 #include <grub/misc.h>
-#include <grub/gzio.h>
 
 grub_dl_t my_mod;
 
@@ -85,7 +84,7 @@ grub_rescue_cmd_multiboot_loader (int ar
       goto fail;
     }
 
-  file = grub_gzfile_open (argv[0], 1);
+  file = grub_file_open (argv[0]);
   if (! file)
     {
       grub_error (GRUB_ERR_BAD_ARGUMENT, "Couldn't open file");
Index: loader/i386/pc/multiboot.c
===================================================================
RCS file: /sources/grub/grub2/loader/i386/pc/multiboot.c,v
retrieving revision 1.14
diff -u -p -r1.14 multiboot.c
--- loader/i386/pc/multiboot.c	31 Oct 2007 22:14:50 -0000	1.14
+++ loader/i386/pc/multiboot.c	19 Nov 2007 09:51:25 -0000
@@ -42,7 +42,6 @@
 #include <grub/dl.h>
 #include <grub/mm.h>
 #include <grub/misc.h>
-#include <grub/gzio.h>
 
 extern grub_dl_t my_mod;
 static struct grub_multiboot_info *mbi;
@@ -253,7 +252,7 @@ grub_multiboot (int argc, char *argv[])
       goto fail;
     }
 
-  file = grub_gzfile_open (argv[0], 1);
+  file = grub_file_open (argv[0]);
   if (! file)
     {
       grub_error (GRUB_ERR_BAD_ARGUMENT, "Couldn't open file");
@@ -362,7 +361,7 @@ grub_module  (int argc, char *argv[])
       goto fail;
     }
 
-  file = grub_gzfile_open (argv[0], 1);
+  file = grub_file_open (argv[0]);
   if (! file)
     goto fail;
 

Re: Transparent decompression with file system filter

[Bean]

[-- Attachment #1: Type: text/plain, Size: 267 bytes --]

Hi,

Changes in this new patch:

1, change function name grub_file_open_raw to grub_file_ropen
2, replace grub_file_open in command/blocklist.c and
util/i386/pc/grub-setup.c to grub_file_ropen.

If nobody objects, i would like to commit this in a few days.

-- 
Bean

[-- Attachment #2: grub2-fshook3.diff --]
[-- Type: application/octet-stream, Size: 9700 bytes --]

2007-12-31  Bean  <bean123ch@gmail.com>

	* include/grub/file.h (grub_fshook): New structure.
	(grub_fshook_register): New function.
	(grub_fshook_unregister): New function.
	(grub_open_file_raw): New function.

	* kern/file.c (grub_fshook_list): New variable.
	(grub_fshook_register): New function.
	(grub_fshook_unregister): New function.
	(grub_open_rfile): The original grub_open_file.
	(grub_open_file): New function.

	* io/gzio.c (grub_fshook_open): New function.
	(grub_fshook_gzio): New variable.
	
	* commands/cat.c (grub_cmd_cat): Replace grub_gzfile_open with grub_file_open.

	* commands/cmp.c (grub_cmd_cmp): Likewise.

	* commands/hexdump.c (grub_cmd_hexdump): Likewise.

	* kern/elf.c (grub_elf_open): Likewise.

	* loader/multiboot2.c (grub_multiboot2): Likewise.
	(grub_module2): Likewise.

	* loader/multiboot_loader.c (grub_rescue_cmd_multiboot_loader): Likewise.

	* loader/i386/pc/multiboot.c (grub_multiboot): Likewise.
	(grub_module): Likewise.

	* command/blocklist.c (grub_cmd_blocklist): Replace grub_file_open with grub_file_ropen.

	* util/i386/pc/grub-setup.c (setup): Likewise.


diff --git a/commands/blocklist.c b/commands/blocklist.c
index df0e632..fde5c58 100644
--- a/commands/blocklist.c
+++ b/commands/blocklist.c
@@ -82,7 +82,7 @@ grub_cmd_blocklist (struct grub_arg_list *state __attribute__ ((unused)),
   if (argc < 1)
     return grub_error (GRUB_ERR_BAD_ARGUMENT, "no file specified");
 
-  file = grub_file_open (args[0]);
+  file = grub_file_ropen (args[0]);
   if (! file)
     return grub_errno;
 
diff --git a/commands/cat.c b/commands/cat.c
index 35b4c14..c6db38b 100644
--- a/commands/cat.c
+++ b/commands/cat.c
@@ -24,7 +24,6 @@
 #include <grub/disk.h>
 #include <grub/term.h>
 #include <grub/misc.h>
-#include <grub/gzio.h>
 
 static grub_err_t
 grub_cmd_cat (struct grub_arg_list *state __attribute__ ((unused)),
@@ -38,7 +37,7 @@ grub_cmd_cat (struct grub_arg_list *state __attribute__ ((unused)),
   if (argc != 1)
     return grub_error (GRUB_ERR_BAD_ARGUMENT, "file name required");
 
-  file = grub_gzfile_open (args[0], 1);
+  file = grub_file_open (args[0]);
   if (! file)
     return 0;
   
diff --git a/commands/cmp.c b/commands/cmp.c
index bb4de60..d7eaf4a 100644
--- a/commands/cmp.c
+++ b/commands/cmp.c
@@ -23,7 +23,6 @@
 #include <grub/misc.h>
 #include <grub/file.h>
 #include <grub/mm.h>
-#include <grub/gzio.h>
 
 #define BUFFER_SIZE 512
 
@@ -44,8 +43,8 @@ grub_cmd_cmp (struct grub_arg_list *state __attribute__ ((unused)),
   grub_printf ("Compare `%s' and `%s':\n", args[0],
 	       args[1]);
 
-  file1 = grub_gzfile_open (args[0], 1);
-  file2 = grub_gzfile_open (args[1], 1);
+  file1 = grub_file_open (args[0]);
+  file2 = grub_file_open (args[1]);
   if (! file1 || ! file2)
     goto cleanup;
 
diff --git a/commands/hexdump.c b/commands/hexdump.c
index d353d5e..a5d5f16 100644
--- a/commands/hexdump.c
+++ b/commands/hexdump.c
@@ -23,7 +23,6 @@
 #include <grub/file.h>
 #include <grub/disk.h>
 #include <grub/misc.h>
-#include <grub/gzio.h>
 #include <grub/hexdump.h>
 
 static const struct grub_arg_option options[] = {
@@ -100,7 +99,7 @@ grub_cmd_hexdump (struct grub_arg_list *state, int argc, char **args)
 
   if (is_file)
     {
-      file = grub_gzfile_open (args[0], 1);
+      file = grub_file_open (args[0]);
       if (!file)
 	return 0;
 
diff --git a/include/grub/file.h b/include/grub/file.h
index df2e9e4..008ebd0 100644
--- a/include/grub/file.h
+++ b/include/grub/file.h
@@ -48,9 +48,20 @@ struct grub_file
 };
 typedef struct grub_file *grub_file_t;
 
+struct grub_fshook
+{
+  grub_file_t (*open_func)(grub_file_t file);
+  struct grub_fshook *next;
+};
+typedef struct grub_fshook *grub_fshook_t;
+
+void EXPORT_FUNC(grub_fshook_register) (grub_fshook_t fshook);
+void EXPORT_FUNC(grub_fshook_unregister) (grub_fshook_t fshook);
+
 /* Get a device name from NAME.  */
 char *EXPORT_FUNC(grub_file_get_device_name) (const char *name);
 
+grub_file_t EXPORT_FUNC(grub_file_ropen) (const char *name);
 grub_file_t EXPORT_FUNC(grub_file_open) (const char *name);
 grub_ssize_t EXPORT_FUNC(grub_file_read) (grub_file_t file, char *buf,
 					  grub_size_t len);
diff --git a/io/gzio.c b/io/gzio.c
index 0fada6c..7696aa8 100644
--- a/io/gzio.c
+++ b/io/gzio.c
@@ -36,6 +36,7 @@
 
 #include <grub/err.h>
 #include <grub/types.h>
+#include <grub/dl.h>
 #include <grub/mm.h>
 #include <grub/misc.h>
 #include <grub/fs.h>
@@ -1241,3 +1242,25 @@ static struct grub_fs grub_gzio_fs =
     .label = 0,
     .next = 0
   };
+
+static grub_file_t grub_fshook_open(grub_file_t file)
+{
+  return grub_gzio_open (file, 1);
+}
+
+static struct grub_fshook grub_fshook_gzio =
+  {
+    .open_func = grub_fshook_open,
+    .next = 0
+  };
+
+GRUB_MOD_INIT(gzio)
+{
+  (void) mod;			/* To stop warning. */
+  grub_fshook_register(&grub_fshook_gzio);
+}
+
+GRUB_MOD_FINI(gzio)
+{
+  grub_fshook_unregister(&grub_fshook_gzio);
+}
diff --git a/kern/elf.c b/kern/elf.c
index b362949..948035d 100644
--- a/kern/elf.c
+++ b/kern/elf.c
@@ -21,7 +21,6 @@
 #include <grub/elf.h>
 #include <grub/elfload.h>
 #include <grub/file.h>
-#include <grub/gzio.h>
 #include <grub/misc.h>
 #include <grub/mm.h>
 
@@ -96,7 +95,7 @@ grub_elf_open (const char *name)
 {
   grub_file_t file;
 
-  file = grub_gzfile_open (name, 1);
+  file = grub_file_open (name);
   if (! file)
     return 0;
 
diff --git a/kern/file.c b/kern/file.c
index adf55da..f19e823 100644
--- a/kern/file.c
+++ b/kern/file.c
@@ -23,6 +23,30 @@
 #include <grub/mm.h>
 #include <grub/fs.h>
 #include <grub/device.h>
+#include <grub/env.h>
+
+static grub_fshook_t grub_fshook_list;
+
+void
+grub_fshook_register (grub_fshook_t fshook)
+{
+  fshook->next = grub_fshook_list;
+  grub_fshook_list = fshook;
+}
+
+void
+grub_fshook_unregister (grub_fshook_t fshook)
+{
+  grub_fshook_t *p, q;
+
+  for (p = &grub_fshook_list, q = *p; q; p = &(q->next), q = q->next)
+    if (q == fshook)
+      {
+	*p = q->next;
+	break;
+      }
+}
+
 
 /* Get the device part of the filename NAME. It is enclosed by parentheses.  */
 char *
@@ -52,7 +76,7 @@ grub_file_get_device_name (const char *name)
 }
 
 grub_file_t
-grub_file_open (const char *name)
+grub_file_ropen (const char *name)
 {
   grub_device_t device;
   grub_file_t file = 0;
@@ -110,6 +134,34 @@ grub_file_open (const char *name)
   return 0;
 }
 
+grub_file_t
+grub_file_open (const char *name)
+{
+  grub_file_t file;
+  grub_fshook_t p;
+  char *val;
+
+  file=grub_file_ropen (name);
+
+  if (! file)
+    return file;
+
+  val= grub_env_get ("nofshook");
+  if ((val) && (val[0]=='1'))
+    return file;
+
+  for (p=grub_fshook_list;p;p=p->next)
+    {
+      grub_file_t new_file;
+
+      new_file = (*p->open_func) (file);
+      if (new_file)
+        return new_file;
+    }
+
+  return file;
+}
+
 grub_ssize_t
 grub_file_read (grub_file_t file, char *buf, grub_size_t len)
 {
diff --git a/loader/i386/pc/multiboot.c b/loader/i386/pc/multiboot.c
index 250ef47..772962b 100644
--- a/loader/i386/pc/multiboot.c
+++ b/loader/i386/pc/multiboot.c
@@ -42,7 +42,6 @@
 #include <grub/dl.h>
 #include <grub/mm.h>
 #include <grub/misc.h>
-#include <grub/gzio.h>
 
 extern grub_dl_t my_mod;
 static struct grub_multiboot_info *mbi;
@@ -253,7 +252,7 @@ grub_multiboot (int argc, char *argv[])
       goto fail;
     }
 
-  file = grub_gzfile_open (argv[0], 1);
+  file = grub_file_open (argv[0]);
   if (! file)
     {
       grub_error (GRUB_ERR_BAD_ARGUMENT, "Couldn't open file");
@@ -362,7 +361,7 @@ grub_module  (int argc, char *argv[])
       goto fail;
     }
 
-  file = grub_gzfile_open (argv[0], 1);
+  file = grub_file_open (argv[0]);
   if (! file)
     goto fail;
 
diff --git a/loader/multiboot2.c b/loader/multiboot2.c
index 65fdea1..8d6cc34 100644
--- a/loader/multiboot2.c
+++ b/loader/multiboot2.c
@@ -28,7 +28,6 @@
 #include <grub/dl.h>
 #include <grub/mm.h>
 #include <grub/misc.h>
-#include <grub/gzio.h>
 
 static grub_addr_t entry;
 extern grub_dl_t my_mod;
@@ -323,7 +322,7 @@ grub_multiboot2 (int argc, char *argv[])
       goto fail;
     }
 
-  file = grub_gzfile_open (argv[0], 1);
+  file = grub_file_open (argv[0]);
   if (! file)
     {
       grub_error (GRUB_ERR_BAD_ARGUMENT, "Couldn't open file");
@@ -423,7 +422,7 @@ grub_module2 (int argc, char *argv[])
     }
 
   /* Load module data.  */
-  file = grub_gzfile_open (argv[0], 1);
+  file = grub_file_open (argv[0]);
   if (! file)
     goto out;
 
diff --git a/loader/multiboot_loader.c b/loader/multiboot_loader.c
index d6e86f1..6c9c753 100644
--- a/loader/multiboot_loader.c
+++ b/loader/multiboot_loader.c
@@ -28,7 +28,6 @@
 #include <grub/dl.h>
 #include <grub/mm.h>
 #include <grub/misc.h>
-#include <grub/gzio.h>
 
 grub_dl_t my_mod;
 
@@ -85,7 +84,7 @@ grub_rescue_cmd_multiboot_loader (int argc, char *argv[])
       goto fail;
     }
 
-  file = grub_gzfile_open (argv[0], 1);
+  file = grub_file_open (argv[0]);
   if (! file)
     {
       grub_error (GRUB_ERR_BAD_ARGUMENT, "Couldn't open file");
diff --git a/util/i386/pc/grub-setup.c b/util/i386/pc/grub-setup.c
index abbea4c..fb31230 100644
--- a/util/i386/pc/grub-setup.c
+++ b/util/i386/pc/grub-setup.c
@@ -356,7 +356,7 @@ setup (const char *prefix, const char *dir,
       
       grub_disk_cache_invalidate_all ();
       
-      file = grub_file_open (core_path);
+      file = grub_file_ropen (core_path);
       if (file)
 	{
 	  if (grub_file_size (file) != core_size)
@@ -426,7 +426,7 @@ setup (const char *prefix, const char *dir,
     }
   
   /* Now read the core image to determine where the sectors are.  */
-  file = grub_file_open (core_path);
+  file = grub_file_ropen (core_path);
   if (! file)
     grub_util_error ("%s", grub_errmsg);
   

Re: Transparent decompression with file system filter

[Robert Millan]

On Tue, Jan 01, 2008 at 12:59:57AM +0800, Bean wrote:
> Hi,
> 
> Changes in this new patch:
> 
> 1, change function name grub_file_open_raw to grub_file_ropen
> 2, replace grub_file_open in command/blocklist.c and
> util/i386/pc/grub-setup.c to grub_file_ropen.
> 
> If nobody objects, i would like to commit this in a few days.

Please add the missing spaces in:

+grub_file_t
+grub_file_open (const char *name)
+{
+  grub_file_t file;
+  grub_fshook_t p;
+  char *val;
+
+  file=grub_file_ropen (name);
      ^^

+
+  if (! file)
+    return file;
+
+  val= grub_env_get ("nofshook");
+  if ((val) && (val[0]=='1'))
                       ^^^

+    return file;
+
+  for (p=grub_fshook_list;p;p=p->next)
        ^^                ^^^^^

+    {
+      grub_file_t new_file;
+
+      new_file = (*p->open_func) (file);
+      if (new_file)
+        return new_file;
+    }
+
+  return file;
+}

-- 
Robert Millan

<GPLv2> I know my rights; I want my phone call!
<DRM> What use is a phone call, if you are unable to speak?
(as seen on /.)

Re: Transparent decompression with file system filter

[Bean]

[-- Attachment #1: Type: text/plain, Size: 1369 bytes --]

On Jan 1, 2008 2:40 AM, Robert Millan <rmh@aybabtu.com> wrote:
>
> On Tue, Jan 01, 2008 at 12:59:57AM +0800, Bean wrote:
> > Hi,
> >
> > Changes in this new patch:
> >
> > 1, change function name grub_file_open_raw to grub_file_ropen
> > 2, replace grub_file_open in command/blocklist.c and
> > util/i386/pc/grub-setup.c to grub_file_ropen.
> >
> > If nobody objects, i would like to commit this in a few days.
>
> Please add the missing spaces in:
>
> +grub_file_t
> +grub_file_open (const char *name)
> +{
> +  grub_file_t file;
> +  grub_fshook_t p;
> +  char *val;
> +
> +  file=grub_file_ropen (name);
>       ^^
>
> +
> +  if (! file)
> +    return file;
> +
> +  val= grub_env_get ("nofshook");
> +  if ((val) && (val[0]=='1'))
>                        ^^^
>
> +    return file;
> +
> +  for (p=grub_fshook_list;p;p=p->next)
>         ^^                ^^^^^
>
> +    {
> +      grub_file_t new_file;
> +
> +      new_file = (*p->open_func) (file);
> +      if (new_file)
> +        return new_file;
> +    }
> +
> +  return file;
> +}
>
> --
> Robert Millan
>
> <GPLv2> I know my rights; I want my phone call!
> <DRM> What use is a phone call, if you are unable to speak?
> (as seen on /.)
>
>
>
> _______________________________________________
> Grub-devel mailing list
> Grub-devel@gnu.org
> http://lists.gnu.org/mailman/listinfo/grub-devel
>

Ok.

-- 
Bean

[-- Attachment #2: grub2-fshook3.diff --]
[-- Type: application/octet-stream, Size: 9710 bytes --]

2007-12-31  Bean  <bean123ch@gmail.com>

	* include/grub/file.h (grub_fshook): New structure.
	(grub_fshook_register): New function.
	(grub_fshook_unregister): New function.
	(grub_open_file_raw): New function.

	* kern/file.c (grub_fshook_list): New variable.
	(grub_fshook_register): New function.
	(grub_fshook_unregister): New function.
	(grub_open_rfile): The original grub_open_file.
	(grub_open_file): New function.

	* io/gzio.c (grub_fshook_open): New function.
	(grub_fshook_gzio): New variable.
	
	* commands/cat.c (grub_cmd_cat): Replace grub_gzfile_open with grub_file_open.

	* commands/cmp.c (grub_cmd_cmp): Likewise.

	* commands/hexdump.c (grub_cmd_hexdump): Likewise.

	* kern/elf.c (grub_elf_open): Likewise.

	* loader/multiboot2.c (grub_multiboot2): Likewise.
	(grub_module2): Likewise.

	* loader/multiboot_loader.c (grub_rescue_cmd_multiboot_loader): Likewise.

	* loader/i386/pc/multiboot.c (grub_multiboot): Likewise.
	(grub_module): Likewise.

	* command/blocklist.c (grub_cmd_blocklist): Replace grub_file_open with grub_file_ropen.

	* util/i386/pc/grub-setup.c (setup): Likewise.


diff --git a/commands/blocklist.c b/commands/blocklist.c
index df0e632..fde5c58 100644
--- a/commands/blocklist.c
+++ b/commands/blocklist.c
@@ -82,7 +82,7 @@ grub_cmd_blocklist (struct grub_arg_list *state __attribute__ ((unused)),
   if (argc < 1)
     return grub_error (GRUB_ERR_BAD_ARGUMENT, "no file specified");
 
-  file = grub_file_open (args[0]);
+  file = grub_file_ropen (args[0]);
   if (! file)
     return grub_errno;
 
diff --git a/commands/cat.c b/commands/cat.c
index 35b4c14..c6db38b 100644
--- a/commands/cat.c
+++ b/commands/cat.c
@@ -24,7 +24,6 @@
 #include <grub/disk.h>
 #include <grub/term.h>
 #include <grub/misc.h>
-#include <grub/gzio.h>
 
 static grub_err_t
 grub_cmd_cat (struct grub_arg_list *state __attribute__ ((unused)),
@@ -38,7 +37,7 @@ grub_cmd_cat (struct grub_arg_list *state __attribute__ ((unused)),
   if (argc != 1)
     return grub_error (GRUB_ERR_BAD_ARGUMENT, "file name required");
 
-  file = grub_gzfile_open (args[0], 1);
+  file = grub_file_open (args[0]);
   if (! file)
     return 0;
   
diff --git a/commands/cmp.c b/commands/cmp.c
index bb4de60..d7eaf4a 100644
--- a/commands/cmp.c
+++ b/commands/cmp.c
@@ -23,7 +23,6 @@
 #include <grub/misc.h>
 #include <grub/file.h>
 #include <grub/mm.h>
-#include <grub/gzio.h>
 
 #define BUFFER_SIZE 512
 
@@ -44,8 +43,8 @@ grub_cmd_cmp (struct grub_arg_list *state __attribute__ ((unused)),
   grub_printf ("Compare `%s' and `%s':\n", args[0],
 	       args[1]);
 
-  file1 = grub_gzfile_open (args[0], 1);
-  file2 = grub_gzfile_open (args[1], 1);
+  file1 = grub_file_open (args[0]);
+  file2 = grub_file_open (args[1]);
   if (! file1 || ! file2)
     goto cleanup;
 
diff --git a/commands/hexdump.c b/commands/hexdump.c
index d353d5e..a5d5f16 100644
--- a/commands/hexdump.c
+++ b/commands/hexdump.c
@@ -23,7 +23,6 @@
 #include <grub/file.h>
 #include <grub/disk.h>
 #include <grub/misc.h>
-#include <grub/gzio.h>
 #include <grub/hexdump.h>
 
 static const struct grub_arg_option options[] = {
@@ -100,7 +99,7 @@ grub_cmd_hexdump (struct grub_arg_list *state, int argc, char **args)
 
   if (is_file)
     {
-      file = grub_gzfile_open (args[0], 1);
+      file = grub_file_open (args[0]);
       if (!file)
 	return 0;
 
diff --git a/include/grub/file.h b/include/grub/file.h
index df2e9e4..008ebd0 100644
--- a/include/grub/file.h
+++ b/include/grub/file.h
@@ -48,9 +48,20 @@ struct grub_file
 };
 typedef struct grub_file *grub_file_t;
 
+struct grub_fshook
+{
+  grub_file_t (*open_func) (grub_file_t file);
+  struct grub_fshook *next;
+};
+typedef struct grub_fshook *grub_fshook_t;
+
+void EXPORT_FUNC(grub_fshook_register) (grub_fshook_t fshook);
+void EXPORT_FUNC(grub_fshook_unregister) (grub_fshook_t fshook);
+
 /* Get a device name from NAME.  */
 char *EXPORT_FUNC(grub_file_get_device_name) (const char *name);
 
+grub_file_t EXPORT_FUNC(grub_file_ropen) (const char *name);
 grub_file_t EXPORT_FUNC(grub_file_open) (const char *name);
 grub_ssize_t EXPORT_FUNC(grub_file_read) (grub_file_t file, char *buf,
 					  grub_size_t len);
diff --git a/io/gzio.c b/io/gzio.c
index 0fada6c..7696aa8 100644
--- a/io/gzio.c
+++ b/io/gzio.c
@@ -36,6 +36,7 @@
 
 #include <grub/err.h>
 #include <grub/types.h>
+#include <grub/dl.h>
 #include <grub/mm.h>
 #include <grub/misc.h>
 #include <grub/fs.h>
@@ -1241,3 +1242,25 @@ static struct grub_fs grub_gzio_fs =
     .label = 0,
     .next = 0
   };
+
+static grub_file_t grub_fshook_open(grub_file_t file)
+{
+  return grub_gzio_open (file, 1);
+}
+
+static struct grub_fshook grub_fshook_gzio =
+  {
+    .open_func = grub_fshook_open,
+    .next = 0
+  };
+
+GRUB_MOD_INIT(gzio)
+{
+  (void) mod;			/* To stop warning. */
+  grub_fshook_register(&grub_fshook_gzio);
+}
+
+GRUB_MOD_FINI(gzio)
+{
+  grub_fshook_unregister(&grub_fshook_gzio);
+}
diff --git a/kern/elf.c b/kern/elf.c
index b362949..948035d 100644
--- a/kern/elf.c
+++ b/kern/elf.c
@@ -21,7 +21,6 @@
 #include <grub/elf.h>
 #include <grub/elfload.h>
 #include <grub/file.h>
-#include <grub/gzio.h>
 #include <grub/misc.h>
 #include <grub/mm.h>
 
@@ -96,7 +95,7 @@ grub_elf_open (const char *name)
 {
   grub_file_t file;
 
-  file = grub_gzfile_open (name, 1);
+  file = grub_file_open (name);
   if (! file)
     return 0;
 
diff --git a/kern/file.c b/kern/file.c
index adf55da..f19e823 100644
--- a/kern/file.c
+++ b/kern/file.c
@@ -23,6 +23,30 @@
 #include <grub/mm.h>
 #include <grub/fs.h>
 #include <grub/device.h>
+#include <grub/env.h>
+
+static grub_fshook_t grub_fshook_list;
+
+void
+grub_fshook_register (grub_fshook_t fshook)
+{
+  fshook->next = grub_fshook_list;
+  grub_fshook_list = fshook;
+}
+
+void
+grub_fshook_unregister (grub_fshook_t fshook)
+{
+  grub_fshook_t *p, q;
+
+  for (p = &grub_fshook_list, q = *p; q; p = &(q->next), q = q->next)
+    if (q == fshook)
+      {
+	*p = q->next;
+	break;
+      }
+}
+
 
 /* Get the device part of the filename NAME. It is enclosed by parentheses.  */
 char *
@@ -52,7 +76,7 @@ grub_file_get_device_name (const char *name)
 }
 
 grub_file_t
-grub_file_open (const char *name)
+grub_file_ropen (const char *name)
 {
   grub_device_t device;
   grub_file_t file = 0;
@@ -110,6 +134,34 @@ grub_file_open (const char *name)
   return 0;
 }
 
+grub_file_t
+grub_file_open (const char *name)
+{
+  grub_file_t file;
+  grub_fshook_t p;
+  char *val;
+
+  file = grub_file_ropen (name);
+
+  if (! file)
+    return file;
+
+  val = grub_env_get ("nofshook");
+  if ((val) && (val[0] == '1'))
+    return file;
+
+  for (p = grub_fshook_list; p; p=p->next)
+    {
+      grub_file_t new_file;
+
+      new_file = (*p->open_func) (file);
+      if (new_file)
+        return new_file;
+    }
+
+  return file;
+}
+
 grub_ssize_t
 grub_file_read (grub_file_t file, char *buf, grub_size_t len)
 {
diff --git a/loader/i386/pc/multiboot.c b/loader/i386/pc/multiboot.c
index 250ef47..772962b 100644
--- a/loader/i386/pc/multiboot.c
+++ b/loader/i386/pc/multiboot.c
@@ -42,7 +42,6 @@
 #include <grub/dl.h>
 #include <grub/mm.h>
 #include <grub/misc.h>
-#include <grub/gzio.h>
 
 extern grub_dl_t my_mod;
 static struct grub_multiboot_info *mbi;
@@ -253,7 +252,7 @@ grub_multiboot (int argc, char *argv[])
       goto fail;
     }
 
-  file = grub_gzfile_open (argv[0], 1);
+  file = grub_file_open (argv[0]);
   if (! file)
     {
       grub_error (GRUB_ERR_BAD_ARGUMENT, "Couldn't open file");
@@ -362,7 +361,7 @@ grub_module  (int argc, char *argv[])
       goto fail;
     }
 
-  file = grub_gzfile_open (argv[0], 1);
+  file = grub_file_open (argv[0]);
   if (! file)
     goto fail;
 
diff --git a/loader/multiboot2.c b/loader/multiboot2.c
index 65fdea1..8d6cc34 100644
--- a/loader/multiboot2.c
+++ b/loader/multiboot2.c
@@ -28,7 +28,6 @@
 #include <grub/dl.h>
 #include <grub/mm.h>
 #include <grub/misc.h>
-#include <grub/gzio.h>
 
 static grub_addr_t entry;
 extern grub_dl_t my_mod;
@@ -323,7 +322,7 @@ grub_multiboot2 (int argc, char *argv[])
       goto fail;
     }
 
-  file = grub_gzfile_open (argv[0], 1);
+  file = grub_file_open (argv[0]);
   if (! file)
     {
       grub_error (GRUB_ERR_BAD_ARGUMENT, "Couldn't open file");
@@ -423,7 +422,7 @@ grub_module2 (int argc, char *argv[])
     }
 
   /* Load module data.  */
-  file = grub_gzfile_open (argv[0], 1);
+  file = grub_file_open (argv[0]);
   if (! file)
     goto out;
 
diff --git a/loader/multiboot_loader.c b/loader/multiboot_loader.c
index d6e86f1..6c9c753 100644
--- a/loader/multiboot_loader.c
+++ b/loader/multiboot_loader.c
@@ -28,7 +28,6 @@
 #include <grub/dl.h>
 #include <grub/mm.h>
 #include <grub/misc.h>
-#include <grub/gzio.h>
 
 grub_dl_t my_mod;
 
@@ -85,7 +84,7 @@ grub_rescue_cmd_multiboot_loader (int argc, char *argv[])
       goto fail;
     }
 
-  file = grub_gzfile_open (argv[0], 1);
+  file = grub_file_open (argv[0]);
   if (! file)
     {
       grub_error (GRUB_ERR_BAD_ARGUMENT, "Couldn't open file");
diff --git a/util/i386/pc/grub-setup.c b/util/i386/pc/grub-setup.c
index abbea4c..fb31230 100644
--- a/util/i386/pc/grub-setup.c
+++ b/util/i386/pc/grub-setup.c
@@ -356,7 +356,7 @@ setup (const char *prefix, const char *dir,
       
       grub_disk_cache_invalidate_all ();
       
-      file = grub_file_open (core_path);
+      file = grub_file_ropen (core_path);
       if (file)
 	{
 	  if (grub_file_size (file) != core_size)
@@ -426,7 +426,7 @@ setup (const char *prefix, const char *dir,
     }
   
   /* Now read the core image to determine where the sectors are.  */
-  file = grub_file_open (core_path);
+  file = grub_file_ropen (core_path);
   if (! file)
     grub_util_error ("%s", grub_errmsg);
   

Re: Transparent decompression with file system filter

[Yoshinori K. Okuji]

On Monday 31 December 2007 17:59, Bean wrote:
> Hi,
>
> Changes in this new patch:
>
> 1, change function name grub_file_open_raw to grub_file_ropen
> 2, replace grub_file_open in command/blocklist.c and
> util/i386/pc/grub-setup.c to grub_file_ropen.
>
> If nobody objects, i would like to commit this in a few days.

Please wait a minute. Personally, I don't want grub_file_open to decompress a 
content automatically. The name should stand for what it does. That was why I 
named grub_file_open and grub_gzfile_open like this. This was really one of 
what I didn't like in GRUB Legacy.

If you want to have a function to open any kind of compressed file, please add 
something else (e.g. grub_compressed_file_open).

Okuji

Re: Transparent decompression with file system filter

[Bean]

[-- Attachment #1: Type: text/plain, Size: 1123 bytes --]

On Jan 3, 2008 7:19 AM, Yoshinori K. Okuji <okuji@enbug.org> wrote:
>
> On Monday 31 December 2007 17:59, Bean wrote:
> > Hi,
> >
> > Changes in this new patch:
> >
> > 1, change function name grub_file_open_raw to grub_file_ropen
> > 2, replace grub_file_open in command/blocklist.c and
> > util/i386/pc/grub-setup.c to grub_file_ropen.
> >
> > If nobody objects, i would like to commit this in a few days.
>
> Please wait a minute. Personally, I don't want grub_file_open to decompress a
> content automatically. The name should stand for what it does. That was why I
> named grub_file_open and grub_gzfile_open like this. This was really one of
> what I didn't like in GRUB Legacy.
>
> If you want to have a function to open any kind of compressed file, please add
> something else (e.g. grub_compressed_file_open).

ok, i keep grub_file_open as it is, and add a new function
grub_zfile_open to handle compressed file. i also expand the list of
commands that use auto decompression:

ls
to get the uncompressed file size in long list

loopback
to open compressed image file

font
to open compressed font file.

-- 
Bean

[-- Attachment #2: grub2-fshook4.diff --]
[-- Type: application/octet-stream, Size: 9614 bytes --]

2008-01-03  Bean  <bean123ch@gmail.com>

	* include/grub/file.h (grub_fshook): New structure.
	(grub_fshook_register): New function.
	(grub_fshook_unregister): New function.
	(grub_zfile_open): New function.

	* kern/file.c (grub_fshook_list): New variable.
	(grub_fshook_register): New function.
	(grub_fshook_unregister): New function.
	(grub_zfile_open): New function.

	* io/gzio.c (grub_fshook_open): New function.
	(grub_fshook_gzio): New variable.

	* commands/cat.c (grub_cmd_cat): Use grub_zfile_open instead of grub_gzfile_open.

	* commands/cmp.c (grub_cmd_cmp): Likewise.

	* commands/hexdump.c (grub_cmd_hexdump): Likewise.

	* kern/elf.c (grub_elf_open): Likewise.

	* loader/multiboot2.c (grub_multiboot2): Likewise.
	(grub_module2): Likewise.

	* loader/multiboot_loader.c (grub_rescue_cmd_multiboot_loader): Likewise.

	* loader/i386/pc/multiboot.c (grub_multiboot): Likewise.
	(grub_module): Likewise.
	
	* commands/ls.c (grub_ls_list_files): Use grub_zfile_open instead of grub_file_open.

	* disk/loopback.c (grub_loopback_open): Likewise.

	* font/manager.c (add_font): Likewise.


diff --git a/commands/cat.c b/commands/cat.c
index 35b4c14..466ce16 100644
--- a/commands/cat.c
+++ b/commands/cat.c
@@ -24,7 +24,6 @@
 #include <grub/disk.h>
 #include <grub/term.h>
 #include <grub/misc.h>
-#include <grub/gzio.h>
 
 static grub_err_t
 grub_cmd_cat (struct grub_arg_list *state __attribute__ ((unused)),
@@ -38,7 +37,7 @@ grub_cmd_cat (struct grub_arg_list *state __attribute__ ((unused)),
   if (argc != 1)
     return grub_error (GRUB_ERR_BAD_ARGUMENT, "file name required");
 
-  file = grub_gzfile_open (args[0], 1);
+  file = grub_zfile_open (args[0]);
   if (! file)
     return 0;
   
diff --git a/commands/cmp.c b/commands/cmp.c
index bb4de60..3267ebc 100644
--- a/commands/cmp.c
+++ b/commands/cmp.c
@@ -23,7 +23,6 @@
 #include <grub/misc.h>
 #include <grub/file.h>
 #include <grub/mm.h>
-#include <grub/gzio.h>
 
 #define BUFFER_SIZE 512
 
@@ -44,8 +43,8 @@ grub_cmd_cmp (struct grub_arg_list *state __attribute__ ((unused)),
   grub_printf ("Compare `%s' and `%s':\n", args[0],
 	       args[1]);
 
-  file1 = grub_gzfile_open (args[0], 1);
-  file2 = grub_gzfile_open (args[1], 1);
+  file1 = grub_zfile_open (args[0]);
+  file2 = grub_zfile_open (args[1]);
   if (! file1 || ! file2)
     goto cleanup;
 
diff --git a/commands/hexdump.c b/commands/hexdump.c
index d353d5e..48547a6 100644
--- a/commands/hexdump.c
+++ b/commands/hexdump.c
@@ -23,7 +23,6 @@
 #include <grub/file.h>
 #include <grub/disk.h>
 #include <grub/misc.h>
-#include <grub/gzio.h>
 #include <grub/hexdump.h>
 
 static const struct grub_arg_option options[] = {
@@ -100,7 +99,7 @@ grub_cmd_hexdump (struct grub_arg_list *state, int argc, char **args)
 
   if (is_file)
     {
-      file = grub_gzfile_open (args[0], 1);
+      file = grub_zfile_open (args[0]);
       if (!file)
 	return 0;
 
diff --git a/commands/ls.c b/commands/ls.c
index e7b6302..d937b11 100644
--- a/commands/ls.c
+++ b/commands/ls.c
@@ -97,7 +97,7 @@ grub_ls_list_files (char *dirname, int longlist, int all, int human)
 
 	  /* XXX: For ext2fs symlinks are detected as files while they
 	     should be reported as directories.  */
-	  file = grub_file_open (pathname);
+	  file = grub_zfile_open (pathname);
 	  if (! file)
 	    {
 	      grub_errno = 0;
diff --git a/disk/loopback.c b/disk/loopback.c
index 31d8116..1570085 100644
--- a/disk/loopback.c
+++ b/disk/loopback.c
@@ -171,7 +171,7 @@ grub_loopback_open (const char *name, grub_disk_t disk)
   if (! dev)
     return grub_error (GRUB_ERR_UNKNOWN_DEVICE, "Can't open device");
 
-  file = grub_file_open (dev->filename);
+  file = grub_zfile_open (dev->filename);
   if (! file)
     return grub_errno;
   
diff --git a/font/manager.c b/font/manager.c
index 8d5f021..5b165ce 100644
--- a/font/manager.c
+++ b/font/manager.c
@@ -48,7 +48,7 @@ add_font (const char *filename)
   grub_uint32_t num, i;
   struct font *font = 0;
 
-  file = grub_file_open (filename);
+  file = grub_zfile_open (filename);
   if (! file)
     goto fail;
 
diff --git a/include/grub/file.h b/include/grub/file.h
index df2e9e4..a6ef0ca 100644
--- a/include/grub/file.h
+++ b/include/grub/file.h
@@ -48,10 +48,21 @@ struct grub_file
 };
 typedef struct grub_file *grub_file_t;
 
+struct grub_fshook
+{
+  grub_file_t (*open_func) (grub_file_t file);
+  struct grub_fshook *next;
+};
+typedef struct grub_fshook *grub_fshook_t;
+
+void EXPORT_FUNC(grub_fshook_register) (grub_fshook_t fshook);
+void EXPORT_FUNC(grub_fshook_unregister) (grub_fshook_t fshook);
+
 /* Get a device name from NAME.  */
 char *EXPORT_FUNC(grub_file_get_device_name) (const char *name);
 
 grub_file_t EXPORT_FUNC(grub_file_open) (const char *name);
+grub_file_t EXPORT_FUNC(grub_zfile_open) (const char *name);
 grub_ssize_t EXPORT_FUNC(grub_file_read) (grub_file_t file, char *buf,
 					  grub_size_t len);
 grub_off_t EXPORT_FUNC(grub_file_seek) (grub_file_t file, grub_off_t offset);
diff --git a/io/gzio.c b/io/gzio.c
index 0fada6c..7696aa8 100644
--- a/io/gzio.c
+++ b/io/gzio.c
@@ -36,6 +36,7 @@
 
 #include <grub/err.h>
 #include <grub/types.h>
+#include <grub/dl.h>
 #include <grub/mm.h>
 #include <grub/misc.h>
 #include <grub/fs.h>
@@ -1241,3 +1242,25 @@ static struct grub_fs grub_gzio_fs =
     .label = 0,
     .next = 0
   };
+
+static grub_file_t grub_fshook_open(grub_file_t file)
+{
+  return grub_gzio_open (file, 1);
+}
+
+static struct grub_fshook grub_fshook_gzio =
+  {
+    .open_func = grub_fshook_open,
+    .next = 0
+  };
+
+GRUB_MOD_INIT(gzio)
+{
+  (void) mod;			/* To stop warning. */
+  grub_fshook_register(&grub_fshook_gzio);
+}
+
+GRUB_MOD_FINI(gzio)
+{
+  grub_fshook_unregister(&grub_fshook_gzio);
+}
diff --git a/kern/elf.c b/kern/elf.c
index b362949..c90d6f3 100644
--- a/kern/elf.c
+++ b/kern/elf.c
@@ -21,7 +21,6 @@
 #include <grub/elf.h>
 #include <grub/elfload.h>
 #include <grub/file.h>
-#include <grub/gzio.h>
 #include <grub/misc.h>
 #include <grub/mm.h>
 
@@ -96,7 +95,7 @@ grub_elf_open (const char *name)
 {
   grub_file_t file;
 
-  file = grub_gzfile_open (name, 1);
+  file = grub_zfile_open (name);
   if (! file)
     return 0;
 
diff --git a/kern/file.c b/kern/file.c
index adf55da..7e60bb7 100644
--- a/kern/file.c
+++ b/kern/file.c
@@ -23,6 +23,30 @@
 #include <grub/mm.h>
 #include <grub/fs.h>
 #include <grub/device.h>
+#include <grub/env.h>
+
+static grub_fshook_t grub_fshook_list;
+
+void
+grub_fshook_register (grub_fshook_t fshook)
+{
+  fshook->next = grub_fshook_list;
+  grub_fshook_list = fshook;
+}
+
+void
+grub_fshook_unregister (grub_fshook_t fshook)
+{
+  grub_fshook_t *p, q;
+
+  for (p = &grub_fshook_list, q = *p; q; p = &(q->next), q = q->next)
+    if (q == fshook)
+      {
+	*p = q->next;
+	break;
+      }
+}
+
 
 /* Get the device part of the filename NAME. It is enclosed by parentheses.  */
 char *
@@ -110,6 +134,34 @@ grub_file_open (const char *name)
   return 0;
 }
 
+grub_file_t
+grub_zfile_open (const char *name)
+{
+  grub_file_t file;
+  grub_fshook_t p;
+  char *val;
+
+  file = grub_file_open (name);
+
+  if (! file)
+    return file;
+
+  val = grub_env_get ("nofshook");
+  if ((val) && (val[0] == '1'))
+    return file;
+
+  for (p = grub_fshook_list; p; p=p->next)
+    {
+      grub_file_t new_file;
+
+      new_file = (*p->open_func) (file);
+      if (new_file)
+        return new_file;
+    }
+
+  return file;
+}
+
 grub_ssize_t
 grub_file_read (grub_file_t file, char *buf, grub_size_t len)
 {
diff --git a/loader/i386/pc/multiboot.c b/loader/i386/pc/multiboot.c
index 250ef47..82e567b 100644
--- a/loader/i386/pc/multiboot.c
+++ b/loader/i386/pc/multiboot.c
@@ -42,7 +42,6 @@
 #include <grub/dl.h>
 #include <grub/mm.h>
 #include <grub/misc.h>
-#include <grub/gzio.h>
 
 extern grub_dl_t my_mod;
 static struct grub_multiboot_info *mbi;
@@ -253,7 +252,7 @@ grub_multiboot (int argc, char *argv[])
       goto fail;
     }
 
-  file = grub_gzfile_open (argv[0], 1);
+  file = grub_zfile_open (argv[0]);
   if (! file)
     {
       grub_error (GRUB_ERR_BAD_ARGUMENT, "Couldn't open file");
@@ -362,7 +361,7 @@ grub_module  (int argc, char *argv[])
       goto fail;
     }
 
-  file = grub_gzfile_open (argv[0], 1);
+  file = grub_zfile_open (argv[0]);
   if (! file)
     goto fail;
 
diff --git a/loader/multiboot2.c b/loader/multiboot2.c
index 65fdea1..77697b2 100644
--- a/loader/multiboot2.c
+++ b/loader/multiboot2.c
@@ -28,7 +28,6 @@
 #include <grub/dl.h>
 #include <grub/mm.h>
 #include <grub/misc.h>
-#include <grub/gzio.h>
 
 static grub_addr_t entry;
 extern grub_dl_t my_mod;
@@ -323,7 +322,7 @@ grub_multiboot2 (int argc, char *argv[])
       goto fail;
     }
 
-  file = grub_gzfile_open (argv[0], 1);
+  file = grub_zfile_open (argv[0]);
   if (! file)
     {
       grub_error (GRUB_ERR_BAD_ARGUMENT, "Couldn't open file");
@@ -423,7 +422,7 @@ grub_module2 (int argc, char *argv[])
     }
 
   /* Load module data.  */
-  file = grub_gzfile_open (argv[0], 1);
+  file = grub_zfile_open (argv[0]);
   if (! file)
     goto out;
 
diff --git a/loader/multiboot_loader.c b/loader/multiboot_loader.c
index d6e86f1..16d9999 100644
--- a/loader/multiboot_loader.c
+++ b/loader/multiboot_loader.c
@@ -28,7 +28,6 @@
 #include <grub/dl.h>
 #include <grub/mm.h>
 #include <grub/misc.h>
-#include <grub/gzio.h>
 
 grub_dl_t my_mod;
 
@@ -85,7 +84,7 @@ grub_rescue_cmd_multiboot_loader (int argc, char *argv[])
       goto fail;
     }
 
-  file = grub_gzfile_open (argv[0], 1);
+  file = grub_zfile_open (argv[0]);
   if (! file)
     {
       grub_error (GRUB_ERR_BAD_ARGUMENT, "Couldn't open file");

Re: Transparent decompression with file system filter

[Robert Millan]

On Thu, Jan 03, 2008 at 04:49:24PM +0800, Bean wrote:
> On Jan 3, 2008 7:19 AM, Yoshinori K. Okuji <okuji@enbug.org> wrote:
> >
> > On Monday 31 December 2007 17:59, Bean wrote:
> > > Hi,
> > >
> > > Changes in this new patch:
> > >
> > > 1, change function name grub_file_open_raw to grub_file_ropen
> > > 2, replace grub_file_open in command/blocklist.c and
> > > util/i386/pc/grub-setup.c to grub_file_ropen.
> > >
> > > If nobody objects, i would like to commit this in a few days.
> >
> > Please wait a minute. Personally, I don't want grub_file_open to decompress a
> > content automatically. The name should stand for what it does. That was why I
> > named grub_file_open and grub_gzfile_open like this. This was really one of
> > what I didn't like in GRUB Legacy.
> >
> > If you want to have a function to open any kind of compressed file, please add
> > something else (e.g. grub_compressed_file_open).
> 
> ok, i keep grub_file_open as it is, and add a new function
> grub_zfile_open to handle compressed file.

How does grub_zfile_open differ from grub_gzfile_open ?

-- 
Robert Millan

<GPLv2> I know my rights; I want my phone call!
<DRM> What use is a phone call, if you are unable to speak?
(as seen on /.)

Re: Transparent decompression with file system filter

[Bean]

On Jan 3, 2008 7:35 PM, Robert Millan <rmh@aybabtu.com> wrote:
> How does grub_zfile_open differ from grub_gzfile_open ?

grub_gzfile_open only handle gzip file, while grub_zfile_open handle
compressed file in general. when module like gzio starts, it register
a structure that grub_zfile_open would use to call the decompression
routine. the binding is dynamic, the modules that use grub_zfile_open
doesn't depend on the specific decompression module.

-- 
Bean

Re: Transparent decompression with file system filter

[Robert Millan]

On Thu, Jan 03, 2008 at 08:53:19PM +0800, Bean wrote:
> On Jan 3, 2008 7:35 PM, Robert Millan <rmh@aybabtu.com> wrote:
> > How does grub_zfile_open differ from grub_gzfile_open ?
> 
> grub_gzfile_open only handle gzip file, while grub_zfile_open handle
> compressed file in general. when module like gzio starts, it register
> a structure that grub_zfile_open would use to call the decompression
> routine. the binding is dynamic, the modules that use grub_zfile_open
> doesn't depend on the specific decompression module.

Ah, sounds nice.

-- 
Robert Millan

<GPLv2> I know my rights; I want my phone call!
<DRM> What use is a phone call, if you are unable to speak?
(as seen on /.)

Re: Transparent decompression with file system filter

[Vesa Jääskeläinen]

Bean wrote:
> On Jan 3, 2008 7:19 AM, Yoshinori K. Okuji <okuji@enbug.org> wrote:
>> Please wait a minute. Personally, I don't want grub_file_open to decompress a
>> content automatically. The name should stand for what it does. That was why I
>> named grub_file_open and grub_gzfile_open like this. This was really one of
>> what I didn't like in GRUB Legacy.
>>
>> If you want to have a function to open any kind of compressed file, please add
>> something else (e.g. grub_compressed_file_open).
> 
> ok, i keep grub_file_open as it is, and add a new function
> grub_zfile_open to handle compressed file. i also expand the list of
> commands that use auto decompression:

For me zfile doesn't say a thing. Where as name proposed by Okuji is
clear to me instantly. Our current compilers can handle long symbol
names fine, so why not use them if they make things clearer :) ?

Re: Transparent decompression with file system filter

[Yoshinori K. Okuji]

On Thursday 03 January 2008 16:53, Vesa Jääskeläinen wrote:
> Bean wrote:
> > On Jan 3, 2008 7:19 AM, Yoshinori K. Okuji <okuji@enbug.org> wrote:
> >> Please wait a minute. Personally, I don't want grub_file_open to
> >> decompress a content automatically. The name should stand for what it
> >> does. That was why I named grub_file_open and grub_gzfile_open like
> >> this. This was really one of what I didn't like in GRUB Legacy.
> >>
> >> If you want to have a function to open any kind of compressed file,
> >> please add something else (e.g. grub_compressed_file_open).
> >
> > ok, i keep grub_file_open as it is, and add a new function
> > grub_zfile_open to handle compressed file. i also expand the list of
> > commands that use auto decompression:
>
> For me zfile doesn't say a thing. Where as name proposed by Okuji is
> clear to me instantly. Our current compilers can handle long symbol
> names fine, so why not use them if they make things clearer :) ?

Agreed.

Okuji

Re: Transparent decompression with file system filter

[Bean]

[-- Attachment #1: Type: text/plain, Size: 1647 bytes --]

On Jan 5, 2008 9:29 AM, Yoshinori K. Okuji <okuji@enbug.org> wrote:
> On Thursday 03 January 2008 16:53, Vesa Jääskeläinen wrote:
> > Bean wrote:
> > > On Jan 3, 2008 7:19 AM, Yoshinori K. Okuji <okuji@enbug.org> wrote:
> > >> Please wait a minute. Personally, I don't want grub_file_open to
> > >> decompress a content automatically. The name should stand for what it
> > >> does. That was why I named grub_file_open and grub_gzfile_open like
> > >> this. This was really one of what I didn't like in GRUB Legacy.
> > >>
> > >> If you want to have a function to open any kind of compressed file,
> > >> please add something else (e.g. grub_compressed_file_open).
> > >
> > > ok, i keep grub_file_open as it is, and add a new function
> > > grub_zfile_open to handle compressed file. i also expand the list of
> > > commands that use auto decompression:
> >
> > For me zfile doesn't say a thing. Where as name proposed by Okuji is
> > clear to me instantly. Our current compilers can handle long symbol
> > names fine, so why not use them if they make things clearer :) ?
>
> Agreed.

I'm thinking that compressed might not be the right word, as filter
doesn't limit to decompression, it can support decryption or format
conversion as well. I guess the name grub_file_open_ex may be better.
I also add a flag to the function, if flag!=0, filters are used, and
if flag=0, grub_file_open_ex is just like grub_file_open.

I also change the name fshook to filehook, as these filters only
affect file based operation. The controlling variable is now filehook.
When filehook=0, filters are disabled globally.

-- 
Bean

[-- Attachment #2: grub2-filehook.diff --]
[-- Type: application/octet-stream, Size: 9809 bytes --]

2008-01-05  Bean  <bean123ch@gmail.com>

	* include/grub/file.h (grub_filehook): New structure.
	(grub_filehook_register): New function.
	(grub_filehook_unregister): New function.
	(grub_file_open_ex): New function.

	* kern/file.c (grub_filehook_list): New variable.
	(grub_filehook_register): New function.
	(grub_filehook_unregister): New function.
	(grub_file_open_ex): New function.

	* io/gzio.c (grub_filehook_open): New function.
	(grub_filehook_gzio): New variable.

	* commands/cat.c (grub_cmd_cat): Use grub_file_open_ex instead of grub_gzfile_open.

	* commands/cmp.c (grub_cmd_cmp): Likewise.

	* commands/hexdump.c (grub_cmd_hexdump): Likewise.

	* kern/elf.c (grub_elf_open): Likewise.

	* loader/multiboot2.c (grub_multiboot2): Likewise.
	(grub_module2): Likewise.

	* loader/multiboot_loader.c (grub_rescue_cmd_multiboot_loader): Likewise.

	* loader/i386/pc/multiboot.c (grub_multiboot): Likewise.
	(grub_module): Likewise.
	
	* commands/ls.c (grub_ls_list_files): Use grub_file_open_ex instead of grub_file_open.

	* disk/loopback.c (grub_loopback_open): Likewise.

	* font/manager.c (add_font): Likewise.


diff --git a/commands/cat.c b/commands/cat.c
index 35b4c14..e4e4b4d 100644
--- a/commands/cat.c
+++ b/commands/cat.c
@@ -24,7 +24,6 @@
 #include <grub/disk.h>
 #include <grub/term.h>
 #include <grub/misc.h>
-#include <grub/gzio.h>
 
 static grub_err_t
 grub_cmd_cat (struct grub_arg_list *state __attribute__ ((unused)),
@@ -38,7 +37,7 @@ grub_cmd_cat (struct grub_arg_list *state __attribute__ ((unused)),
   if (argc != 1)
     return grub_error (GRUB_ERR_BAD_ARGUMENT, "file name required");
 
-  file = grub_gzfile_open (args[0], 1);
+  file = grub_file_open_ex (args[0], 1);
   if (! file)
     return 0;
   
diff --git a/commands/cmp.c b/commands/cmp.c
index bb4de60..71f71df 100644
--- a/commands/cmp.c
+++ b/commands/cmp.c
@@ -23,7 +23,6 @@
 #include <grub/misc.h>
 #include <grub/file.h>
 #include <grub/mm.h>
-#include <grub/gzio.h>
 
 #define BUFFER_SIZE 512
 
@@ -44,8 +43,8 @@ grub_cmd_cmp (struct grub_arg_list *state __attribute__ ((unused)),
   grub_printf ("Compare `%s' and `%s':\n", args[0],
 	       args[1]);
 
-  file1 = grub_gzfile_open (args[0], 1);
-  file2 = grub_gzfile_open (args[1], 1);
+  file1 = grub_file_open_ex (args[0], 1);
+  file2 = grub_file_open_ex (args[1], 1);
   if (! file1 || ! file2)
     goto cleanup;
 
diff --git a/commands/hexdump.c b/commands/hexdump.c
index d353d5e..de4355a 100644
--- a/commands/hexdump.c
+++ b/commands/hexdump.c
@@ -23,7 +23,6 @@
 #include <grub/file.h>
 #include <grub/disk.h>
 #include <grub/misc.h>
-#include <grub/gzio.h>
 #include <grub/hexdump.h>
 
 static const struct grub_arg_option options[] = {
@@ -100,7 +99,7 @@ grub_cmd_hexdump (struct grub_arg_list *state, int argc, char **args)
 
   if (is_file)
     {
-      file = grub_gzfile_open (args[0], 1);
+      file = grub_file_open_ex (args[0], 1);
       if (!file)
 	return 0;
 
diff --git a/commands/ls.c b/commands/ls.c
index e7b6302..9c3ca8c 100644
--- a/commands/ls.c
+++ b/commands/ls.c
@@ -97,7 +97,7 @@ grub_ls_list_files (char *dirname, int longlist, int all, int human)
 
 	  /* XXX: For ext2fs symlinks are detected as files while they
 	     should be reported as directories.  */
-	  file = grub_file_open (pathname);
+	  file = grub_file_open_ex (pathname, 1);
 	  if (! file)
 	    {
 	      grub_errno = 0;
diff --git a/disk/loopback.c b/disk/loopback.c
index 31d8116..b63bbdb 100644
--- a/disk/loopback.c
+++ b/disk/loopback.c
@@ -171,7 +171,7 @@ grub_loopback_open (const char *name, grub_disk_t disk)
   if (! dev)
     return grub_error (GRUB_ERR_UNKNOWN_DEVICE, "Can't open device");
 
-  file = grub_file_open (dev->filename);
+  file = grub_file_open_ex (dev->filename, 1);
   if (! file)
     return grub_errno;
   
diff --git a/font/manager.c b/font/manager.c
index 8d5f021..c196de5 100644
--- a/font/manager.c
+++ b/font/manager.c
@@ -48,7 +48,7 @@ add_font (const char *filename)
   grub_uint32_t num, i;
   struct font *font = 0;
 
-  file = grub_file_open (filename);
+  file = grub_file_open_ex (filename, 1);
   if (! file)
     goto fail;
 
diff --git a/include/grub/file.h b/include/grub/file.h
index df2e9e4..7d2e776 100644
--- a/include/grub/file.h
+++ b/include/grub/file.h
@@ -48,10 +48,21 @@ struct grub_file
 };
 typedef struct grub_file *grub_file_t;
 
+struct grub_filehook
+{
+  grub_file_t (*open_func) (grub_file_t file);
+  struct grub_filehook *next;
+};
+typedef struct grub_filehook *grub_filehook_t;
+
+void EXPORT_FUNC(grub_filehook_register) (grub_filehook_t filehook);
+void EXPORT_FUNC(grub_filehook_unregister) (grub_filehook_t filehook);
+
 /* Get a device name from NAME.  */
 char *EXPORT_FUNC(grub_file_get_device_name) (const char *name);
 
 grub_file_t EXPORT_FUNC(grub_file_open) (const char *name);
+grub_file_t EXPORT_FUNC(grub_file_open_ex) (const char *name, int flag);
 grub_ssize_t EXPORT_FUNC(grub_file_read) (grub_file_t file, char *buf,
 					  grub_size_t len);
 grub_off_t EXPORT_FUNC(grub_file_seek) (grub_file_t file, grub_off_t offset);
diff --git a/io/gzio.c b/io/gzio.c
index 0fada6c..e2aadcb 100644
--- a/io/gzio.c
+++ b/io/gzio.c
@@ -36,6 +36,7 @@
 
 #include <grub/err.h>
 #include <grub/types.h>
+#include <grub/dl.h>
 #include <grub/mm.h>
 #include <grub/misc.h>
 #include <grub/fs.h>
@@ -1241,3 +1242,25 @@ static struct grub_fs grub_gzio_fs =
     .label = 0,
     .next = 0
   };
+
+static grub_file_t grub_filehook_open(grub_file_t file)
+{
+  return grub_gzio_open (file, 1);
+}
+
+static struct grub_filehook grub_filehook_gzio =
+  {
+    .open_func = grub_filehook_open,
+    .next = 0
+  };
+
+GRUB_MOD_INIT(gzio)
+{
+  (void) mod;			/* To stop warning. */
+  grub_filehook_register(&grub_filehook_gzio);
+}
+
+GRUB_MOD_FINI(gzio)
+{
+  grub_filehook_unregister(&grub_filehook_gzio);
+}
diff --git a/kern/elf.c b/kern/elf.c
index b362949..3dd7ad4 100644
--- a/kern/elf.c
+++ b/kern/elf.c
@@ -21,7 +21,6 @@
 #include <grub/elf.h>
 #include <grub/elfload.h>
 #include <grub/file.h>
-#include <grub/gzio.h>
 #include <grub/misc.h>
 #include <grub/mm.h>
 
@@ -96,7 +95,7 @@ grub_elf_open (const char *name)
 {
   grub_file_t file;
 
-  file = grub_gzfile_open (name, 1);
+  file = grub_file_open_ex (name, 1);
   if (! file)
     return 0;
 
diff --git a/kern/file.c b/kern/file.c
index adf55da..04f1d5f 100644
--- a/kern/file.c
+++ b/kern/file.c
@@ -23,6 +23,29 @@
 #include <grub/mm.h>
 #include <grub/fs.h>
 #include <grub/device.h>
+#include <grub/env.h>
+
+static grub_filehook_t grub_filehook_list;
+
+void
+grub_filehook_register (grub_filehook_t filehook)
+{
+  filehook->next = grub_filehook_list;
+  grub_filehook_list = filehook;
+}
+
+void
+grub_filehook_unregister (grub_filehook_t filehook)
+{
+  grub_filehook_t *p, q;
+
+  for (p = &grub_filehook_list, q = *p; q; p = &(q->next), q = q->next)
+    if (q == filehook)
+      {
+	*p = q->next;
+	break;
+      }
+}
 
 /* Get the device part of the filename NAME. It is enclosed by parentheses.  */
 char *
@@ -110,6 +133,34 @@ grub_file_open (const char *name)
   return 0;
 }
 
+grub_file_t
+grub_file_open_ex (const char *name, int flag)
+{
+  grub_file_t file;
+  grub_filehook_t p;
+  char *val;
+
+  file = grub_file_open (name);
+
+  if ((! file) || (! flag))
+    return file;
+
+  val = grub_env_get ("filehook");
+  if ((val) && (val[0] == '0'))
+    return file;
+
+  for (p = grub_filehook_list; p; p=p->next)
+    {
+      grub_file_t new_file;
+
+      new_file = (*p->open_func) (file);
+      if (new_file)
+        return new_file;
+    }
+
+  return file;
+}
+
 grub_ssize_t
 grub_file_read (grub_file_t file, char *buf, grub_size_t len)
 {
diff --git a/loader/i386/pc/multiboot.c b/loader/i386/pc/multiboot.c
index 250ef47..c46b1a8 100644
--- a/loader/i386/pc/multiboot.c
+++ b/loader/i386/pc/multiboot.c
@@ -42,7 +42,6 @@
 #include <grub/dl.h>
 #include <grub/mm.h>
 #include <grub/misc.h>
-#include <grub/gzio.h>
 
 extern grub_dl_t my_mod;
 static struct grub_multiboot_info *mbi;
@@ -253,7 +252,7 @@ grub_multiboot (int argc, char *argv[])
       goto fail;
     }
 
-  file = grub_gzfile_open (argv[0], 1);
+  file = grub_file_open_ex (argv[0], 1);
   if (! file)
     {
       grub_error (GRUB_ERR_BAD_ARGUMENT, "Couldn't open file");
@@ -362,7 +361,7 @@ grub_module  (int argc, char *argv[])
       goto fail;
     }
 
-  file = grub_gzfile_open (argv[0], 1);
+  file = grub_file_open_ex (argv[0], 1);
   if (! file)
     goto fail;
 
diff --git a/loader/multiboot2.c b/loader/multiboot2.c
index 65fdea1..d58ea9d 100644
--- a/loader/multiboot2.c
+++ b/loader/multiboot2.c
@@ -28,7 +28,6 @@
 #include <grub/dl.h>
 #include <grub/mm.h>
 #include <grub/misc.h>
-#include <grub/gzio.h>
 
 static grub_addr_t entry;
 extern grub_dl_t my_mod;
@@ -323,7 +322,7 @@ grub_multiboot2 (int argc, char *argv[])
       goto fail;
     }
 
-  file = grub_gzfile_open (argv[0], 1);
+  file = grub_file_open_ex (argv[0], 1);
   if (! file)
     {
       grub_error (GRUB_ERR_BAD_ARGUMENT, "Couldn't open file");
@@ -423,7 +422,7 @@ grub_module2 (int argc, char *argv[])
     }
 
   /* Load module data.  */
-  file = grub_gzfile_open (argv[0], 1);
+  file = grub_file_open_ex (argv[0], 1);
   if (! file)
     goto out;
 
diff --git a/loader/multiboot_loader.c b/loader/multiboot_loader.c
index d6e86f1..9bf2987 100644
--- a/loader/multiboot_loader.c
+++ b/loader/multiboot_loader.c
@@ -28,7 +28,6 @@
 #include <grub/dl.h>
 #include <grub/mm.h>
 #include <grub/misc.h>
-#include <grub/gzio.h>
 
 grub_dl_t my_mod;
 
@@ -85,7 +84,7 @@ grub_rescue_cmd_multiboot_loader (int argc, char *argv[])
       goto fail;
     }
 
-  file = grub_gzfile_open (argv[0], 1);
+  file = grub_file_open_ex (argv[0], 1);
   if (! file)
     {
       grub_error (GRUB_ERR_BAD_ARGUMENT, "Couldn't open file");

Re: Transparent decompression with file system filter

[Yoshinori K. Okuji]

On Saturday 05 January 2008 07:30, Bean wrote:
> I'm thinking that compressed might not be the right word, as filter
> doesn't limit to decompression, it can support decryption or format
> conversion as well. I guess the name grub_file_open_ex may be better.
> I also add a flag to the function, if flag!=0, filters are used, and
> if flag=0, grub_file_open_ex is just like grub_file_open.

I feel that this is too theoretical. In reality, if you want to support, say, 
an encrypted file to be decrypted automatically, you need to supply a private 
key or a passphrase or something like that. How can this be unified and 
transparent?

Okuji

Re: Transparent decompression with file system filter

[Bean]

On Jan 5, 2008 6:39 PM, Yoshinori K. Okuji <okuji@enbug.org> wrote:
> On Saturday 05 January 2008 07:30, Bean wrote:
> > I'm thinking that compressed might not be the right word, as filter
> > doesn't limit to decompression, it can support decryption or format
> > conversion as well. I guess the name grub_file_open_ex may be better.
> > I also add a flag to the function, if flag!=0, filters are used, and
> > if flag=0, grub_file_open_ex is just like grub_file_open.
>
> I feel that this is too theoretical. In reality, if you want to support, say,
> an encrypted file to be decrypted automatically, you need to supply a private
> key or a passphrase or something like that. How can this be unified and
> transparent?

the filter can export a command/function to set key, then it can use
it to decrypted files. if the password is file based, it can prompt
the user  before opening the file.

-- 
Bean

Re: Transparent decompression with file system filter

[Bean]

Hi,

any suggestion about this patch ? i would like to commit it soon.

-- 
Bean

Re: Transparent decompression with file system filter

[Yoshinori K. Okuji]

On Sunday 13 January 2008 08:38, Bean wrote:
> Hi,
>
> any suggestion about this patch ? i would like to commit it soon.

I am sorry that I forgot to reply.

I would like to know how you intend to approach the following issues:

- If you want to support, for example, an ecrypted and compressed file, in the 
current implementation, each hook must try other hooks recursively. I believe 
that this should be done at grub_file_open_ex rather than at every hook. Is 
there any pitfall with this way?

- There are some cases where the user wants to skip some decoding features 
(i.e. decryptions and decompressions). For instance, gunzip does not make 
sense for initrd, since the linux kernel performs this. But if the user uses 
other compression algorithms (e.g. LZMA), GRUB must decompress it before 
transferring control to the kernel. Or, in the case of Multiboot modules, an 
OS image might want to keep compressed modules as they are, but have GRUB to 
decrypt them, if they are encrypted. How does the user select which hooks 
should be applied (from the viewpoint of UI)?

Because your aim is quite broad (which is a good thing), I think we need to 
consider possible use cases carefully, to make sure that the API is good 
enough.

Okuji

Re: Transparent decompression with file system filter

[Bean]

On Jan 14, 2008 3:51 AM, Yoshinori K. Okuji <okuji@enbug.org> wrote:
> On Sunday 13 January 2008 08:38, Bean wrote:
> > Hi,
> >
> > any suggestion about this patch ? i would like to commit it soon.
>
> I am sorry that I forgot to reply.
>
> I would like to know how you intend to approach the following issues:
>
> - If you want to support, for example, an ecrypted and compressed file, in the
> current implementation, each hook must try other hooks recursively. I believe
> that this should be done at grub_file_open_ex rather than at every hook. Is
> there any pitfall with this way?

currently, at most one hook is applied to one file, recursive hook can
be messy. but we can add a priority value, this would allow the hooks
to be apply in a particular order.

>
> - There are some cases where the user wants to skip some decoding features
> (i.e. decryptions and decompressions). For instance, gunzip does not make
> sense for initrd, since the linux kernel performs this. But if the user uses
> other compression algorithms (e.g. LZMA), GRUB must decompress it before
> transferring control to the kernel. Or, in the case of Multiboot modules, an
> OS image might want to keep compressed modules as they are, but have GRUB to
> decrypt them, if they are encrypted. How does the user select which hooks
> should be applied (from the viewpoint of UI)?

i think this can be controlled by variables, for example, we can use
filehook.gzip to control whether or not to use gzip, etc.

-- 
Bean


-- 
Bean

Re: Transparent decompression with file system filter

[Yoshinori K. Okuji]

On Sunday 13 January 2008 21:16, Bean wrote:
> > - If you want to support, for example, an ecrypted and compressed file,
> > in the current implementation, each hook must try other hooks
> > recursively. I believe that this should be done at grub_file_open_ex
> > rather than at every hook. Is there any pitfall with this way?
>
> currently, at most one hook is applied to one file, recursive hook can
> be messy. but we can add a priority value, this would allow the hooks
> to be apply in a particular order.

In reality, the user can do both encryption+compression and 
compression+encryption, thus I don't think we can put priorities a priori.

> > - There are some cases where the user wants to skip some decoding
> > features (i.e. decryptions and decompressions). For instance, gunzip does
> > not make sense for initrd, since the linux kernel performs this. But if
> > the user uses other compression algorithms (e.g. LZMA), GRUB must
> > decompress it before transferring control to the kernel. Or, in the case
> > of Multiboot modules, an OS image might want to keep compressed modules
> > as they are, but have GRUB to decrypt them, if they are encrypted. How
> > does the user select which hooks should be applied (from the viewpoint of
> > UI)?
>
> i think this can be controlled by variables, for example, we can use
> filehook.gzip to control whether or not to use gzip, etc.

Using variables has too much side effect. For example:

grub> set something=foo,bar
grub> command

Since the value of the variable is applied to all files the command opens, it 
can lead to undesired results. Even if it is a rare case that a command wants 
to open multiple files differently, I don't think this is really clean.

The traditional approach in GRUB is to specify options to a command. For 
example, --no-decompression. Then, the command hardcodes where the options 
are applied. As long as the combinations of such options fulfill all 
(realistic) use cases, this approach works quite well.

But it is a nightmare to specify all possibilities, such 
as --no-gunzip, --no-bunzip2, and so on. So my feeling is that we need a kind 
of categorization in hooks. For example, gzio belongs to "compression". Once 
this is done, I think it would be good enough for Multiboot.

For other boot protocols (such as Linux), I think loaders would require 
hardcoding more specific filtering (such as excluding only gzip). If hooks 
are named, we might be able to use the same trick as for grub_dprintf.

Okuji