[PATCH] add pkzip support for gzio

[PATCH] add pkzip support for gzio

[y.volta]

[-- Attachment #1: Type: text/plain, Size: 323 bytes --]

hi,

    i've made a little patch for pkziped ( deflate mode ) file. by using this, we can have both gunzip and pkunzip supported: they have only header different with the same file. ;-)

    please note, if your file is too small, pkzip will not compress it but store the orginal data to the output file.

y.volta
 

[-- Attachment #2: gzio.patch --]
[-- Type: application/octet-stream, Size: 3784 bytes --]

Index: gzio.c
===================================================================
--- gzio.c	(revision 1800)
+++ gzio.c	(working copy)
@@ -161,9 +161,81 @@
 typedef unsigned short ush;
 typedef unsigned long ulg;
 
+
+/* PKZIP header.  See
+ * <http://www.pkware.com/products/enterprise/white_papers/appnote.html>.
+ */
+struct pkzip_header {
+  grub_int32_t magic;
+  grub_int16_t version;
+  grub_int16_t flags;
+  grub_int16_t method;
+  grub_int16_t modified_time;
+  grub_int16_t modified_date;
+  grub_int32_t crc;
+  grub_int32_t zbytes;
+  grub_int32_t dbytes;
+  grub_int16_t filename_len;
+  grub_int16_t extra_len;
+} __attribute__ ((packed));
+/* (followed by optional and variable length "filename" and "extra"
+   fields) */
+
+/* pkzip flag byte */
+#define PK_ENCRYPTED        0x01    /* bit 0 set: file is encrypted */
+#define PK_DATADESC         0x08    /* bit 3 set: file has trailing "data
+                                        descriptor" */
+#define PK_UNSUPPORTED      0xFFF0  /* All other bits must be zero */
+#define PKZIP_MAGIC	grub_le_to_cpu32 (0x04034B50)
+   
+/* test for pkzip file header */
 static int
-test_header (grub_file_t file)
+pkunzip_test_header (grub_file_t file)
 {
+  struct pkzip_header  pkzh;
+  int pkzip_header_size = sizeof(struct pkzip_header);
+  
+  grub_gzio_t gzio = file->data;
+  
+  if (grub_file_tell (gzio->file) != 0)
+    grub_file_seek (gzio->file, 0);
+      
+  /* read header */
+  if ( grub_file_read (gzio->file, (char *)&pkzh, pkzip_header_size) != pkzip_header_size 
+        || pkzh.magic != PKZIP_MAGIC)
+    {
+      grub_error (GRUB_ERR_BAD_FILE_TYPE, "no pkzip magic found");
+      return 0;
+    }
+    
+  if (pkzh.flags & PK_ENCRYPTED        /* error("pkzip file is encrypted; not supported"); */
+    || pkzh.flags & PK_DATADESC        /* error("pkzip file uses data_descriptor field; not supported"); */
+    || pkzh.flags & PK_UNSUPPORTED     /* error("pkzip file has unsupported flags"); */
+    || pkzh.method != 8)               /* error("pkzip file uses invalid method"); */
+                                       /* We only support method #8, DEFLATED!! */
+    {
+      grub_error (GRUB_ERR_BAD_FILE_TYPE, "no pkzip magic found");
+      return 0;
+    }
+   
+  gzio->data_offset = pkzip_header_size;          /* skip header */
+    
+  gzio->data_offset += pkzh.filename_len;         /* skip filename */
+    
+  gzio->data_offset += pkzh.extra_len;            /* skip extra field */
+
+  /* gzip_crc = pkzh.crc;                            crc */
+  file->size = pkzh.dbytes;                       /* decompressed size */
+
+  initialize_tables (file);
+  
+  return 1;
+}
+
+/* test for gzip file header */
+static int
+gunzip_test_header (grub_file_t file)
+{
   unsigned char buf[10] __attribute__ ((aligned));
   grub_gzio_t gzio = file->data;
 
@@ -1128,19 +1200,25 @@
   file->read_hook = 0;
   file->fs = &grub_gzio_fs;
 
-  if (! test_header (file))
+  if (! gunzip_test_header (file))
     {
-      grub_free (gzio);
-      grub_free (file);
-      grub_file_seek (io, 0);
+      /* before we go, reset the error num. */
+      grub_errno = GRUB_ERR_NONE;
+      
+      if (! pkunzip_test_header (file))
+        {
+          grub_free (gzio);
+          grub_free (file);
+          grub_file_seek (io, 0);
 
-      if (grub_errno == GRUB_ERR_BAD_FILE_TYPE && transparent)
-	{
-	  grub_errno = GRUB_ERR_NONE;
-	  return io;
+          if (grub_errno == GRUB_ERR_BAD_FILE_TYPE && transparent)
+	    {
+	      grub_errno = GRUB_ERR_NONE;
+    	      return io;
+	    }
+          else
+	    return 0;
 	}
-      else
-	return 0;
     }
   
   return file;

Re: [PATCH] add pkzip support for gzio

[Colin D Bennett]

On Mon, 11 Aug 2008 23:41:38 +0800
"y.volta" <y.volta@gmail.com> wrote:

> hi,
> 
>     i've made a little patch for pkziped ( deflate mode ) file. by
> using this, we can have both gunzip and pkunzip supported: they have
> only header different with the same file. ;-)
> 
>     please note, if your file is too small, pkzip will not compress
> it but store the orginal data to the output file.
> 
> y.volta

What happens if you open a ZIP file containing multiple files?

Regards,
Colin

Re: [PATCH] add pkzip support for gzio

[Marco Gerards]

Hi,

"y.volta" <y.volta@gmail.com> writes:

>     i've made a little patch for pkziped ( deflate mode ) file. by
> using this, we can have both gunzip and pkunzip supported: they have
> only header different with the same file. ;-)
>
>     please note, if your file is too small, pkzip will not compress it
> but store the orginal data to the output file.

Isn't this patented?

--
Marco

Re: [PATCH] add pkzip support for gzio

[Gregg Levine]

On Mon, Aug 11, 2008 at 1:22 PM, Marco Gerards <mgerards@xs4all.nl> wrote:
> Hi,
>
> "y.volta" <y.volta@gmail.com> writes:
>
>>     i've made a little patch for pkziped ( deflate mode ) file. by
>> using this, we can have both gunzip and pkunzip supported: they have
>> only header different with the same file. ;-)
>>
>>     please note, if your file is too small, pkzip will not compress it
>> but store the orginal data to the output file.
>
> Isn't this patented?
>
> --
> Marco
>
Hello!
Correct.
The name Pkzip is indeed copyrighted, by the firm who created the
original tools.

However the zip and unzip methods aren't and are freely available from
where ever the Info-Zip people live online.

-----
Gregg C Levine gregg.drwho8@gmail.com
"This signature was once found posting rude
 messages in English in the Moscow subway."

Re: [PATCH] add pkzip support for gzio

[Marco Gerards]

"Gregg Levine" <gregg.drwho8@gmail.com> writes:

> On Mon, Aug 11, 2008 at 1:22 PM, Marco Gerards <mgerards@xs4all.nl> wrote:
>> Hi,
>>
>> "y.volta" <y.volta@gmail.com> writes:
>>
>>>     i've made a little patch for pkziped ( deflate mode ) file. by
>>> using this, we can have both gunzip and pkunzip supported: they have
>>> only header different with the same file. ;-)
>>>
>>>     please note, if your file is too small, pkzip will not compress it
>>> but store the orginal data to the output file.
>>
>> Isn't this patented?
>>
>> --
>> Marco
>>
> Hello!
> Correct.
> The name Pkzip is indeed copyrighted, by the firm who created the
> original tools.

You are confusing copyright and patents, they are a different thing
:-)

But if people say it is no longer patented, zip is fine for me.

--
Marco

RE: [PATCH] add pkzip support for gzio

[Gregg C Levine]

Hello!
Perhaps.
But then again perhaps the crew from PKWare might also have been confusing
the issue. That's what comes from the problem of presenting software via a
clone who only knows how to present things, not properly explain them.

I do know that we should call the module simply zip and support the basic
methods behind how the zip and unzip programs work on most Linux
distributions.
--
Gregg C Levine hansolofalcon@worldnet.att.net
"The Force will be with you always." Obi-Wan Kenobi
  


> -----Original Message-----
> From: grub-devel-bounces+hansolofalcon=worldnet.att.net@gnu.org
[mailto:grub-devel-
> bounces+hansolofalcon=worldnet.att.net@gnu.org] On Behalf Of Marco Gerards
> Sent: Wednesday, August 13, 2008 5:54 AM
> To: The development of GRUB 2
> Subject: Re: [PATCH] add pkzip support for gzio
> 
> "Gregg Levine" <gregg.drwho8@gmail.com> writes:
> 
> > On Mon, Aug 11, 2008 at 1:22 PM, Marco Gerards <mgerards@xs4all.nl>
wrote:
> >> Hi,
> >>
> >> "y.volta" <y.volta@gmail.com> writes:
> >>
> >>>     i've made a little patch for pkziped ( deflate mode ) file. by
> >>> using this, we can have both gunzip and pkunzip supported: they have
> >>> only header different with the same file. ;-)
> >>>
> >>>     please note, if your file is too small, pkzip will not compress it
> >>> but store the orginal data to the output file.
> >>
> >> Isn't this patented?
> >>
> >> --
> >> Marco
> >>
> > Hello!
> > Correct.
> > The name Pkzip is indeed copyrighted, by the firm who created the
> > original tools.
> 
> You are confusing copyright and patents, they are a different thing
> :-)
> 
> But if people say it is no longer patented, zip is fine for me.
> 
> --
> Marco
> 
> 
> 
> _______________________________________________
> Grub-devel mailing list
> Grub-devel@gnu.org
> http://lists.gnu.org/mailman/listinfo/grub-devel

Re: [PATCH] add pkzip support for gzio

[Robert Millan]

On Mon, Aug 11, 2008 at 07:22:53PM +0200, Marco Gerards wrote:
> Hi,
> 
> "y.volta" <y.volta@gmail.com> writes:
> 
> >     i've made a little patch for pkziped ( deflate mode ) file. by
> > using this, we can have both gunzip and pkunzip supported: they have
> > only header different with the same file. ;-)
> >
> >     please note, if your file is too small, pkzip will not compress it
> > but store the orginal data to the output file.
> 
> Isn't this patented?

According to Wikipedia [1], the usual compression algorithm in ZIPs is the
deflate, same as gzip.  I believe older variants use the LZW algorithm, but
the patents [2] for these have expired (at least in the US), and we happen
to be using it already in our LZMA (and LZO?) code.

[1] http://en.wikipedia.org/wiki/ZIP_(file_format)
[2] http://en.wikipedia.org/wiki/LZW#Patents

-- 
Robert Millan

  The DRM opt-in fallacy: "Your data belongs to us. We will decide when (and
  how) you may access your data; but nobody's threatening your freedom: we
  still allow you to remove your data and not access it at all."

Re: [PATCH] add pkzip support for gzio

[Robert Millan]

On Mon, Aug 11, 2008 at 11:41:38PM +0800, y.volta wrote:
> hi,
> 
>     i've made a little patch for pkziped ( deflate mode ) file. by using this, we can have both gunzip and pkunzip supported: they have only header different with the same file. ;-)
> 
>     please note, if your file is too small, pkzip will not compress it but store the orginal data to the output file.

Can this be a separate module instead?

-- 
Robert Millan

  The DRM opt-in fallacy: "Your data belongs to us. We will decide when (and
  how) you may access your data; but nobody's threatening your freedom: we
  still allow you to remove your data and not access it at all."

Re: [PATCH] add pkzip support for gzio

[y.volta]

i just conveniently make this kind of archive be supported in grub2 - since
there is no other addtional work required.

this is a little patch, so if there are mutli-files in this archive, we will
failed to decompress it.

As I know, there are many GUI tools(e.g. 7-Zip/Winrar/Winzip) can make the
deflated zip file, so, this patch should be useful for those GUI users.

Re: [PATCH] add pkzip support for gzio

[Bean]

On Tue, Aug 12, 2008 at 10:02 AM, y.volta <y.volta@gmail.com> wrote:
> i just conveniently make this kind of archive be supported in grub2 - since
> there is no other addtional work required.
>
> this is a little patch, so if there are mutli-files in this archive, we will
> failed to decompress it.
>
> As I know, there are many GUI tools(e.g. 7-Zip/Winrar/Winzip) can make the
> deflated zip file, so, this patch should be useful for those GUI users.

Hi,

As zip support multiple files, it's better to use zipfs, just like cpio/tar.

Also, it's better not to call gzio. I'm thinking about seperating the
deflate decompression routine and create a zlib like library, so that
the decoder can be used in both gzio, png and zipfs.

-- 
Bean

Re: [PATCH] add pkzip support for gzio

[Colin D Bennett]

On Tue, 12 Aug 2008 10:45:50 +0800
Bean <bean123ch@gmail.com> wrote:

> Also, it's better not to call gzio. I'm thinking about seperating the
> deflate decompression routine and create a zlib like library, so that
> the decoder can be used in both gzio, png and zipfs.

Also, pure DEFLATE compression can be used to compress font data in the
new font file format.  I postponed implementing font compression to
wait for DEFLATE support.

Regards,
Colin

Re: [PATCH] add pkzip support for gzio

[Bean]

On Wed, Aug 13, 2008 at 3:07 AM, Colin D Bennett <colin@gibibit.com> wrote:
> On Tue, 12 Aug 2008 10:45:50 +0800
> Bean <bean123ch@gmail.com> wrote:
>
>> Also, it's better not to call gzio. I'm thinking about seperating the
>> deflate decompression routine and create a zlib like library, so that
>> the decoder can be used in both gzio, png and zipfs.
>
> Also, pure DEFLATE compression can be used to compress font data in the
> new font file format.  I postponed implementing font compression to
> wait for DEFLATE support.

Hi Colin,

Actually, I think you can use lzma. The encoder and decoder are both
inside /lib now, there is no need to use external library. Besides,
LZMA is much better than DEFLATE in compression ratio.

-- 
Bean