Update for NTFS file system driver

Update for NTFS file system driver

[Bean]

[-- Attachment #1: Type: text/plain, Size: 127 bytes --]

Add unicode support for NTFS file system driver
Fix a bug in the unicode conversion function grub_utf16_to_utf8 in kern/misc.c

[-- Attachment #2: ntfs.c.gz --]
[-- Type: application/octet-stream, Size: 8306 bytes --]

[-- Attachment #3: utf.patch.gz --]
[-- Type: application/octet-stream, Size: 187 bytes --]

Re: Update for NTFS file system driver

[Marco Gerards]

Bean <bean123@126.com> writes:

> Add unicode support for NTFS file system driver
> Fix a bug in the unicode conversion function grub_utf16_to_utf8 in kern/misc.c

Can you please include the patches inline?  That way they are easier
to review.

The normal procedure for patches is to make them consistent with the
GCS (GNU Coding Standards), include a changelog.  That way it is
easier to review the patch and to include it in GRUB 2.

Did you write all the code yourself or did you use code from another
project, like Linux or grub4dos or so?

--
Marco

Re: Update for NTFS file system driver

[Bean]

On Sat, Jun 09, 2007 at 08:05:45PM +0200, Marco Gerards wrote:
>
> The normal procedure for patches is to make them consistent with the
> GCS (GNU Coding Standards), include a changelog.  That way it is
> easier to review the patch and to include it in GRUB 2.
>

Can you give me an example ?

>
> Did you write all the code yourself or did you use code from another
> project, like Linux or grub4dos or so?
>

Yes, I write the code myself, which is based on the NTFS driver fsys_ntfs.c in grub4dos.

-- 
Bean <bean123@126.com>

Re: Update for NTFS file system driver

[Marco Gerards]

Bean <bean123@126.com> writes:

> On Sat, Jun 09, 2007 at 08:05:45PM +0200, Marco Gerards wrote:
>>
>> The normal procedure for patches is to make them consistent with the
>> GCS (GNU Coding Standards), include a changelog.  That way it is
>> easier to review the patch and to include it in GRUB 2.
>
> Can you give me an example ?

Here is an example of a patch I sent in recently:
http://lists.gnu.org/archive/html/grub-devel/2007-05/msg00114.html

The GNU Coding Standards:
http://www.gnu.org/prep/standards/standards.html

The specific part on ChangeLog entries:
http://www.gnu.org/prep/standards/standards.html#Change-Logs

The biggest part is on indentation, etc.

--
Marco

Re: Update for NTFS file system driver

[Thomas Schwinge]

[-- Attachment #1: Type: text/plain, Size: 1075 bytes --]

Hello!

On Sun, Jun 10, 2007 at 09:57:44AM +0200, Marco Gerards wrote:
> Bean <bean123@126.com> writes:
> > On Sat, Jun 09, 2007 at 08:05:45PM +0200, Marco Gerards wrote:
> >> The normal procedure for patches is to make them consistent with the
> >> GCS (GNU Coding Standards), include a changelog.  That way it is
> >> easier to review the patch and to include it in GRUB 2.
> 
> The GNU Coding Standards:
> http://www.gnu.org/prep/standards/standards.html
> 
> The specific part on ChangeLog entries:
> http://www.gnu.org/prep/standards/standards.html#Change-Logs
> 
> The biggest part is on indentation, etc.

Sorry, but that is also the part which the upstream committers (e.g. you,
or me, or some more others) can trivially fix if you don't like it the
way the first-time contributor has done.  That's my idea of this eternal
``your-formatting-only-does-stop-us-from-committing-your-patch'' thing.
And that is also what I would expect if I were a first-time contributor.
Otherwise I might decide to spend my time elsewhere.


Regards,
 Thomas

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 191 bytes --]

Re: Update for NTFS file system driver

[Marco Gerards]

Thomas Schwinge <tschwinge@gnu.org> writes:

> Hello!
>
> On Sun, Jun 10, 2007 at 09:57:44AM +0200, Marco Gerards wrote:
>> Bean <bean123@126.com> writes:
>> > On Sat, Jun 09, 2007 at 08:05:45PM +0200, Marco Gerards wrote:
>> >> The normal procedure for patches is to make them consistent with the
>> >> GCS (GNU Coding Standards), include a changelog.  That way it is
>> >> easier to review the patch and to include it in GRUB 2.
>> 
>> The GNU Coding Standards:
>> http://www.gnu.org/prep/standards/standards.html
>> 
>> The specific part on ChangeLog entries:
>> http://www.gnu.org/prep/standards/standards.html#Change-Logs
>> 
>> The biggest part is on indentation, etc.
>
> Sorry, but that is also the part which the upstream committers (e.g. you,
> or me, or some more others) can trivially fix if you don't like it the
> way the first-time contributor has done.  That's my idea of this eternal
> ``your-formatting-only-does-stop-us-from-committing-your-patch'' thing.
> And that is also what I would expect if I were a first-time contributor.
> Otherwise I might decide to spend my time elsewhere.

The problem is, as you may know, that we have very little time to
clean up all the patches.  But I am certainly willing to help people
wherever possible with these kind of tasks.  Besides that, for my
feeling the coding style is part of the job.

But if you do have the time, please help us out!

--
Marco

history of fsys_ntfs.c (Re: Update for NTFS file system driver)

[Robert Millan]

On Sun, Jun 10, 2007 at 11:42:53AM +0800, Bean wrote:
> > Did you write all the code yourself or did you use code from another
> > project, like Linux or grub4dos or so?
> >
> 
> Yes, I write the code myself, which is based on the NTFS driver fsys_ntfs.c in grub4dos.

I had a look at that file, trying to trace its origin.  As for the file header,
it seems that Samuel Leo <samuel@szonline.net> wrote it.  GRUB maintainers, do
we need to contact Samuel about this, or can we take the copyright as ok now ?

His patch (for GRUB Legacy) was sent to bug-grub in 2003:

  http://lists.gnu.org/archive/html/bug-grub/2003-04/msg00032.html

It was also discussed on 2005:

  http://lists.gnu.org/archive/html/bug-grub/2005-05/msg00046.html

There are also a few bugs in Savannah:

  http://savannah.gnu.org/bugs/?4149
  http://savannah.gnu.org/bugs/?9180

Some contain improvements/fixes that I guess they have made into grub4dos
already, hence present in the patch Bean sent.

-- 
Robert Millan

My spam trap is honeypot@aybabtu.com.  Note: this address is only intended
for spam harvesters.  Writing to it will get you added to my black list.

Re: history of fsys_ntfs.c (Re: Update for NTFS file system driver)

[Bean]

On Sun, Jun 10, 2007 at 12:18:47PM +0200, Robert Millan wrote:
> I had a look at that file, trying to trace its origin.  As for the file header,
> it seems that Samuel Leo <samuel@szonline.net> wrote it.  GRUB maintainers, do
> we need to contact Samuel about this, or can we take the copyright as ok now ?

This NTFS driver is different from the Samuel's version. Ago from the wubi project
report that the grub4dos NTFS driver sometimes fails to find files, I try to fix 
the problem, but is not successfully. Finally, I decide to write the code based on
my NTFS boot sector for grub4dos. Therefore, this new driver is not a patch, but a 
complete rewrite.

-- 
Bean <bean123@126.com>

Re: Update for NTFS file system driver

[Bean]

Ok, I reformat the patch, hopefully it will conform to standard.

-- 
Bean <bean123@126.com>

2007-06-10  Bean  <bean123@126.com>

	* conf/common.rmk (pkgdata_MODULES): Add ntfs.mod.

	* fs/ntfs.c: New file.

	* kern/misc.c (grub_utf16_to_utf8): Fix unicode
	conversion bug.


Index: conf/common.rmk
===================================================================
RCS file: /sources/grub/grub2/conf/common.rmk,v
retrieving revision 1.14
diff -u -r1.14 common.rmk
--- conf/common.rmk	20 May 2007 09:10:06 -0000	1.14
+++ conf/common.rmk	10 Jun 2007 11:35:23 -0000
@@ -55,7 +55,7 @@
 # Filing systems.
 pkgdata_MODULES += fshelp.mod fat.mod ufs.mod ext2.mod		\
 	minix.mod hfs.mod jfs.mod iso9660.mod xfs.mod affs.mod	\
-	sfs.mod hfsplus.mod
+	sfs.mod hfsplus.mod ntfs.mod
 
 # For fshelp.mod.
 fshelp_mod_SOURCES = fs/fshelp.c
@@ -117,6 +117,11 @@
 hfsplus_mod_CFLAGS = $(COMMON_CFLAGS)
 hfsplus_mod_LDFLAGS = $(COMMON_LDFLAGS)
 
+# For ntfs.mod.
+ntfs_mod_SOURCES = fs/ntfs.c
+ntfs_mod_CFLAGS = $(COMMON_CFLAGS)
+ntfs_mod_LDFLAGS = $(COMMON_LDFLAGS)
+
 # Partition maps.
 pkgdata_MODULES += amiga.mod apple.mod pc.mod sun.mod acorn.mod gpt.mod
 
Index: kern/misc.c
===================================================================
RCS file: /sources/grub/grub2/kern/misc.c,v
retrieving revision 1.34
diff -u -r1.34 misc.c
--- kern/misc.c	17 May 2007 15:43:32 -0000	1.34
+++ kern/misc.c	10 Jun 2007 11:35:24 -0000
@@ -961,8 +961,8 @@
 	    }
 	  else
 	    {
-	      *dest++ = (code >> 16) | 0xE0;
-	      *dest++ = ((code >> 12) & 0x3F) | 0x80;
+	      *dest++ = (code >> 12) | 0xE0;
+	      *dest++ = ((code >> 6) & 0x3F) | 0x80;
 	      *dest++ = (code & 0x3F) | 0x80;
 	    }
 	}
Index: fs/ntfs.c
===================================================================
RCS file: /sources/grub/grub2/fs/ntfs.c,v
diff -Nu fs/ntfs.c
--- /dev/null	2007-01-31 15:03:12.000000000 +0800
+++ fs/ntfs.c	2007-06-10 19:44:02.000000000 +0800
@@ -0,0 +1,1553 @@
+/* ntfs.c - NTFS filesystem */
+/*
+ *  GRUB  --  GRand Unified Bootloader
+ *  Copyright (C) 2000,2001,2002,2003,2004,2005,2007  Free Software Foundation, Inc.
+ *
+ *  This program is free software; you can redistribute it and/or modify
+ *  it under the terms of the GNU General Public License as published by
+ *  the Free Software Foundation; either version 2 of the License, or
+ *  (at your option) any later version.
+ *
+ *  This program is distributed in the hope that it will be useful,
+ *  but WITHOUT ANY WARRANTY; without even the implied warranty of
+ *  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ *  GNU General Public License for more details.
+ *
+ *  You should have received a copy of the GNU General Public License
+ *  along with this program; if not, write to the Free Software
+ *  Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA.
+ */
+
+#include <grub/fs.h>
+#include <grub/disk.h>
+#include <grub/file.h>
+#include <grub/types.h>
+#include <grub/misc.h>
+#include <grub/mm.h>
+#include <grub/err.h>
+#include <grub/dl.h>
+
+#define FILE_MFT      0
+#define FILE_MFTMIRR  1
+#define FILE_LOGFILE  2
+#define FILE_VOLUME   3
+#define FILE_ATTRDEF  4
+#define FILE_ROOT     5
+#define FILE_BITMAP   6
+#define FILE_BOOT     7
+#define FILE_BADCLUS  8
+#define FILE_QUOTA    9
+#define FILE_UPCASE  10
+
+#define AT_STANDARD_INFORMATION	0x10
+#define AT_ATTRIBUTE_LIST	0x20
+#define AT_FILENAME		0x30
+#define AT_OBJECT_ID		0x40
+#define AT_SECURITY_DESCRIPTOR	0x50
+#define AT_VOLUME_NAME		0x60
+#define AT_VOLUME_INFORMATION	0x70
+#define AT_DATA			0x80
+#define AT_INDEX_ROOT		0x90
+#define AT_INDEX_ALLOCATION	0xA0
+#define AT_BITMAP		0xB0
+#define AT_SYMLINK		0xC0
+#define AT_EA_INFORMATION	0xD0
+#define AT_EA			0xE0
+
+#define ATTR_READ_ONLY		0x1
+#define ATTR_HIDDEN		0x2
+#define ATTR_SYSTEM		0x4
+#define ATTR_ARCHIVE		0x20
+#define ATTR_DEVICE		0x40
+#define ATTR_NORMAL		0x80
+#define ATTR_TEMPORARY		0x100
+#define ATTR_SPARSE		0x200
+#define ATTR_REPARSE		0x400
+#define ATTR_COMPRESSED		0x800
+#define ATTR_OFFLINE		0x1000
+#define ATTR_NOT_INDEXED	0x2000
+#define ATTR_ENCRYPTED		0x4000
+#define ATTR_DIRECTORY		0x10000000
+#define ATTR_INDEX_VIEW		0x20000000
+
+#define FLAG_COMPRESSED		1
+#define FLAG_ENCRYPTED		0x4000
+#define FLAG_SPARSE		0x8000
+
+#define BLK_SHR		9
+
+#define MAX_MFT		(1024 >> BLK_SHR)
+#define MAX_IDX		(16384 >> BLK_SHR)
+#define MAX_SPC		(4096 >> BLK_SHR)
+
+#define AF_ALST		1
+#define AF_MMFT		2
+#define AF_GPOS		4
+
+#define RF_COMP		1
+#define RF_CBLK		2
+#define RF_BLNK		4
+
+#define valueat(buf,ofs,type)	*((type*)(((char*)buf)+ofs))
+
+#define u16at(buf,ofs)	grub_le_to_cpu16(valueat(buf,ofs,unsigned short))
+#define u32at(buf,ofs)	grub_le_to_cpu32(valueat(buf,ofs,unsigned long))
+#define u64at(buf,ofs)	grub_le_to_cpu64(valueat(buf,ofs,unsigned long long))
+
+#define v16at(buf,ofs)	valueat(buf,ofs,unsigned short)
+#define v32at(buf,ofs)	valueat(buf,ofs,unsigned long)
+#define v64at(buf,ofs)	valueat(buf,ofs,unsigned long long)
+
+struct grub_ntfs_bpb
+{
+  grub_uint8_t jmp_boot[3];
+  grub_uint8_t oem_name[8];
+  grub_uint16_t bytes_per_sector;
+  grub_uint8_t sectors_per_cluster;
+  grub_uint8_t reserved_1[7];
+  grub_uint8_t media;
+  grub_uint16_t reserved_2;
+  grub_uint16_t sectors_per_track;
+  grub_uint16_t num_heads;
+  grub_uint32_t num_hidden_sectors;
+  grub_uint32_t reserved_3[2];
+  grub_uint64_t num_total_sectors;
+  grub_uint64_t mft_lcn;
+  grub_uint64_t mft_mirr_lcn;
+  grub_int8_t clusters_per_mft;
+  grub_int8_t reserved_4[3];
+  grub_int8_t clusters_per_index;
+  grub_int8_t reserved_5[3];
+  grub_uint64_t serial_number;
+  grub_uint32_t checksum;
+} __attribute__ ((packed));
+
+struct grub_ntfs_attr
+{
+  int flags;
+  char *emft_buf, *edat_buf;
+  char *attr_cur, *attr_nxt, *attr_end;
+  unsigned long save_pos;
+  char *sbuf;
+  struct grub_ntfs_file *mft;
+};
+
+struct grub_ntfs_file
+{
+  char *buf;
+  unsigned long filemax;
+  struct grub_ntfs_glob *glob;
+  struct grub_ntfs_attr attr;
+  void (*read_hook) (grub_disk_addr_t sector, unsigned offset,
+		     unsigned length);
+};
+
+struct grub_ntfs_glob
+{
+  grub_uint32_t mft_size;
+  grub_uint32_t idx_size;
+  grub_uint32_t spc;
+  grub_uint32_t blocksize;
+  grub_uint32_t mft_start;
+  struct grub_ntfs_file mmft;
+  grub_disk_t disk;
+};
+
+struct grub_ntfs_comp
+{
+  int comp_head, comp_tail;
+  unsigned long comp_table[16][2];
+  unsigned long cbuf_ofs, cbuf_vcn, spc;
+  char *cbuf;
+  grub_disk_t disk;
+};
+
+struct grub_ntfs_rlst
+{
+  int flags;
+  unsigned long target_vcn, curr_vcn, next_vcn, curr_lcn;
+  unsigned long vcn_offset;
+  char *cur_run;
+  struct grub_ntfs_attr *attr;
+  struct grub_ntfs_comp comp;
+};
+
+
+#ifndef GRUB_UTIL
+static grub_dl_t my_mod;
+#endif
+
+static int
+fixup (struct grub_ntfs_glob *glob, char *buf, int len, char *magic)
+{
+  int ss;
+  char *pu;
+  unsigned us;
+
+  if (grub_memcmp (buf, magic, 4))
+    {
+      grub_error (GRUB_ERR_BAD_FS, "%s label not found", magic);
+      return 0;
+    }
+
+  ss = u16at (buf, 6) - 1;
+  if (ss * (int) glob->blocksize != len * 512)
+    {
+      grub_error (GRUB_ERR_BAD_FS, "Size not match",
+		  ss * (int) glob->blocksize, len * 512);
+      return 0;
+    }
+  pu = buf + u16at (buf, 4);
+  us = u16at (pu, 0);
+  buf -= 2;
+  while (ss > 0)
+    {
+      buf += glob->blocksize;
+      pu += 2;
+      if (u16at (buf, 0) != us)
+	{
+	  grub_error (GRUB_ERR_BAD_FS, "Fixup signature not match");
+	  return 0;
+	}
+      v16at (buf, 0) = v16at (pu, 0);
+      ss--;
+    }
+  return 1;
+}
+
+static int read_mft (struct grub_ntfs_glob *glob, char *buf,
+		     unsigned long mftno);
+static int read_attr (struct grub_ntfs_attr *at, char *dest,
+		      unsigned long ofs, unsigned long len, int cached);
+static int read_data (struct grub_ntfs_attr *at, char *pa, char *dest,
+		      unsigned long ofs, unsigned long len, int cached);
+
+static void
+init_attr (struct grub_ntfs_attr *at, struct grub_ntfs_file *mft)
+{
+  at->mft = mft;
+  at->flags = (mft == &mft->glob->mmft) ? AF_MMFT : 0;
+  at->attr_nxt = mft->buf + u16at (mft->buf, 0x14);
+  at->attr_end = at->emft_buf = at->edat_buf = at->sbuf = NULL;
+}
+
+static void
+free_attr (struct grub_ntfs_attr *at)
+{
+  grub_free (at->emft_buf);
+  grub_free (at->edat_buf);
+  grub_free (at->sbuf);
+}
+
+static char *
+find_attr (struct grub_ntfs_attr *at, unsigned char attr)
+{
+  if (at->flags & AF_ALST)
+    {
+    back:
+      while (at->attr_nxt < at->attr_end)
+	{
+	  at->attr_cur = at->attr_nxt;
+	  at->attr_nxt += u16at (at->attr_cur, 4);
+	  if (((unsigned char) *at->attr_cur == attr) || (attr == 0))
+	    {
+	      char *new_pos;
+
+	      if (at->flags & AF_MMFT)
+		{
+		  if ((grub_disk_read
+		       (at->mft->glob->disk, v32at (at->attr_cur, 0x10), 0,
+			512, at->emft_buf))
+		      ||
+		      (grub_disk_read
+		       (at->mft->glob->disk, v32at (at->attr_cur, 0x14), 0,
+			512, at->emft_buf + 512)))
+		    return NULL;
+
+		  if (!fixup
+		      (at->mft->glob, at->emft_buf, at->mft->glob->mft_size,
+		       "FILE"))
+		    return NULL;
+		}
+	      else
+		{
+		  if (!read_mft
+		      (at->mft->glob, at->emft_buf,
+		       u32at (at->attr_cur, 0x10)))
+		    return NULL;
+		}
+
+	      new_pos = &at->emft_buf[u16at (at->emft_buf, 0x14)];
+	      while ((unsigned char) *new_pos != 0xFF)
+		{
+		  if (((unsigned char) *new_pos ==
+		       (unsigned char) *at->attr_cur)
+		      && (u16at (new_pos, 0xE) == u16at (at->attr_cur, 0x18)))
+		    {
+		      return new_pos;
+		    }
+		  new_pos += u16at (new_pos, 4);
+		}
+	      grub_error (GRUB_ERR_BAD_FS,
+			  "Can\'t find 0x%X in attribute list",
+			  (unsigned char) *at->attr_cur);
+	      return NULL;
+	    }
+	}
+      return NULL;
+    }
+  at->attr_cur = at->attr_nxt;
+  while ((unsigned char) *at->attr_cur != 0xFF)
+    {
+      at->attr_nxt += u16at (at->attr_cur, 4);
+      if ((unsigned char) *at->attr_cur == AT_ATTRIBUTE_LIST)
+	at->attr_end = at->attr_cur;
+      if (((unsigned char) *at->attr_cur == attr) || (attr == 0))
+	return at->attr_cur;
+      at->attr_cur = at->attr_nxt;
+    }
+  if (at->attr_end)
+    {
+      char *pa;
+
+      at->emft_buf = grub_malloc (at->mft->glob->mft_size << BLK_SHR);
+      if (at->emft_buf == NULL)
+	return NULL;
+
+      pa = at->attr_end;
+      if (pa[8])
+	{
+	  if (u32at (pa, 0x28) > 4096)
+	    {
+	      grub_error (GRUB_ERR_BAD_FS,
+			  "Non-resident attribute list too large");
+	      return NULL;
+	    }
+	  at->attr_cur = at->attr_end;
+	  at->edat_buf = grub_malloc (u32at (pa, 0x28));
+	  if (!at->edat_buf)
+	    return NULL;
+	  if (!read_data (at, pa, at->edat_buf, 0, u32at (pa, 0x28), 0))
+	    {
+	      grub_error (GRUB_ERR_BAD_FS,
+			  "Fail to read non-resident attribute list");
+	      return NULL;
+	    }
+	  at->attr_nxt = at->edat_buf;
+	  at->attr_end = at->edat_buf + u32at (pa, 0x30);
+	}
+      else
+	{
+	  at->attr_nxt = at->attr_end + u16at (pa, 0x14);
+	  at->attr_end = at->attr_end + u32at (pa, 4);
+	}
+      at->flags |= AF_ALST;
+      while (at->attr_nxt < at->attr_end)
+	{
+	  if (((unsigned char) *at->attr_nxt == attr) || (attr == 0))
+	    break;
+	  at->attr_nxt += u16at (at->attr_nxt, 4);
+	}
+      if (at->attr_nxt >= at->attr_end)
+	return NULL;
+
+      if ((at->flags & AF_MMFT) && (attr == AT_DATA))
+	{
+	  at->flags |= AF_GPOS;
+	  at->attr_cur = at->attr_nxt;
+	  pa = at->attr_cur;
+	  v32at (pa, 0x10) = at->mft->glob->mft_start;
+	  v32at (pa, 0x14) = at->mft->glob->mft_start + 1;
+	  pa = at->attr_nxt + u16at (pa, 4);
+	  while (pa < at->attr_end)
+	    {
+	      if ((unsigned char) *pa != attr)
+		break;
+	      if (!read_attr
+		  (at, pa + 0x10,
+		   u32at (pa, 0x10) * (at->mft->glob->mft_size << BLK_SHR),
+		   at->mft->glob->mft_size << BLK_SHR, 0))
+		return NULL;
+	      pa += u16at (pa, 4);
+	    }
+	  at->attr_nxt = at->attr_cur;
+	  at->flags &= ~AF_GPOS;
+	}
+      goto back;
+    }
+  return NULL;
+}
+
+static char *
+locate_attr (struct grub_ntfs_attr *at, struct grub_ntfs_file *mft,
+	     unsigned char attr)
+{
+  char *pa;
+
+  init_attr (at, mft);
+  if ((pa = find_attr (at, attr)) == NULL)
+    return NULL;
+  if ((at->flags & AF_ALST) == 0)
+    {
+      while (1)
+	{
+	  if ((pa = find_attr (at, attr)) == NULL)
+	    break;
+	  if (at->flags & AF_ALST)
+	    return pa;
+	}
+      grub_errno = GRUB_ERR_NONE;
+      free_attr (at);
+      init_attr (at, mft);
+      pa = find_attr (at, attr);
+    }
+  return pa;
+}
+
+static char *
+read_run_data (char *run, int nn, unsigned long *val, int sig)
+{
+  unsigned long r, v;
+
+  r = 0;
+  v = 1;
+
+  while (nn--)
+    {
+      r += v * (*(unsigned char *) (run++));
+      v <<= 8;
+    }
+
+  if ((sig) && (r & (v >> 1)))
+    r -= v;
+
+  *val = r;
+  return run;
+}
+
+static char *
+read_run_list (struct grub_ntfs_rlst *ctx, char *run)
+{
+  int c1, c2;
+  unsigned long val;
+
+back:
+  c1 = ((unsigned char) (*run) & 0xF);
+  c2 = ((unsigned char) (*run) >> 4);
+  if (!c1)
+    {
+      if ((ctx->attr) && (ctx->attr->flags & AF_ALST))
+	{
+	  void (*save_hook) (grub_disk_addr_t sector, unsigned offset,
+			     unsigned length);
+
+	  save_hook = ctx->comp.disk->read_hook;
+	  ctx->comp.disk->read_hook = NULL;
+	  run = find_attr (ctx->attr, (unsigned char) *ctx->attr->attr_cur);
+	  ctx->comp.disk->read_hook = save_hook;
+	  if (run)
+	    {
+	      if (run[8] == 0)
+		{
+		  grub_error (GRUB_ERR_BAD_FS,
+			      "$DATA should be non-resident");
+		  return NULL;
+		}
+	      run += u16at (run, 0x20);
+	      ctx->curr_lcn = 0;
+	      goto back;
+	    }
+	}
+      grub_error (GRUB_ERR_BAD_FS, "Run list overflow\n");
+      return NULL;
+    }
+  run = read_run_data (run + 1, c1, &val, 0);	/* length of current VCN */
+  ctx->curr_vcn = ctx->next_vcn;
+  ctx->next_vcn += val;
+  run = read_run_data (run, c2, &val, 1);	/* offset to previous LCN */
+  ctx->curr_lcn += val;
+  if (val == 0)
+    ctx->flags |= RF_BLNK;
+  else
+    ctx->flags &= ~RF_BLNK;
+  return run;
+}
+
+static int
+decomp_nextvcn (struct grub_ntfs_comp *cc)
+{
+  if (cc->comp_head >= cc->comp_tail)
+    {
+      grub_error (GRUB_ERR_BAD_FS, "C1");
+      return 0;
+    }
+  if (grub_disk_read
+      (cc->disk,
+       (cc->comp_table[cc->comp_head][1] -
+	(cc->comp_table[cc->comp_head][0] - cc->cbuf_vcn)) * cc->spc, 0,
+       cc->spc << BLK_SHR, cc->cbuf))
+    return 0;
+  cc->cbuf_vcn++;
+  if ((cc->cbuf_vcn >= cc->comp_table[cc->comp_head][0]))
+    cc->comp_head++;
+  cc->cbuf_ofs = 0;
+  return 1;
+}
+
+static int
+decomp_getch (struct grub_ntfs_comp *cc)
+{
+  if (cc->cbuf_ofs >= (cc->spc << BLK_SHR))
+    {
+      if (!decomp_nextvcn (cc))
+	return 0;
+    }
+  return (unsigned char) cc->cbuf[cc->cbuf_ofs++];
+}
+
+/* Decompress a block (4096 bytes) */
+static int
+decomp_block (struct grub_ntfs_comp *cc, char *dest)
+{
+  unsigned short flg, cnt;
+
+  flg = decomp_getch (cc);
+  flg += decomp_getch (cc) * 256;
+  cnt = (flg & 0xFFF) + 1;
+
+  if (grub_errno)
+    return 0;
+
+  if (dest)
+    {
+      if (flg & 0x8000)
+	{
+	  unsigned long bits, copied, tag;
+
+	  bits = copied = tag = 0;
+	  while (cnt > 0)
+	    {
+	      if (grub_errno)
+		return 0;
+
+	      if (copied > 4096)
+		{
+		  grub_error (GRUB_ERR_BAD_FS, "B1");
+		  return 0;
+		}
+	      if (!bits)
+		{
+		  tag = decomp_getch (cc);
+		  bits = 8;
+		  cnt--;
+		  if (cnt <= 0)
+		    break;
+		}
+	      if (tag & 1)
+		{
+		  unsigned long i, len, delta, code, lmask, dshift;
+
+		  code = decomp_getch (cc);
+		  code += decomp_getch (cc) * 256;
+		  cnt -= 2;
+
+		  if (!copied)
+		    {
+		      grub_error (GRUB_ERR_BAD_FS, "B2");
+		      return 0;
+		    }
+
+		  for (i = copied - 1, lmask = 0xFFF, dshift = 12; i >= 0x10;
+		       i >>= 1)
+		    {
+		      lmask >>= 1;
+		      dshift--;
+		    }
+
+		  delta = code >> dshift;
+		  len = (code & lmask) + 3;
+
+		  for (i = 0; i < len; i++)
+		    {
+		      dest[copied] = dest[copied - delta - 1];
+		      copied++;
+		    }
+		}
+	      else
+		{
+		  dest[copied++] = decomp_getch (cc);
+		  cnt--;
+		}
+	      tag >>= 1;
+	      bits--;
+	    }
+	  return 1;
+	}
+      else
+	{
+	  if (cnt != 4096)
+	    {
+	      grub_error (GRUB_ERR_BAD_FS, "B3");
+	      return 0;
+	    }
+	}
+    }
+
+  while (cnt > 0)
+    {
+      int n;
+
+      n = (cc->spc << BLK_SHR) - cc->cbuf_ofs;
+      if (n > cnt)
+	n = cnt;
+      if ((dest) && (n))
+	{
+	  memcpy (dest, &cc->cbuf[cc->cbuf_ofs], n);
+	  dest += n;
+	}
+      cnt -= n;
+      cc->cbuf_ofs += n;
+      if ((cnt) && (!decomp_nextvcn (cc)))
+	return 0;
+    }
+  return 1;
+}
+
+static int
+read_block (struct grub_ntfs_rlst *ctx, char *buf, int num)
+{
+  if (ctx->flags & RF_COMP)
+    {
+      int cpb = (8 / ctx->comp.spc);
+
+      while (num)
+	{
+	  int nn;
+
+	  if ((ctx->target_vcn & 0xF) == 0)
+	    {
+
+	      if (ctx->comp.comp_head != ctx->comp.comp_tail)
+		{
+		  grub_error (GRUB_ERR_BAD_FS, "A1");
+		  return 0;
+		}
+	      ctx->comp.comp_head = ctx->comp.comp_tail = 0;
+	      ctx->comp.cbuf_vcn = ctx->target_vcn;
+	      ctx->comp.cbuf_ofs = (ctx->comp.spc << BLK_SHR);
+	      if (ctx->target_vcn >= ctx->next_vcn)
+		{
+		  ctx->cur_run = read_run_list (ctx, ctx->cur_run);
+		  if (ctx->cur_run == NULL)
+		    return 0;
+		}
+	      while (ctx->target_vcn + 16 > ctx->next_vcn)
+		{
+		  if (ctx->flags & RF_BLNK)
+		    break;
+		  ctx->comp.comp_table[ctx->comp.comp_tail][0] =
+		    ctx->next_vcn;
+		  ctx->comp.comp_table[ctx->comp.comp_tail][1] =
+		    ctx->curr_lcn + ctx->next_vcn - ctx->curr_vcn;
+		  ctx->comp.comp_tail++;
+		  ctx->cur_run = read_run_list (ctx, ctx->cur_run);
+		  if (ctx->cur_run == NULL)
+		    return 0;
+		}
+	      if (ctx->target_vcn + 16 < ctx->next_vcn)
+		{
+		  grub_error (GRUB_ERR_BAD_FS, "A2");
+		  return 0;
+		}
+	    }
+
+	  nn = (16 - (ctx->target_vcn & 0xF)) / cpb;
+	  if (nn > num)
+	    nn = num;
+	  num -= nn;
+
+	  if (ctx->flags & RF_BLNK)
+	    {
+	      ctx->target_vcn += nn * cpb;
+	      if (ctx->comp.comp_tail == 0)
+		{
+		  if (buf)
+		    {
+		      grub_memset (buf, 0, nn * 4096);
+		      buf += nn * 4096;
+		    }
+		}
+	      else
+		{
+		  while (nn)
+		    {
+		      if (!decomp_block (&ctx->comp, buf))
+			return 0;
+		      if (buf)
+			buf += 4096;
+		      nn--;
+		    }
+		}
+	    }
+	  else
+	    {
+	      nn *= cpb;
+	      while ((ctx->comp.comp_head < ctx->comp.comp_tail) && (nn))
+		{
+		  int tt;
+
+		  tt =
+		    ctx->comp.comp_table[ctx->comp.comp_head][0] -
+		    ctx->target_vcn;
+		  if (tt > nn)
+		    tt = nn;
+		  ctx->target_vcn += tt;
+		  if (buf)
+		    {
+		      if (grub_disk_read
+			  (ctx->comp.disk,
+			   (ctx->comp.comp_table[ctx->comp.comp_head][1] -
+			    (ctx->comp.comp_table[ctx->comp.comp_head][0] -
+			     ctx->target_vcn)) * ctx->comp.spc, 0,
+			   tt * (ctx->comp.spc << BLK_SHR), buf))
+			return 0;
+		      buf += tt * (ctx->comp.spc << BLK_SHR);
+		    }
+		  nn -= tt;
+		  if (ctx->target_vcn >=
+		      ctx->comp.comp_table[ctx->comp.comp_head][0])
+		    ctx->comp.comp_head++;
+		}
+	      if (nn)
+		{
+		  if (buf)
+		    {
+		      if (grub_disk_read
+			  (ctx->comp.disk, ctx->curr_lcn * ctx->comp.spc, 0,
+			   nn * (ctx->comp.spc << BLK_SHR), buf))
+			return 0;
+		      buf += nn * (ctx->comp.spc << BLK_SHR);
+		    }
+		  ctx->target_vcn += nn;
+		}
+	    }
+	}
+    }
+  else
+    {
+      while (num)
+	{
+	  int nn, ss;
+
+	  nn =
+	    (ctx->next_vcn - ctx->target_vcn) * ctx->comp.spc -
+	    ctx->vcn_offset;
+
+	  if (nn > num)
+	    nn = num;
+
+	  if (buf)
+	    {
+	      if (ctx->flags & RF_BLNK)
+		grub_memset (buf, 0, nn << BLK_SHR);
+	      else
+		if (grub_disk_read
+		    (ctx->comp.disk,
+		     (ctx->target_vcn - ctx->curr_vcn +
+		      ctx->curr_lcn) * ctx->comp.spc + ctx->vcn_offset, 0,
+		     nn << BLK_SHR, buf))
+		return 0;
+	      buf += (nn << BLK_SHR);
+	    }
+	  ss = ctx->target_vcn * ctx->comp.spc + ctx->vcn_offset + nn;
+	  ctx->target_vcn = ss / ctx->comp.spc;
+	  ctx->vcn_offset = ss % ctx->comp.spc;
+	  num -= nn;
+	  if (num == 0)
+	    break;
+
+	  if (ctx->target_vcn >= ctx->next_vcn)
+	    {
+	      ctx->cur_run = read_run_list (ctx, ctx->cur_run);
+	      if (ctx->cur_run == NULL)
+		return 0;
+	    }
+	}
+    }
+  return 1;
+}
+
+static int
+read_data (struct grub_ntfs_attr *at, char *pa, char *dest, unsigned long ofs,
+	   unsigned long len, int cached)
+{
+  unsigned long vcn, blk_size;
+  struct grub_ntfs_rlst cc, *ctx;
+  int ret;
+
+  if (len == 0)
+    return 1;
+
+  grub_memset (&cc, 0, sizeof (cc));
+  ctx = &cc;
+  ctx->attr = at;
+  ctx->comp.spc = at->mft->glob->spc;
+  ctx->comp.disk = at->mft->glob->disk;
+
+  if (pa[8] == 0)
+    {
+      if (ofs + len > u32at (pa, 0x10))
+	{
+	  grub_error (GRUB_ERR_BAD_FS, "Read out of range");
+	  return 0;
+	}
+      grub_memcpy (dest, pa + u32at (pa, 0x14) + ofs, len);
+      return 1;
+    }
+
+  if (u16at (pa, 0xC) & FLAG_COMPRESSED)
+    ctx->flags |= RF_COMP;
+  else
+    ctx->flags &= ~RF_COMP;
+  ctx->cur_run = pa + u16at (pa, 0x20);
+  blk_size = (ctx->flags & RF_COMP) ? 4096 : 512;
+
+  if ((ctx->flags & RF_COMP) && (!cached))
+    {
+      grub_error (GRUB_ERR_BAD_FS, "Attribute can\'t be compressed");
+      return 0;
+    }
+
+  if (cached)
+    {
+      if (at->sbuf)
+	{
+	  if ((ofs & (~(blk_size - 1))) == at->save_pos)
+	    {
+	      unsigned long n;
+
+	      n = blk_size - (ofs - at->save_pos);
+	      if (n > len)
+		n = len;
+
+	      grub_memcpy (dest, at->sbuf + ofs - at->save_pos, n);
+	      if (n == len)
+		return 1;
+
+	      dest += n;
+	      len -= n;
+	      ofs += n;
+	    }
+	}
+      else
+	{
+	  at->sbuf = grub_malloc (blk_size);
+	  if (at->sbuf == NULL)
+	    return 0;
+	  at->save_pos = 1;
+	}
+    }
+
+  if (ctx->flags & RF_COMP)
+    {
+      vcn = ctx->target_vcn = (ofs / 4096) * (8 / ctx->comp.spc);
+      ctx->vcn_offset = 0;
+      ctx->target_vcn &= ~0xF;
+    }
+  else
+    {
+      vcn = ctx->target_vcn = (ofs >> BLK_SHR) / ctx->comp.spc;
+      ctx->vcn_offset = (ofs >> BLK_SHR) % ctx->comp.spc;
+    }
+
+  ctx->next_vcn = u32at (pa, 0x10);
+  ctx->curr_lcn = 0;
+  while (ctx->next_vcn <= ctx->target_vcn)
+    {
+      ctx->cur_run = read_run_list (ctx, ctx->cur_run);
+      if (ctx->cur_run == NULL)
+	return 0;
+    }
+
+  if (at->flags & AF_GPOS)
+    {
+      unsigned long st0, st1;
+
+      st0 =
+	(ctx->target_vcn - ctx->curr_vcn + ctx->curr_lcn) * ctx->comp.spc +
+	ctx->vcn_offset;
+      st1 = st0 + 1;
+      if (st1 ==
+	  (ctx->next_vcn - ctx->curr_vcn + ctx->curr_lcn) * ctx->comp.spc)
+	{
+	  ctx->cur_run = read_run_list (ctx, ctx->cur_run);
+	  if (ctx->cur_run == NULL)
+	    return 0;
+	  st1 = ctx->curr_lcn * ctx->comp.spc;
+	}
+      v32at (dest, 0) = st0;
+      v32at (dest, 4) = st1;
+      return 1;
+    }
+
+  ctx->comp.comp_head = ctx->comp.comp_tail = 0;
+  if (ctx->flags & RF_COMP)
+    {
+      ctx->comp.cbuf = grub_malloc ((ctx->comp.spc) << BLK_SHR);
+      if (!ctx->comp.cbuf)
+	return 0;
+    }
+
+  ret = 0;
+
+  if ((cached) && (u16at (pa, 0xC) & (FLAG_COMPRESSED + FLAG_SPARSE)) == 0)
+    ctx->comp.disk->read_hook = at->mft->read_hook;
+
+  if ((vcn > ctx->target_vcn) &&
+      (!read_block
+       (ctx, NULL, ((vcn - ctx->target_vcn) * ctx->comp.spc) / 8)))
+    goto fail;
+
+  if (ofs % blk_size)
+    {
+      unsigned long t, n, o;
+
+      if (!cached)
+	{
+	  grub_error (GRUB_ERR_BAD_FS, "Invalid range");
+	  goto fail;
+	}
+
+      t = ctx->target_vcn * (ctx->comp.spc << BLK_SHR);
+      if (!read_block (ctx, at->sbuf, 1))
+	goto fail;
+
+      at->save_pos = t;
+
+      o = ofs % blk_size;
+      n = blk_size - o;
+      if (n > len)
+	n = len;
+      grub_memcpy (dest, &at->sbuf[o], n);
+      if (n == len)
+	goto done;
+      dest += n;
+      len -= n;
+    }
+
+  if (!read_block (ctx, dest, len / blk_size))
+    goto fail;
+
+  dest += (len / blk_size) * blk_size;
+  len = len % blk_size;
+  if (len)
+    {
+      unsigned long t;
+
+      if (!cached)
+	{
+	  grub_error (GRUB_ERR_BAD_FS, "Invalid range");
+	  goto fail;
+	}
+
+      t = ctx->target_vcn * (ctx->comp.spc << BLK_SHR);
+      if (!read_block (ctx, at->sbuf, 1))
+	goto fail;
+
+      at->save_pos = t;
+
+      grub_memcpy (dest, at->sbuf, len);
+    }
+done:
+  ret = 1;
+fail:
+  ctx->comp.disk->read_hook = NULL;
+  if (ctx->comp.cbuf)
+    grub_free (ctx->comp.cbuf);
+  return ret;
+}
+
+static int
+read_attr (struct grub_ntfs_attr *at, char *dest, unsigned long ofs,
+	   unsigned long len, int cached)
+{
+  char *save_cur;
+  unsigned char attr;
+  char *pp;
+  int ret;
+
+  save_cur = at->attr_cur;
+  at->attr_nxt = at->attr_cur;
+  attr = (unsigned char) *at->attr_nxt;
+  if (at->flags & AF_ALST)
+    {
+      char *pa;
+      unsigned long vcn;
+
+      vcn = ofs / (at->mft->glob->spc << BLK_SHR);
+      pa = at->attr_nxt + u16at (at->attr_nxt, 4);
+      while (pa < at->attr_end)
+	{
+	  if ((unsigned char) *pa != attr)
+	    break;
+	  if (u32at (pa, 8) > vcn)
+	    break;
+	  at->attr_nxt = pa;
+	  pa += u16at (pa, 4);
+	}
+    }
+  pp = find_attr (at, attr);
+  ret = (pp) ? read_data (at, pp, dest, ofs, len, cached) : 0;
+  at->attr_cur = save_cur;
+  return ret;
+}
+
+static int
+read_mft (struct grub_ntfs_glob *glob, char *buf, unsigned long mftno)
+{
+  if (!read_attr
+      (&glob->mmft.attr, buf, mftno * (glob->mft_size << BLK_SHR),
+       glob->mft_size << BLK_SHR, 0))
+    {
+      grub_error (GRUB_ERR_BAD_FS, "Read MFT 0x%X fails", mftno);
+      return 0;
+    }
+  return fixup (glob, buf, glob->mft_size, "FILE");
+}
+
+static int
+init_file (struct grub_ntfs_file *mft, unsigned long mftno)
+{
+  unsigned short flag;
+
+  mft->buf = grub_malloc (mft->glob->mft_size << BLK_SHR);
+  if (mft->buf == NULL)
+    return 0;
+
+  if (!read_mft (mft->glob, mft->buf, mftno))
+    return 0;
+
+  flag = u16at (mft->buf, 0x16);
+  if ((flag & 1) == 0)
+    {
+      grub_error (GRUB_ERR_BAD_FS, "MFT 0x%X is not in use", mftno);
+      return 0;
+    }
+  if ((flag & 2) == 0)
+    {
+      char *pa;
+
+      pa = locate_attr (&mft->attr, mft, AT_DATA);
+      if (pa == NULL)
+	{
+	  grub_error (GRUB_ERR_BAD_FS, "No $DATA in MFT 0x%X", mftno);
+	  return 0;
+	}
+
+      if (!pa[8])
+	mft->filemax = u32at (pa, 0x10);
+      else
+	mft->filemax = u32at (pa, 0x30);
+
+      if ((mft->attr.flags & AF_ALST) == 0)
+	mft->attr.attr_end = 0;	/*  Don't jump to attribute list */
+    }
+  return 1;
+}
+
+static void
+free_file (struct grub_ntfs_file *mft)
+{
+  free_attr (&mft->attr);
+  grub_free (mft->buf);
+}
+
+static int
+list_file (struct grub_ntfs_file *mft, char *fn, char *pos,
+	   int (*hook) (const char *filename, int dir))
+{
+  char *np;
+  int i, ns, len;
+
+  len = grub_strlen (fn);
+  while (1)
+    {
+      char *ustr;
+      if (pos[0xC] & 2)		/* end signature */
+	break;
+
+      np = pos + 0x52;
+      ns = (unsigned char) *(np - 2);
+      ustr = grub_malloc (ns * 4 + 1);
+      if (ustr == NULL)
+	return 0;
+      *grub_utf16_to_utf8 ((grub_uint8_t *) ustr, (grub_uint16_t *) np, ns) =
+	'\0';
+      ns = grub_strlen (ustr);
+      if (((hook) && (ns >= len)) || ((!hook) && (ns == len)))
+	{
+	  for (i = 0; i < len; i++)
+	    if (grub_tolower (fn[i]) != grub_tolower (ustr[i]))
+	      break;
+	  if (i >= len)
+	    {
+	      if (hook)
+		{
+		  if ((i)
+		      || ((*ustr != '$') && ((*ustr != '.') || (ns != 1))))
+		    {
+		      (*hook) (ustr,
+			       (u32at (pos, 0x48) & ATTR_DIRECTORY) != 0);
+		    }
+		}
+	      else
+		{
+		  if (u16at (pos, 4))
+		    {
+		      grub_error (GRUB_ERR_BAD_FS, "64-bit MFT number\n");
+		      return 0;
+		    }
+		  free_file (mft);
+		  grub_free (ustr);
+		  return init_file (mft, u32at (pos, 0));
+		}
+	    }
+	}
+      grub_free (ustr);
+      pos += u16at (pos, 8);
+    }
+  return -1;
+}
+
+static int
+scan_dir (struct grub_ntfs_file *mft, char *fn,
+	  int (*hook) (const char *filename, int dir))
+{
+  unsigned char *bitmap;
+  struct grub_ntfs_attr attr, *at;
+  char *cur_pos, *indx;
+  int ret = 0;
+
+  if ((u16at (mft->buf, 0x16) & 2) == 0)
+    {
+      grub_error (GRUB_ERR_FILE_NOT_FOUND, "file not found");
+      return 0;
+    }
+
+  indx = NULL;
+  at = &attr;
+  init_attr (at, mft);
+  while (1)
+    {
+      if ((cur_pos = find_attr (at, AT_INDEX_ROOT)) == NULL)
+	{
+	  grub_error (GRUB_ERR_BAD_FS, "No $INDEX_ROOT");
+	  goto fail;
+	}
+
+      /* Resident, Namelen=4, Offset=0x18, Flags=0x00, Name="$I30" */
+      if ((u32at (cur_pos, 8) != 0x180400) ||
+	  (u32at (cur_pos, 0x18) != 0x490024) ||
+	  (u32at (cur_pos, 0x1C) != 0x300033))
+	continue;
+      cur_pos += u16at (cur_pos, 0x14);
+      if (*cur_pos != 0x30)	/* Not filename index */
+	continue;
+      break;
+    }
+
+  cur_pos += 0x10;		/* Skip index root */
+  ret = list_file (mft, fn, cur_pos + u16at (cur_pos, 0), hook);
+  if (ret >= 0)
+    goto done;
+
+  bitmap = NULL;
+  free_attr (at);
+  init_attr (at, mft);
+  while ((cur_pos = find_attr (at, AT_BITMAP)) != NULL)
+    {
+      /* Resident, Namelen=4, Offset=0x18, Flags=0x00, Name="$I30" */
+      if ((u32at (cur_pos, 8) == 0x180400) &&
+	  (u32at (cur_pos, 0x18) == 0x490024) &&
+	  (u32at (cur_pos, 0x1C) == 0x300033))
+	{
+	  bitmap = (unsigned char *) cur_pos;
+	  if (at->flags & AF_ALST)
+	    {
+	      grub_error (GRUB_ERR_BAD_FS,
+			  "$BITMAP should not in attribute list");
+	      goto fail;
+	    }
+	  if (bitmap[8])
+	    {
+	      grub_error (GRUB_ERR_BAD_FS,
+			  "Non-resident $BITMAP not supported");
+	      goto fail;
+	    }
+	  break;
+	}
+    }
+
+  free_attr (at);
+  cur_pos = locate_attr (at, mft, AT_INDEX_ALLOCATION);
+  while (cur_pos != NULL)
+    {
+      /* Non-resident, Namelen=4, Offset=0x40, Flags=0, Name="$I30" */
+      if ((u32at (cur_pos, 8) == 0x400401) &&
+	  (u32at (cur_pos, 0x40) == 0x490024) &&
+	  (u32at (cur_pos, 0x44) == 0x300033))
+	break;
+      cur_pos = find_attr (at, AT_INDEX_ALLOCATION);
+    }
+
+  if ((!cur_pos) && (bitmap))
+    {
+      grub_error (GRUB_ERR_BAD_FS, "$BITMAP without $INDEX_ALLOCATION");
+      goto fail;
+    }
+
+  if (bitmap)
+    {
+      unsigned long v, i, len;
+
+      indx = grub_malloc (mft->glob->idx_size << BLK_SHR);
+      if (indx == NULL)
+	goto fail;
+
+      len = u32at (bitmap, 0x10);
+      bitmap += u16at (bitmap, 0x14);
+      v = 1;
+      for (i = 0; i < len * 8; i++)
+	{
+	  if (*bitmap & v)
+	    {
+	      if ((!read_attr
+		   (at, indx, i * (mft->glob->idx_size << BLK_SHR),
+		    (mft->glob->idx_size << BLK_SHR), 0))
+		  || (!fixup (mft->glob, indx, mft->glob->idx_size, "INDX")))
+		goto fail;
+	      ret =
+		list_file (mft, fn, &indx[0x18 + u16at (indx, 0x18)], hook);
+	      if (ret >= 0)
+		goto done;
+	    }
+	  v <<= 1;
+	  if (v >= 0x100)
+	    {
+	      v = 1;
+	      bitmap++;
+	    }
+	}
+    }
+
+fail:
+  ret = 0;
+
+done:
+  free_attr (at);
+  grub_free (indx);
+  return ret;
+}
+
+static int
+find_dir (struct grub_ntfs_file *mft, const char *path,
+	  int (*hook) (const char *filename, int dir))
+{
+  char *next, *dirname, *save;
+  int ret;
+
+  while (*path == '/')
+    path++;
+
+  save = dirname = grub_strdup (path);
+
+  ret = 0;
+  if (!init_file (mft, FILE_ROOT))
+    goto fail;
+
+  while ((next = grub_strchr (dirname, '/')) != NULL)
+    {
+      *next = 0;
+      ret = scan_dir (mft, dirname, NULL);
+      if (!ret)
+	goto fail;
+      dirname = next + 1;
+    }
+  ret = scan_dir (mft, dirname, hook);
+fail:
+  grub_free (save);
+  return ret;
+}
+
+static struct grub_ntfs_glob *
+grub_ntfs_mount (grub_disk_t disk)
+{
+  struct grub_ntfs_bpb bpb;
+  struct grub_ntfs_glob *glob = 0;
+
+  if (!disk)
+    goto fail;
+
+  glob = (struct grub_ntfs_glob *) grub_malloc (sizeof (*glob));
+  if (!glob)
+    goto fail;
+
+  grub_memset (glob, 0, sizeof (*glob));
+
+  glob->disk = disk;
+
+  /* Read the BPB.  */
+  if (grub_disk_read (disk, 0, 0, sizeof (bpb), (char *) &bpb))
+    goto fail;
+
+  if (grub_memcmp ((char *) &bpb.oem_name, "NTFS", 4))
+    goto fail;
+
+  glob->blocksize = grub_le_to_cpu16 (bpb.bytes_per_sector);
+  glob->spc = bpb.sectors_per_cluster * (glob->blocksize >> BLK_SHR);
+
+  if (bpb.clusters_per_mft > 0)
+    glob->mft_size = glob->spc * bpb.clusters_per_mft;
+  else
+    glob->mft_size = 1 << (-bpb.clusters_per_mft - BLK_SHR);
+
+  if (bpb.clusters_per_index > 0)
+    glob->idx_size = glob->spc * bpb.clusters_per_index;
+  else
+    glob->idx_size = 1 << (-bpb.clusters_per_index - BLK_SHR);
+
+  glob->mft_start = grub_le_to_cpu64 (bpb.mft_lcn) * glob->spc;
+
+  if ((glob->mft_size > MAX_MFT) || (glob->idx_size > MAX_IDX) ||
+      (glob->spc > MAX_SPC) || (glob->spc > glob->idx_size))
+    {
+      grub_error (GRUB_ERR_BAD_FS, "large structure");
+      goto fail;
+    }
+
+  glob->mmft.glob = glob;
+
+  glob->mmft.buf = grub_malloc (glob->mft_size << BLK_SHR);
+  if (!glob->mmft.buf)
+    goto fail;
+
+  if (grub_disk_read
+      (disk, glob->mft_start, 0, glob->mft_size << BLK_SHR, glob->mmft.buf))
+    goto fail_1;
+
+  if (!fixup (glob, glob->mmft.buf, glob->mft_size, "FILE"))
+    goto fail_1;
+
+  if (!locate_attr (&glob->mmft.attr, &glob->mmft, AT_DATA))
+    goto fail_1;
+
+  return glob;
+
+fail_1:
+  grub_free (glob->mmft.buf);
+fail:
+  grub_free (glob);
+  if (grub_errno == GRUB_ERR_NONE)
+    grub_error (GRUB_ERR_BAD_FS, "not a ntfs filesystem");
+  return NULL;
+}
+
+static grub_err_t
+grub_ntfs_dir (grub_device_t device, const char *path,
+	       int (*hook) (const char *filename, int dir))
+{
+  struct grub_ntfs_glob *glob = 0;
+  struct grub_ntfs_file *mft = 0;
+
+#ifndef GRUB_UTIL
+  grub_dl_ref (my_mod);
+#endif
+
+  glob = grub_ntfs_mount (device->disk);
+  if (!glob)
+    goto fail;
+
+  mft = (struct grub_ntfs_file *) grub_malloc (sizeof (*mft));
+  if (!mft)
+    goto fail_1;
+  grub_memset (mft, 0, sizeof (*mft));
+  mft->glob = glob;
+
+  find_dir (mft, path, hook);
+
+  free_file (mft);
+  grub_free (mft);
+
+fail_1:
+  free_file (&glob->mmft);
+  grub_free (glob);
+
+fail:
+
+#ifndef GRUB_UTIL
+  grub_dl_unref (my_mod);
+#endif
+
+  return grub_errno;
+}
+
+static grub_err_t
+grub_ntfs_open (grub_file_t file, const char *name)
+{
+  struct grub_ntfs_glob *glob = 0;
+  struct grub_ntfs_file *mft = 0;
+
+#ifndef GRUB_UTIL
+  grub_dl_ref (my_mod);
+#endif
+
+  glob = grub_ntfs_mount (file->device->disk);
+  if (!glob)
+    goto fail;
+
+  mft = (struct grub_ntfs_file *) grub_malloc (sizeof (*mft));
+  if (!mft)
+    goto fail_1;
+  grub_memset (mft, 0, sizeof (*mft));
+  mft->glob = glob;
+
+  if (!find_dir (mft, name, NULL))
+    goto fail_2;
+
+  file->data = mft;
+  file->size = mft->filemax;
+
+  return GRUB_ERR_NONE;
+
+fail_2:
+  free_file (mft);
+  grub_free (mft);
+
+fail_1:
+  free_file (&glob->mmft);
+  grub_free (glob);
+fail:
+
+#ifndef GRUB_UTIL
+  grub_dl_unref (my_mod);
+#endif
+
+  return grub_errno;
+}
+
+static grub_ssize_t
+grub_ntfs_read (grub_file_t file, char *buf, grub_size_t len)
+{
+  struct grub_ntfs_file *mft;
+
+  mft = file->data;
+  if ((!file->offset) && (file->read_hook))
+    mft->attr.save_pos = 1;
+  mft->read_hook = file->read_hook;
+  read_attr (&mft->attr, buf, file->offset, len, 1);
+  mft->read_hook = NULL;
+  return (grub_errno) ? 0 : len;
+}
+
+static grub_err_t
+grub_ntfs_close (grub_file_t file)
+{
+  struct grub_ntfs_file *mft;
+  struct grub_ntfs_glob *glob;
+
+  mft = file->data;
+  glob = mft->glob;
+
+  free_file (mft);
+  grub_free (mft);
+  free_file (&glob->mmft);
+  grub_free (glob);
+
+#ifndef GRUB_UTIL
+  grub_dl_unref (my_mod);
+#endif
+
+  return grub_errno;
+}
+
+static grub_err_t
+grub_ntfs_label (grub_device_t device, char **label)
+{
+  struct grub_ntfs_glob *glob;
+  struct grub_ntfs_file *mft;
+  char *pa;
+
+#ifndef GRUB_UTIL
+  grub_dl_ref (my_mod);
+#endif
+
+  glob = grub_ntfs_mount (device->disk);
+  if (!glob)
+    goto fail;
+
+  mft = (struct grub_ntfs_file *) grub_malloc (sizeof (*mft));
+  if (!mft)
+    goto fail_1;
+  grub_memset (mft, 0, sizeof (*mft));
+  mft->glob = glob;
+
+  if (!find_dir (mft, "$VOLUME", NULL))
+    goto fail_2;
+
+  free_attr (&mft->attr);
+  init_attr (&mft->attr, mft);
+  pa = find_attr (&mft->attr, AT_VOLUME_NAME);
+  if ((pa) && (pa[8] == 0) && (u32at (pa, 0x10)))
+    {
+      char *buf;
+      int len;
+
+      len = u32at (pa, 0x10) / 2;
+      buf = grub_malloc (len * 4 + 1);
+      pa += u16at (pa, 0x14);
+      *grub_utf16_to_utf8 ((grub_uint8_t *) buf, (grub_uint16_t *) pa, len) =
+	'\0';
+      *label = buf;
+    }
+  else
+    *label = 0;
+
+fail_2:
+  free_file (mft);
+  grub_free (mft);
+
+fail_1:
+  free_file (&glob->mmft);
+  grub_free (glob);
+
+fail:
+
+#ifndef GRUB_UTIL
+  grub_dl_unref (my_mod);
+#endif
+
+  return grub_errno;
+}
+
+static struct grub_fs grub_ntfs_fs = {
+  .name = "ntfs",
+  .dir = grub_ntfs_dir,
+  .open = grub_ntfs_open,
+  .read = grub_ntfs_read,
+  .close = grub_ntfs_close,
+  .label = grub_ntfs_label,
+  .next = 0
+};
+
+GRUB_MOD_INIT (ntfs)
+{
+  grub_fs_register (&grub_ntfs_fs);
+#ifndef GRUB_UTIL
+  my_mod = mod;
+#endif
+}
+
+GRUB_MOD_FINI (ntfs)
+{
+  grub_fs_unregister (&grub_ntfs_fs);
+}

Re: Update for NTFS file system driver

[Robert Millan]

[-- Attachment #1: Type: text/plain, Size: 1085 bytes --]


I noticed that your patch isn't adding ntfs to the utils (grub-probe, etc).

I'm attaching a fix for that.  ChangeLog entries:

	* conf/powerpc-ieee1275.rmk: Add fs/ntfs.c to grub-probe and grub-emu.

	* conf/i386-efi.rmk: Likewise. Also remove unused grub-setup definition.

	* conf/i386-pc.rmk: Add fs/ntfs.c to grub-probe, grub-emu and
	grub-setup.

On Sun, Jun 10, 2007 at 08:03:29PM +0800, Bean wrote:
> Ok, I reformat the patch, hopefully it will conform to standard.

I think what Marco meant is a multipart/mixed attachment (like the one I'm
sending now).  This is what most mail programs do when you attach a text file
(but not a gzipped one!).

The good thing about this method, is that you can easily extract a patch
(without copy-pasting or being exposed to unwanted format conversions), but
the patch is still part of the message so you can reply to it to discuss
specific parts, keeping their context.

-- 
Robert Millan

My spam trap is honeypot@aybabtu.com.  Note: this address is only intended
for spam harvesters.  Writing to it will get you added to my black list.

[-- Attachment #2: add_ntfs_to_utils.diff --]
[-- Type: text/x-diff, Size: 3933 bytes --]


	* conf/powerpc-ieee1275.rmk: Add fs/ntfs.c to grub-probe and grub-emu.

	* conf/i386-efi.rmk: Likewise. Also remove unused grub-setup definition.

	* conf/i386-pc.rmk: Add fs/ntfs.c to grub-probe, grub-emu and
	grub-setup.

diff -ur grub2.old/conf/i386-efi.rmk grub2/conf/i386-efi.rmk
--- grub2.old/conf/i386-efi.rmk	2007-06-04 21:48:53.000000000 +0200
+++ grub2/conf/i386-efi.rmk	2007-06-10 17:03:39.000000000 +0200
@@ -15,14 +15,6 @@
 grub_mkimage_SOURCES = util/i386/efi/grub-mkimage.c util/misc.c \
 	util/resolve.c
 
-# For grub-setup.
-#grub_setup_SOURCES = util/i386/pc/grub-setup.c util/i386/pc/biosdisk.c	\
-#	util/misc.c util/i386/pc/getroot.c kern/device.c kern/disk.c	\
-#	kern/err.c kern/misc.c fs/fat.c fs/ext2.c fs/xfs.c fs/affs.c	\
-#	fs/sfs.c kern/parser.c kern/partition.c partmap/pc.c		\
-#	fs/ufs.c fs/minix.c fs/hfs.c fs/jfs.c fs/hfsplus.c kern/file.c	\
-#	kern/fs.c kern/env.c fs/fshelp.c
-
 # For grub-mkdevicemap.
 grub_mkdevicemap_SOURCES = util/grub-mkdevicemap.c util/misc.c
 
@@ -32,7 +24,7 @@
 	kern/device.c kern/disk.c kern/err.c kern/misc.c fs/fat.c	\
 	fs/ext2.c kern/parser.c kern/partition.c 			\
 	partmap/pc.c partmap/apple.c partmap/gpt.c 			\
-	fs/ufs.c fs/minix.c fs/hfs.c fs/jfs.c kern/fs.c	\
+	fs/ufs.c fs/minix.c fs/hfs.c fs/jfs.c fs/ntfs.c kern/fs.c	\
 	kern/env.c fs/fshelp.c fs/xfs.c fs/affs.c fs/sfs.c fs/hfsplus.c	\
 	disk/lvm.c disk/raid.c
 
@@ -48,6 +40,7 @@
 	disk/loopback.c							\
 	fs/affs.c fs/ext2.c fs/fat.c fs/fshelp.c fs/hfs.c fs/iso9660.c	\
 	fs/jfs.c fs/minix.c fs/sfs.c fs/ufs.c fs/xfs.c fs/hfsplus.c	\
+	fs/ntfs.c							\
 	io/gzio.c							\
 	kern/device.c kern/disk.c kern/dl.c kern/elf.c kern/env.c	\
 	kern/err.c							\
diff -ur grub2.old/conf/i386-pc.rmk grub2/conf/i386-pc.rmk
--- grub2.old/conf/i386-pc.rmk	2007-06-04 21:48:53.000000000 +0200
+++ grub2/conf/i386-pc.rmk	2007-06-10 17:02:38.000000000 +0200
@@ -67,7 +67,8 @@
 	kern/err.c kern/misc.c fs/fat.c fs/ext2.c fs/xfs.c fs/affs.c	\
 	fs/sfs.c kern/parser.c kern/partition.c	partmap/pc.c		\
 	partmap/gpt.c fs/ufs.c fs/minix.c fs/hfs.c fs/jfs.c		\
-	fs/hfsplus.c kern/file.c kern/fs.c kern/env.c fs/fshelp.c	\
+	fs/hfsplus.c fs/ntfs.c kern/file.c kern/fs.c kern/env.c 	\
+	fs/fshelp.c 							\
 	util/raid.c util/lvm.c
 
 # For grub-mkdevicemap.
@@ -79,7 +80,7 @@
 	kern/device.c kern/disk.c kern/err.c kern/misc.c fs/fat.c	\
 	fs/ext2.c kern/parser.c kern/partition.c 			\
 	partmap/pc.c partmap/apple.c partmap/gpt.c 			\
-	fs/ufs.c fs/minix.c fs/hfs.c fs/jfs.c kern/fs.c	\
+	fs/ufs.c fs/minix.c fs/hfs.c fs/jfs.c fs/ntfs.c kern/fs.c	\
 	kern/env.c fs/fshelp.c fs/xfs.c fs/affs.c fs/sfs.c fs/hfsplus.c	\
 	disk/lvm.c disk/raid.c
 
@@ -95,6 +96,7 @@
 	disk/loopback.c	disk/raid.c disk/lvm.c				\
 	fs/affs.c fs/ext2.c fs/fat.c fs/fshelp.c fs/hfs.c fs/iso9660.c	\
 	fs/jfs.c fs/minix.c fs/sfs.c fs/ufs.c fs/xfs.c fs/hfsplus.c	\
+	fs/ntfs.c							\
 	io/gzio.c							\
 	kern/device.c kern/disk.c kern/dl.c kern/elf.c kern/env.c	\
 	kern/err.c							\
diff -ur grub2.old/conf/powerpc-ieee1275.rmk grub2/conf/powerpc-ieee1275.rmk
--- grub2.old/conf/powerpc-ieee1275.rmk	2007-06-04 21:48:53.000000000 +0200
+++ grub2/conf/powerpc-ieee1275.rmk	2007-06-10 17:04:01.000000000 +0200
@@ -44,7 +44,7 @@
 	kern/device.c kern/disk.c kern/err.c kern/misc.c fs/fat.c	\
 	fs/ext2.c kern/parser.c kern/partition.c 			\
 	partmap/pc.c partmap/apple.c partmap/gpt.c 			\
-	fs/ufs.c fs/minix.c fs/hfs.c fs/jfs.c kern/fs.c	\
+	fs/ufs.c fs/minix.c fs/hfs.c fs/jfs.c fs/ntfs.c kern/fs.c	\
 	kern/env.c fs/fshelp.c fs/xfs.c fs/affs.c fs/sfs.c fs/hfsplus.c	\
 	disk/lvm.c disk/raid.c
 
@@ -59,6 +59,7 @@
 	disk/loopback.c							\
 	fs/affs.c fs/ext2.c fs/fat.c fs/fshelp.c fs/hfs.c fs/iso9660.c	\
 	fs/jfs.c fs/minix.c fs/sfs.c fs/ufs.c fs/xfs.c fs/hfsplus.c	\
+	fs/ntfs.c							\
 	io/gzio.c							\
 	kern/device.c kern/disk.c kern/dl.c kern/elf.c kern/env.c	\
 	kern/err.c kern/file.c kern/fs.c kern/loader.c kern/main.c	\

Re: Update for NTFS file system driver

[Marco Gerards]

Robert Millan <rmh@aybabtu.com> writes:

> I noticed that your patch isn't adding ntfs to the utils (grub-probe, etc).
>
> I'm attaching a fix for that.  ChangeLog entries:
>
> 	* conf/powerpc-ieee1275.rmk: Add fs/ntfs.c to grub-probe and grub-emu.
>
> 	* conf/i386-efi.rmk: Likewise. Also remove unused grub-setup definition.
>
> 	* conf/i386-pc.rmk: Add fs/ntfs.c to grub-probe, grub-emu and
> 	grub-setup.
>
> On Sun, Jun 10, 2007 at 08:03:29PM +0800, Bean wrote:
>> Ok, I reformat the patch, hopefully it will conform to standard.
>
> I think what Marco meant is a multipart/mixed attachment (like the one I'm
> sending now).  This is what most mail programs do when you attach a text file
> (but not a gzipped one!).

Actually, what he did was just fine :-).

> The good thing about this method, is that you can easily extract a patch
> (without copy-pasting or being exposed to unwanted format conversions), but
> the patch is still part of the message so you can reply to it to discuss
> specific parts, keeping their context.

If it is an inline attachment or of the patch is inline in the email
doesn't matter much to me.  I personally like it how Bean sent it :).

--
Marco

Re: Update for NTFS file system driver

[Bean]

On Sun, Jun 10, 2007 at 05:15:53PM +0200, Robert Millan wrote:
> 
> I noticed that your patch isn't adding ntfs to the utils (grub-probe, etc).
> 
> I'm attaching a fix for that.  ChangeLog entries:
> 
> 	* conf/powerpc-ieee1275.rmk: Add fs/ntfs.c to grub-probe and grub-emu.
> 
> 	* conf/i386-efi.rmk: Likewise. Also remove unused grub-setup definition.
> 
> 	* conf/i386-pc.rmk: Add fs/ntfs.c to grub-probe, grub-emu and
> 	grub-setup.
> 

Thanks for pointing out. I'm still learning about the grub2 architecture, 
sometimes I miss a spot.

Another thing I notice, the blocklist command generate messy result when
compared to GRUB Legacy, is it supposed to be so ?

BTW, is it possible for me to join the GRUB2 development term ? I really
want to help out.

-- 
Bean <bean123@126.com>

Re: Update for NTFS file system driver

[Marco Gerards]

Bean <bean123@126.com> writes:

[...]

> Another thing I notice, the blocklist command generate messy result when
> compared to GRUB Legacy, is it supposed to be so ?

What do you mean?  Can you give an example?

> BTW, is it possible for me to join the GRUB2 development term ? I really
> want to help out.

Sure, just keep on sending patches :-)

--
Marco