* gzio is broken
@ 2005-10-15 0:47 Timothy Baldwin
2005-10-15 8:07 ` Yoshinori K. Okuji
0 siblings, 1 reply; 8+ messages in thread
From: Timothy Baldwin @ 2005-10-15 0:47 UTC (permalink / raw)
To: grub-devel
[-- Attachment #1.1: Type: text/plain, Size: 1711 bytes --]
Problem 1: In the function test_header, the array buf is accessed via pointers
to grub_uint32_t and grub_uint16_t without ensuring correct alignment of the
pointers. This does not work on ARM. Fixed by:
2005-10-15 Timothy Baldwin <T.E.Baldwin99@members.leeds.ac.uk>
* io/gzio.c (test_header): Align buf.
diff -purN -x '*.mk' -x '*~' -x autom4te.cache -x configure -x '.#*' -x '*.orig' -x CVS grub2-submitted/io/gzio.c grub2-split/io/gzio.c
--- grub2-submitted/io/gzio.c 2005-08-22 18:28:59.000000000 +0100
+++ grub2-split/io/gzio.c 2005-10-14 22:49:55.000000000 +0100
@@ -165,7 +165,7 @@ typedef unsigned long ulg;
static int
test_header (grub_file_t file)
{
- unsigned char buf[10];
+ unsigned char buf[10] __attribute__((aligned));
grub_gzio_t gzio = file->data;
if (grub_file_tell (gzio->file) != 0)
Problem 2: Reading gzip files all at once results in corruption of the data.
It can be exposed by this patch:
diff -purN -x '*.mk' -x '*~' -x autom4te.cache -x configure -x '.#*' -x '*.orig' -x CVS grub2-submitted/commands/cmp.c grub2-split/commands/cmp.c
--- grub2-submitted/commands/cmp.c 2005-08-26 09:04:11.000000000 +0100
+++ grub2-split/commands/cmp.c 2005-10-15 00:44:12.000000000 +0100
@@ -26,7 +26,7 @@
#include <grub/mm.h>
#include <grub/gzio.h>
-#define BUFFER_SIZE 512
+#define BUFFER_SIZE 1024*1024*4
static grub_err_t
grub_cmd_cmp (struct grub_arg_list *state __attribute__ ((unused)),
Test file at:
http://www.majoroak.f2s.com/tim/grub/downloads/grub2-20051007.tar.gz
Valgrind output attached.
--
Member AFFS, WYLUG, SWP (UK), UAF, RESPECT, StWC
No to software patents! Victory to the iraqi resistance!
[-- Attachment #1.2: valgrind --]
[-- Type: text/plain, Size: 7106 bytes --]
==5120== Memcheck, a memory error detector.
==5120== Copyright (C) 2002-2005, and GNU GPL'd, by Julian Seward et al.
==5120== Using LibVEX rev 1367, a library for dynamic binary translation.
==5120== Copyright (C) 2004-2005, and GNU GPL'd, by OpenWorks LLP.
==5120== Using valgrind-3.0.1, a dynamic binary instrumentation framework.
==5120== Copyright (C) 2000-2005, and GNU GPL'd, by Julian Seward et al.
==5120== For more details, rerun with: -v
==5120==
--5120-- DWARF2 CFI reader: unhandled CFI instruction 0:50
--5120-- DWARF2 CFI reader: unhandled CFI instruction 0:50
==5120== Conditional jump or move depends on uninitialised value(s)
==5120== at 0x1B8EC7D3: (within /lib/ld-2.3.5.so)
==5120== by 0x1B8E631C: (within /lib/ld-2.3.5.so)
==5120== by 0x1B8F2BDD: (within /lib/ld-2.3.5.so)
==5120== by 0x1B8E7675: (within /lib/ld-2.3.5.so)
==5120== by 0x1B8E47C6: (within /lib/ld-2.3.5.so)
==5120==
==5120== Conditional jump or move depends on uninitialised value(s)
==5120== at 0x1B8ECB13: (within /lib/ld-2.3.5.so)
==5120== by 0x1B8E631C: (within /lib/ld-2.3.5.so)
==5120== by 0x1B8F2BDD: (within /lib/ld-2.3.5.so)
==5120== by 0x1B8E7675: (within /lib/ld-2.3.5.so)
==5120== by 0x1B8E47C6: (within /lib/ld-2.3.5.so)
==5120==
==5120== Conditional jump or move depends on uninitialised value(s)
==5120== at 0x1B8EC6B6: (within /lib/ld-2.3.5.so)
==5120== by 0x1B8E6376: (within /lib/ld-2.3.5.so)
==5120== by 0x1B8F2BDD: (within /lib/ld-2.3.5.so)
==5120== by 0x1B8E7675: (within /lib/ld-2.3.5.so)
==5120== by 0x1B8E47C6: (within /lib/ld-2.3.5.so)
==5120==
==5120== Conditional jump or move depends on uninitialised value(s)
==5120== at 0x1B8EC6C2: (within /lib/ld-2.3.5.so)
==5120== by 0x1B8E6376: (within /lib/ld-2.3.5.so)
==5120== by 0x1B8F2BDD: (within /lib/ld-2.3.5.so)
==5120== by 0x1B8E7675: (within /lib/ld-2.3.5.so)
==5120== by 0x1B8E47C6: (within /lib/ld-2.3.5.so)
==5120==
==5120== Conditional jump or move depends on uninitialised value(s)
==5120== at 0x1B8EC7D3: (within /lib/ld-2.3.5.so)
==5120== by 0x1B8E6376: (within /lib/ld-2.3.5.so)
==5120== by 0x1B8F2BDD: (within /lib/ld-2.3.5.so)
==5120== by 0x1B8E7675: (within /lib/ld-2.3.5.so)
==5120== by 0x1B8E47C6: (within /lib/ld-2.3.5.so)
==5120== Warning: noted but unhandled ioctl 0x1261 with no size/direction hints
==5120== This could cause spurious value errors to appear.
==5120== See README_MISSING_SYSCALL_OR_IOCTL for guidance on writing a proper wrapper.
==5120== Warning: noted but unhandled ioctl 0x1261 with no size/direction hints
==5120== This could cause spurious value errors to appear.
==5120== See README_MISSING_SYSCALL_OR_IOCTL for guidance on writing a proper wrapper.
==5120== Warning: noted but unhandled ioctl 0x1261 with no size/direction hints
==5120== This could cause spurious value errors to appear.
==5120== See README_MISSING_SYSCALL_OR_IOCTL for guidance on writing a proper wrapper.
==5120==
==5120== Invalid read of size 1
==5120== at 0x80567DA: memcpy (misc.c:42)
==5120== by 0x80534FC: grub_gzio_read (gzio.c:1200)
==5120== by 0x8055FBC: grub_file_read (file.c:125)
==5120== by 0x80498BE: grub_cmd_cmp (cmp.c:69)
==5120== by 0x805A939: grub_command_execute (command.c:249)
==5120== by 0x805A3C3: grub_cmdline_run (cmdline.c:163)
==5120== by 0x805C92D: grub_menu_run (menu.c:327)
==5120== by 0x805BD1E: grub_normal_execute (main.c:485)
==5120== by 0x805BF1D: grub_enter_normal_mode (main.c:302)
==5120== by 0x805BF8D: grub_rescue_cmd_normal (main.c:512)
==5120== by 0x8058A27: grub_enter_rescue_mode (rescue.c:591)
==5120== by 0x805FB14: main (grub-emu.c:226)
==5120== Address 0x1BAEAB8F is 1 bytes before a block of size 4096 alloc'd
==5120== at 0x1B8FF8A6: malloc (vg_replace_malloc.c:149)
==5120== by 0x805FC4E: xmalloc (misc.c:71)
==5120== by 0x805484D: grub_disk_read (disk.c:160)
==5120== by 0x804BB27: grub_fat_read_data (fat.c:421)
==5120== by 0x8055FBC: grub_file_read (file.c:125)
==5120== by 0x80498A6: grub_cmd_cmp (cmp.c:68)
==5120== by 0x805A939: grub_command_execute (command.c:249)
==5120== by 0x805A3C3: grub_cmdline_run (cmdline.c:163)
==5120== by 0x805C92D: grub_menu_run (menu.c:327)
==5120== by 0x805BD1E: grub_normal_execute (main.c:485)
==5120== by 0x805BF1D: grub_enter_normal_mode (main.c:302)
==5120== by 0x805BF8D: grub_rescue_cmd_normal (main.c:512)
==5120==
==5120== Invalid read of size 1
==5120== at 0x80567D0: memcpy (misc.c:43)
==5120== by 0x80534FC: grub_gzio_read (gzio.c:1200)
==5120== by 0x8055FBC: grub_file_read (file.c:125)
==5120== by 0x80498BE: grub_cmd_cmp (cmp.c:69)
==5120== by 0x805A939: grub_command_execute (command.c:249)
==5120== by 0x805A3C3: grub_cmdline_run (cmdline.c:163)
==5120== by 0x805C92D: grub_menu_run (menu.c:327)
==5120== by 0x805BD1E: grub_normal_execute (main.c:485)
==5120== by 0x805BF1D: grub_enter_normal_mode (main.c:302)
==5120== by 0x805BF8D: grub_rescue_cmd_normal (main.c:512)
==5120== by 0x8058A27: grub_enter_rescue_mode (rescue.c:591)
==5120== by 0x805FB14: main (grub-emu.c:226)
==5120== Address 0x1BAEAB8E is 2 bytes before a block of size 4096 alloc'd
==5120== at 0x1B8FF8A6: malloc (vg_replace_malloc.c:149)
==5120== by 0x805FC4E: xmalloc (misc.c:71)
==5120== by 0x805484D: grub_disk_read (disk.c:160)
==5120== by 0x804BB27: grub_fat_read_data (fat.c:421)
==5120== by 0x8055FBC: grub_file_read (file.c:125)
==5120== by 0x80498A6: grub_cmd_cmp (cmp.c:68)
==5120== by 0x805A939: grub_command_execute (command.c:249)
==5120== by 0x805A3C3: grub_cmdline_run (cmdline.c:163)
==5120== by 0x805C92D: grub_menu_run (menu.c:327)
==5120== by 0x805BD1E: grub_normal_execute (main.c:485)
==5120== by 0x805BF1D: grub_enter_normal_mode (main.c:302)
==5120== by 0x805BF8D: grub_rescue_cmd_normal (main.c:512)
==5120==
==5120== More than 30000 total errors detected. I'm not reporting any more.
==5120== Final error counts will be inaccurate. Go fix your program!
==5120== Rerun with --error-limit=no to disable this cutoff. Note
==5120== that errors may occur in your program without prior warning from
==5120== Valgrind, because errors are no longer being displayed.
==5120==
==5120==
==5120== ERROR SUMMARY: 30000 errors from 7 contexts (suppressed: 0 from 0)
==5120== malloc/free: in use at exit: 2874558 bytes in 1196 blocks.
==5120== malloc/free: 7013 allocs, 5817 frees, 33235044 bytes allocated.
==5120== For counts of detected errors, rerun with: -v
==5120== searching for pointers to 1196 not-freed blocks.
==5120== checked 2965380 bytes.
==5120==
==5120== LEAK SUMMARY:
==5120== definitely lost: 60619 bytes in 85 blocks.
==5120== possibly lost: 1664 bytes in 1 blocks.
==5120== still reachable: 2812275 bytes in 1110 blocks.
==5120== suppressed: 0 bytes in 0 blocks.
==5120== Use --leak-check=full to see details of leaked memory.
[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]
^ permalink raw reply [flat|nested] 8+ messages in thread* Re: gzio is broken
2005-10-15 0:47 gzio is broken Timothy Baldwin
@ 2005-10-15 8:07 ` Yoshinori K. Okuji
2005-10-15 17:34 ` Yoshinori K. Okuji
0 siblings, 1 reply; 8+ messages in thread
From: Yoshinori K. Okuji @ 2005-10-15 8:07 UTC (permalink / raw)
To: The development of GRUB 2
On Saturday 15 October 2005 02:47 am, Timothy Baldwin wrote:
> Problem 1: In the function test_header, the array buf is accessed via
> pointers to grub_uint32_t and grub_uint16_t without ensuring correct
> alignment of the pointers. This does not work on ARM. Fixed by:
Thank you for your report. I'll apply this patch.
> Problem 2: Reading gzip files all at once results in corruption of the
> data. It can be exposed by this patch:
Can you investigate this problem?
Okuji
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: gzio is broken
2005-10-15 8:07 ` Yoshinori K. Okuji
@ 2005-10-15 17:34 ` Yoshinori K. Okuji
2005-10-15 19:55 ` Timothy Baldwin
2005-10-16 9:24 ` gzio is broken (fix) Timothy Baldwin
0 siblings, 2 replies; 8+ messages in thread
From: Yoshinori K. Okuji @ 2005-10-15 17:34 UTC (permalink / raw)
To: The development of GRUB 2
On Saturday 15 October 2005 10:07 am, Yoshinori K. Okuji wrote:
> > Problem 2: Reading gzip files all at once results in corruption of the
> > data. It can be exposed by this patch:
>
> Can you investigate this problem?
I don't see any problem on my laptop (i386). So this might be specific to ARM.
Okuji
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: gzio is broken
2005-10-15 17:34 ` Yoshinori K. Okuji
@ 2005-10-15 19:55 ` Timothy Baldwin
2005-10-15 20:53 ` Marco Gerards
2005-10-16 9:24 ` gzio is broken (fix) Timothy Baldwin
1 sibling, 1 reply; 8+ messages in thread
From: Timothy Baldwin @ 2005-10-15 19:55 UTC (permalink / raw)
To: The development of GRUB 2
[-- Attachment #1: Type: text/plain, Size: 2128 bytes --]
On Saturday 15 Oct 2005 18:34, Yoshinori K. Okuji wrote:
> On Saturday 15 October 2005 10:07 am, Yoshinori K. Okuji wrote:
> > > Problem 2: Reading gzip files all at once results in corruption of the
> > > data. It can be exposed by this patch:
> >
> > Can you investigate this problem?
>
> I don't see any problem on my laptop (i386). So this might be specific to
> ARM.
It isn't, the problem shows up on my i386 desktop in grub-emu. On i386 at least,
it shows up with a clean source + patch from my last email:
grub> cmp /grub2-20051007.tar /grub2-20051007.tar.gz
Compare `/grub2-20051007.tar' and `/grub2-20051007.tar.gz':
Differ at the offset 65536: 0x63 [/grub2-20051007.tar], 0x0 [/grub2-20051007.tar.gz]
I've tried with all these compilers:
$ gcc --version
gcc (GCC) 4.0.2 (Debian 4.0.2-2)
Copyright (C) 2005 Free Software Foundation, Inc.
This is free software; see the source for copying conditions. There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
$ /usr/bin/gcc-3.4 --version
gcc-3.4 (GCC) 3.4.5 20050821 (prerelease) (Debian 3.4.4-8)
Copyright (C) 2004 Free Software Foundation, Inc.
This is free software; see the source for copying conditions. There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
$ /usr/bin/gcc-3.3 --version
gcc-3.3 (GCC) 3.3.6 (Debian 1:3.3.6-10)
Copyright (C) 2003 Free Software Foundation, Inc.
This is free software; see the source for copying conditions. There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
I have double checked that the files are identical after decompression, and
the cmp command works correctly with uncompressed files.
The SHA1 hashes of the test data are:
2ca1f322c8d030bbe661de37eddfbe846c9b935e /mnt/grub2-20051007.tar
fde79ad954b655bbde711684b3a403919928fbb3 /mnt/grub2-20051007.tar.gz
And it is still at:
http://www.majoroak.f2s.com/tim/grub/downloads/grub2-20051007.tar.gz
--
Member AFFS, WYLUG, SWP (UK), UAF, RESPECT, StWC
No to software patents! Victory to the iraqi resistance!
[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: gzio is broken
2005-10-15 19:55 ` Timothy Baldwin
@ 2005-10-15 20:53 ` Marco Gerards
2005-10-15 23:05 ` Timothy Baldwin
0 siblings, 1 reply; 8+ messages in thread
From: Marco Gerards @ 2005-10-15 20:53 UTC (permalink / raw)
To: The development of GRUB 2
Timothy Baldwin <tim.lists@majoroak.f2s.com> writes:
> On Saturday 15 Oct 2005 18:34, Yoshinori K. Okuji wrote:
>> On Saturday 15 October 2005 10:07 am, Yoshinori K. Okuji wrote:
>> > > Problem 2: Reading gzip files all at once results in corruption of the
>> > > data. It can be exposed by this patch:
>> >
>> > Can you investigate this problem?
>>
>> I don't see any problem on my laptop (i386). So this might be specific to
>> ARM.
>
> It isn't, the problem shows up on my i386 desktop in grub-emu. On i386 at least,
> it shows up with a clean source + patch from my last email:
>
> grub> cmp /grub2-20051007.tar /grub2-20051007.tar.gz
> Compare `/grub2-20051007.tar' and `/grub2-20051007.tar.gz':
> Differ at the offset 65536: 0x63 [/grub2-20051007.tar], 0x0 [/grub2-20051007.tar.gz]
This could even be a filesystem bug or disk bug. Can you try copying
these files so two new files are created and compare those. In that
case you could see if the difference is on the same position.
--
Marco
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: gzio is broken
2005-10-15 20:53 ` Marco Gerards
@ 2005-10-15 23:05 ` Timothy Baldwin
0 siblings, 0 replies; 8+ messages in thread
From: Timothy Baldwin @ 2005-10-15 23:05 UTC (permalink / raw)
To: The development of GRUB 2
[-- Attachment #1: Type: text/plain, Size: 3213 bytes --]
On Saturday 15 Oct 2005 21:53, Marco Gerards wrote:
> Timothy Baldwin <tim.lists@majoroak.f2s.com> writes:
> > On Saturday 15 Oct 2005 18:34, Yoshinori K. Okuji wrote:
> >> On Saturday 15 October 2005 10:07 am, Yoshinori K. Okuji wrote:
> >> > > Problem 2: Reading gzip files all at once results in corruption of
> >> > > the data. It can be exposed by this patch:
> >> >
> >> > Can you investigate this problem?
> >>
> >> I don't see any problem on my laptop (i386). So this might be specific
> >> to ARM.
> >
> > It isn't, the problem shows up on my i386 desktop in grub-emu. On i386 at
> > least, it shows up with a clean source + patch from my last email:
> >
> > grub> cmp /grub2-20051007.tar /grub2-20051007.tar.gz
> > Compare `/grub2-20051007.tar' and `/grub2-20051007.tar.gz':
> > Differ at the offset 65536: 0x63 [/grub2-20051007.tar], 0x0
> > [/grub2-20051007.tar.gz]
>
> This could even be a filesystem bug or disk bug. Can you try copying
> these files so two new files are created and compare those. In that
> case you could see if the difference is on the same position.
That was a fat filesystem. The only common filesystem so far is Linux RieserFS
and software raid. So I copied the gzipped files over to the Filecore FS on
the RiscPC, and uncompressed them there. Still an incorrect zero at byte
65536.
It often crashes on both systems, but not when running under valgrind.
Finally, valgrind points out a problem in grub_gzio_read:
==2456==
==2456== Invalid read of size 1
==2456== at 0x8057522: memcpy (misc.c:43)
==2456== by 0x8054826: grub_gzio_read (gzio.c:1200)
==2456== by 0x8056D2C: grub_file_read (file.c:125)
==2456== by 0x8049832: grub_cmd_cmp (cmp.c:68)
==2456== by 0x805B51F: grub_command_execute (command.c:252)
==2456== by 0x805AF93: grub_cmdline_run (cmdline.c:163)
==2456== by 0x805C821: grub_normal_execute (main.c:490)
==2456== by 0x805CA6D: grub_enter_normal_mode (main.c:302)
==2456== by 0x805CAF8: grub_rescue_cmd_normal (main.c:512)
==2456== by 0x8059910: grub_enter_rescue_mode (rescue.c:591)
==2456== by 0x8060485: main (grub-emu.c:228)
==2456== Address 0x1BA46363 is 3683 bytes inside a block of size 4096 free'd
==2456== at 0x1B9003C3: free (vg_replace_malloc.c:235)
==2456== by 0x805570A: grub_disk_read (disk.c:454)
==2456== by 0x804C3A0: grub_fat_read_data (fat.c:373)
==2456== by 0x804CB81: grub_fat_read (fat.c:720)
==2456== by 0x8056D2C: grub_file_read (file.c:125)
==2456== by 0x8053DF2: grub_gzio_open (gzio.c:209)
==2456== by 0x8053ED7: grub_gzfile_open (gzio.c:1158)
==2456== by 0x804973D: grub_cmd_cmp (cmp.c:49)
==2456== by 0x805B51F: grub_command_execute (command.c:252)
==2456== by 0x805AF93: grub_cmdline_run (cmdline.c:163)
==2456== by 0x805C821: grub_normal_execute (main.c:490)
==2456== by 0x805CA6D: grub_enter_normal_mode (main.c:302)
==2456==
==2456== More than 30000 total errors detected. I'm not reporting any more.
==2456== Final error counts will be inaccurate. Go fix your program!
--
Member AFFS, WYLUG, SWP (UK), UAF, RESPECT, StWC
No to software patents! Victory to the iraqi resistance!
[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: gzio is broken (fix)
2005-10-15 17:34 ` Yoshinori K. Okuji
2005-10-15 19:55 ` Timothy Baldwin
@ 2005-10-16 9:24 ` Timothy Baldwin
2005-10-19 23:17 ` Yoshinori K. Okuji
1 sibling, 1 reply; 8+ messages in thread
From: Timothy Baldwin @ 2005-10-16 9:24 UTC (permalink / raw)
To: The development of GRUB 2
[-- Attachment #1: Type: text/plain, Size: 1197 bytes --]
On Saturday 15 Oct 2005 18:34, Yoshinori K. Okuji wrote:
> On Saturday 15 October 2005 10:07 am, Yoshinori K. Okuji wrote:
> > > Problem 2: Reading gzip files all at once results in corruption of the
> > > data. It can be exposed by this patch:
> >
> > Can you investigate this problem?
The patch fixes it:
diff -u -p -r1.2 gzio.c
--- grub2/io/gzio.c 15 Oct 2005 09:22:31 -0000 1.2
+++ grub2/io/gzio.c 16 Oct 2005 08:55:38 -0000
@@ -1193,7 +1193,7 @@ grub_gzio_read (grub_file_t file, char *
inflate_window (file);
srcaddr = (char *) ((offset & (WSIZE - 1)) + gzio->slide);
- size = gzio->saved_offset - file->offset;
+ size = gzio->saved_offset - offset;
if (size > len)
size = len;
2005-10-16 Timothy Baldwin <T.E.Baldwin99@members.leeds.ac.uk>
* io/gzio.c (grub_gzio_read): Use offset instead of file->offset
consistently within loop.
> I don't see any problem on my laptop (i386).
Did you apply my test patch and use a test file with an uncompressed size
larger than 32KiB?
--
Member AFFS, WYLUG, SWP (UK), UAF, RESPECT, StWC
No to software patents! Victory to the iraqi resistance!
[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]
^ permalink raw reply [flat|nested] 8+ messages in thread
end of thread, other threads:[~2005-10-19 23:18 UTC | newest]
Thread overview: 8+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2005-10-15 0:47 gzio is broken Timothy Baldwin
2005-10-15 8:07 ` Yoshinori K. Okuji
2005-10-15 17:34 ` Yoshinori K. Okuji
2005-10-15 19:55 ` Timothy Baldwin
2005-10-15 20:53 ` Marco Gerards
2005-10-15 23:05 ` Timothy Baldwin
2005-10-16 9:24 ` gzio is broken (fix) Timothy Baldwin
2005-10-19 23:17 ` Yoshinori K. Okuji
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.