[PATCH] Don't discard valid UTF-8 following non-valid one

[PATCH] Don't discard valid UTF-8 following non-valid one

[Vladimir 'φ-coder/phcoder' Serbinenko]

[-- Attachment #1.1: Type: text/plain, Size: 170 bytes --]

This discarding is Unicode-uncompliant and can create problems like
accidental discarding of \0 in some cases

-- 
Regards
Vladimir 'φ-coder/phcoder' Serbinenko


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1.2: nodiscard.diff --]
[-- Type: text/x-diff; name="nodiscard.diff", Size: 1693 bytes --]

=== modified file 'ChangeLog'
--- ChangeLog	2010-01-31 00:26:11 +0000
+++ ChangeLog	2010-01-31 00:50:47 +0000
@@ -1,5 +1,11 @@
 2010-01-31  Vladimir Serbinenko  <phcoder@gmail.com>
 
+	* kern/misc.c (grub_utf8_to_ucs4): Don't eat valid characters preceeded
+	by non-valid ones.
+	* kern/term.c (grub_putchar): Likewise.
+
+2010-01-31  Vladimir Serbinenko  <phcoder@gmail.com>
+
 	* font/font.c (find_glyph): Check that bmp_idx is available before
 	using it.
 	(grub_font_get_string_width): Never call grub_font_get_glyph_internal

=== modified file 'kern/misc.c'
--- kern/misc.c	2010-01-20 08:12:47 +0000
+++ kern/misc.c	2010-01-31 00:35:13 +0000
@@ -975,6 +975,10 @@ grub_utf8_to_ucs4 (grub_uint32_t *dest, 
 	    {
 	      /* invalid */
 	      code = '?';
+	      /* Character c may be valid, don't eat it.  */
+	      src--;
+	      if (srcsize != (grub_size_t)-1)
+		srcsize++;
 	      count = 0;
 	    }
 	  else

=== modified file 'kern/term.c'
--- kern/term.c	2010-01-10 12:34:48 +0000
+++ kern/term.c	2010-01-31 00:49:09 +0000
@@ -57,16 +57,17 @@ grub_putchar (int c)
 {
   static grub_size_t size = 0;
   static grub_uint8_t buf[6];
+  grub_uint8_t *rest;
   grub_uint32_t code;
-  grub_size_t ret;
 
   buf[size++] = c;
-  ret = grub_utf8_to_ucs4 (&code, 1, buf, size, 0);
 
-  if (ret != 0)
+  while (grub_utf8_to_ucs4 (&code, 1, buf, size, (const grub_uint8_t **) &rest) 
+	 != 0)
     {
       struct grub_term_output *term;
-      size = 0;
+      size -= rest - buf;
+      grub_memmove (buf, rest, size);
       FOR_ACTIVE_TERM_OUTPUTS(term)
 	grub_putcode (code, term);
       if (code == '\n' && grub_newline_hook)


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 293 bytes --]