Re: [PATCH] UTF-8 to UTF-16 transformation

[Robert Millan <rmh@aybabtu.com>]

On Thu, Aug 27, 2009 at 11:31:28PM +0200, Vladimir 'phcoder' Serbinenko wrote:
> On Wed, Aug 26, 2009 at 2:31 AM, Robert Millan<rmh@aybabtu.com> wrote:
> > On Mon, Aug 24, 2009 at 09:23:22PM +0200, Vladimir 'phcoder' Serbinenko wrote:
> >
> >> 2009-08-24  Vladimir Serbinenko  <phcoder@gmail.com>
> >>
> >>       UTF-8 to UTF-16 transformation.
> >>
> >>       * conf/common.rmk (pkglib_MODULES): Add utf.mod
> >>       (utf_mod_SOURCES): New variable.
> >>       (utf_mod_CFLAGS): Likewise.
> >>       (utf_mod_LDFLAGS): Likewise.
> >>       * include/grub/utf.h: New file.
> >>       * lib/utf.c: New file. (Based on grub_utf8_to_ucs4 from kern/misc.c)
> >
> > Sounds like we could end up needing more of this (to other charsets), so
> > why not give this module a generic name to hint as to where it can be added?
> >
> I'm ok with renaming but whether a conversion goes to charset.mod is
> perhaps to be decided on case-by-case basis-
> > The conversion functions in kern/misc.c could eventually move there as well,
> > once UTF-8 support becomes optional in the kernel.
> utf16_to_utf8 can be moved now out of the kernel but it's used by some
> fs modules (e.g. fat). Perhaps utf16_to_utf8 should be a separate
> module? This would decrease the size of biggest cores with the price
> of its increase in smaller cores.

Uhm perhaps we should use inline functions then.  What do you think?

> >> +       if ((c & 0x80) == 0x00)
> >> +         code = c;
> >> +       else if ((c & 0xe0) == 0xc0)
> >
> > These should be macroified.
> >
> Actually this are accelerated bitchecks (bit numbers follow specific
> and easy pattern) and for real readability would have to be written in
> binary but AFAIK binary notation isn't supported in C code and would
> result in overly long strings

Ah, right.  Then it's not a problem.  Maybe with (1 << 7) instead of 0x80,
if you prefer.

-- 
Robert Millan

  The DRM opt-in fallacy: "Your data belongs to us. We will decide when (and
  how) you may access your data; but nobody's threatening your freedom: we
  still allow you to remove your data and not access it at all."