From: Dave Chinner <david@fromorbit.com>
To: Ben Myers <bpm@sgi.com>
Cc: linux-fsdevel@vger.kernel.org, tinguely@sgi.com, olaf@sgi.com,
xfs@oss.sgi.com
Subject: Re: [PATCH 07/10] xfs: add trie generator and supporting code for UTF-8.
Date: Tue, 23 Sep 2014 06:57:14 +1000 [thread overview]
Message-ID: <20140922205714.GN4267@dastard> (raw)
In-Reply-To: <20140918201518.GJ4482@sgi.com>
On Thu, Sep 18, 2014 at 03:15:19PM -0500, Ben Myers wrote:
> From: Olaf Weber <olaf@sgi.com>
>
> mkutf8data.c is the source for a program that generates utf8data.h, which
> contains the trie that utf8norm.c uses. The trie is generated from the
> Unicode 7.0.0 data files. The format of the utf8data[] table is described
> in utf8norm.c.
>
> Supporting functions for UTF-8 normalization are in utf8norm.c with the
> header utf8norm.h. Two normalization forms are supported: nfkdi and nfkdicf.
>
> nfkdi:
> - Apply unicode normalization form NFKD.
> - Remove any Default_Ignorable_Code_Point.
>
> nfkdicf:
> - Apply unicode normalization form NFKD.
> - Remove any Default_Ignorable_Code_Point.
> - Apply a full casefold (C + F).
>
> For the purposes of the code, a string is valid UTF-8 if:
>
> - The values encoded are 0x1..0x10FFFF.
> - The surrogate codepoints 0xD800..0xDFFFF are not encoded.
> - The shortest possible encoding is used for all values.
>
> The supporting functions work on null-terminated strings (utf8 prefix) and
> on length-limited strings (utf8n prefix).
>
> Signed-off-by: Olaf Weber <olaf@sgi.com>
>
> ---
> [v2: the trie is now separated into utf8norm.ko;
> utf8version is now a function and exported;
> introduced CONFIG_XFS_UTF8. -bpm]
> ---
> fs/xfs/Kconfig | 8 +
> fs/xfs/Makefile | 2 +-
> fs/xfs/utf8norm/Makefile | 37 +
> fs/xfs/utf8norm/mkutf8data.c | 3239 ++++++++++++++++++++++++++++++++++++++++++
> fs/xfs/utf8norm/utf8norm.c | 649 +++++++++
> fs/xfs/utf8norm/utf8norm.h | 116 ++
Again, nothing XFS specific here. It's being built as a separate
module and the only thing that XFS uses are exported functions, so
it really should be generic library code....
Cheers,
Dave.
--
Dave Chinner
david@fromorbit.com
next prev parent reply other threads:[~2014-09-22 20:57 UTC|newest]
Thread overview: 65+ messages / expand[flat|nested] mbox.gz Atom feed top
2014-09-18 19:56 [RFC v2] Unicode/UTF-8 support for XFS Ben Myers
2014-09-18 20:08 ` [PATCH 01/10] xfs: return the first match during case-insensitive lookup Ben Myers
2014-09-18 20:09 ` [PATCH 02/10] xfs: rename XFS_CMP_CASE to XFS_CMP_MATCH Ben Myers
2014-09-18 20:09 ` [PATCH 03/13] libxfs: add xfs_nameops.normhash Ben Myers
2014-09-18 20:10 ` [PATCH 04/10] xfs: change interface of xfs_nameops.normhash Ben Myers
2014-09-18 20:11 ` [PATCH 05/10] xfs: add a superblock feature bit to indicate UTF-8 support Ben Myers
2014-09-18 20:13 ` [PATCH 03/10] xfs: add xfs_nameops.normhash Ben Myers
2014-09-18 20:14 ` [PATCH 06/10] xfs: add unicode character database files Ben Myers
2014-09-22 20:54 ` Dave Chinner
2014-09-26 17:09 ` Christoph Hellwig
2014-09-18 20:15 ` [PATCH 07/10] xfs: add trie generator and supporting code for UTF-8 Ben Myers
2014-09-22 20:57 ` Dave Chinner [this message]
2014-09-23 18:57 ` Ben Myers
2014-09-26 17:10 ` Christoph Hellwig
2014-09-18 20:16 ` [PATCH 08/10] xfs: add xfs_nameops for utf8 and utf8+casefold Ben Myers
2014-09-18 20:17 ` [PATCH 09/10] xfs: apply utf-8 normalization rules to user extended attribute names Ben Myers
2014-09-18 20:18 ` [PATCH 10/10] xfs: implement demand load of utf8norm.ko Ben Myers
2014-09-18 20:31 ` [PATCH 00/13] xfsprogs: Unicode/UTF-8 support for XFS Ben Myers
2014-09-18 20:33 ` [PATCH 01/13] libxfs: return the first match during case-insensitive lookup Ben Myers
2014-09-18 20:33 ` [PATCH 02/13] libxfs: rename XFS_CMP_CASE to XFS_CMP_MATCH Ben Myers
2014-09-18 20:34 ` [PATCH 03/13] libxfs: add xfs_nameops.normhash Ben Myers
2014-09-18 20:35 ` [PATCH 04/13] libxfs: change interface of xfs_nameops.normhash Ben Myers
2014-09-18 20:36 ` [PATCH 05/13] libxfs: add a superblock feature bit to indicate UTF-8 support Ben Myers
2014-09-18 20:37 ` [PATCH 06/13] xfsprogs: add unicode character database files Ben Myers
2014-09-18 20:38 ` [PATCH 07/13] libxfs: add trie generator and supporting code for UTF-8 Ben Myers
2014-09-18 20:38 ` [PATCH 08/13] libxfs: add xfs_nameops for utf8 and utf8+casefold Ben Myers
2014-09-18 20:39 ` [PATCH 09/13] libxfs: apply utf-8 normalization rules to user extended attribute names Ben Myers
2014-09-18 20:40 ` [PATCH 10/13] xfsprogs: add utf8 support to growfs Ben Myers
2014-09-18 20:41 ` [PATCH 11/13] xfsprogs: add utf8 support to mkfs.xfs Ben Myers
2014-09-18 20:42 ` [PATCH 12/13] xfsprogs: add utf8 support to xfs_repair Ben Myers
2014-09-18 20:43 ` [PATCH 13/13] xfsprogs: add a preliminary test for utf8 support Ben Myers
2014-09-19 16:06 ` [PATCH 07a/13] xfsprogs: add trie generator for UTF-8 Ben Myers
2014-09-23 18:34 ` Roger Willcocks
2014-09-24 23:11 ` Ben Myers
2014-09-19 16:07 ` [PATCH 07b/13] libxfs: add supporting code " Ben Myers
2014-09-18 21:10 ` [RFC v2] Unicode/UTF-8 support for XFS Ben Myers
2014-09-18 21:24 ` Zach Brown
2014-09-18 22:23 ` Ben Myers
2014-09-19 16:03 ` [PATCH 07a/10] xfs: add trie generator for UTF-8 Ben Myers
2014-09-19 16:04 ` [PATCH 07b/10] xfs: add supporting code " Ben Myers
2014-09-22 14:55 ` [RFC v2] Unicode/UTF-8 support for XFS Andi Kleen
2014-09-22 18:41 ` Ben Myers
2014-09-22 19:29 ` Andi Kleen
2014-09-23 16:13 ` Olaf Weber
2014-09-23 20:15 ` Andi Kleen
2014-09-23 20:45 ` Ben Myers
2014-09-24 11:07 ` Olaf Weber
2014-09-26 14:06 ` Olaf Weber
2014-09-23 13:01 ` Olaf Weber
2014-09-23 20:02 ` Andi Kleen
2014-09-22 22:26 ` Dave Chinner
2014-09-24 13:21 ` Olaf Weber
2014-09-24 23:10 ` Dave Chinner
2014-09-25 13:33 ` Zuckerman, Boris
2014-09-26 14:50 ` Olaf Weber
2014-09-26 16:56 ` Christoph Hellwig
2014-09-26 17:04 ` Jeremy Allison
2014-09-26 17:06 ` Christoph Hellwig
2014-09-26 17:13 ` Jeremy Allison
2014-09-26 19:37 ` Olaf Weber
2014-09-26 19:46 ` Jeremy Allison
2014-09-26 20:03 ` Olaf Weber
2014-09-29 20:16 ` J. Bruce Fields
2014-09-29 11:06 ` Christoph Hellwig
2014-09-26 17:30 ` Ben Myers
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20140922205714.GN4267@dastard \
--to=david@fromorbit.com \
--cc=bpm@sgi.com \
--cc=linux-fsdevel@vger.kernel.org \
--cc=olaf@sgi.com \
--cc=tinguely@sgi.com \
--cc=xfs@oss.sgi.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).