[Pcre-svn] [1241] code/trunk/maint: Tidies and updates to m…

Kezdőlap
Üzenet törlése
Szerző: Subversion repository
Dátum:  
Címzett: pcre-svn
Tárgy: [Pcre-svn] [1241] code/trunk/maint: Tidies and updates to maintenance programs utf8 and ucptest.
Revision: 1241
          http://www.exim.org/viewvc/pcre2?view=rev&revision=1241
Author:   ph10
Date:     2020-04-01 18:00:33 +0100 (Wed, 01 Apr 2020)
Log Message:
-----------
Tidies and updates to maintenance programs utf8 and ucptest.


Modified Paths:
--------------
    code/trunk/maint/README
    code/trunk/maint/ucptest.c
    code/trunk/maint/ucptestdata/testinput1
    code/trunk/maint/ucptestdata/testoutput1
    code/trunk/maint/utf8.c


Added Paths:
-----------
    code/trunk/maint/ucptestdata/testinput2
    code/trunk/maint/ucptestdata/testoutput2


Modified: code/trunk/maint/README
===================================================================
--- code/trunk/maint/README    2020-03-25 17:18:33 UTC (rev 1240)
+++ code/trunk/maint/README    2020-04-01 17:00:33 UTC (rev 1241)
@@ -54,10 +54,12 @@
 ucptest.c        A short C program for testing the Unicode property macros
                  that do lookups in the pcre2_ucd.c data, mainly useful after
                  rebuilding the Unicode property table. Compile and run this in
-                 the "maint" directory (see comments at its head).
+                 the "maint" directory (see comments at its head). This program 
+                 can also be used to find characters with specific properties. 


-ucptestdata      A directory containing two files, testinput1 and testoutput1,
-                 to use in conjunction with the ucptest program.
+ucptestdata      A directory containing four files, testinput{1,2} and
+                 testoutput{1,2}, for use in conjunction with the ucptest
+                 program.


 utf8.c           A short, freestanding C program for converting a Unicode code
                  point into a sequence of bytes in the UTF-8 encoding, and vice
@@ -65,7 +67,7 @@
                  outputs a list of the equivalent UTF-8 bytes. If its argument
                  is a sequence of concatenated UTF-8 bytes (e.g. e188b4) it
                  treats them as a UTF-8 character and outputs the equivalent
-                 code point in hex.
+                 code point in hex. See comments at its head for details.



Updating to a new Unicode release
@@ -96,9 +98,10 @@

The ucptest program can be compiled and used to check that the new tables in
pcre2_ucd.c work properly, using the data files in ucptestdata to check a
-number of test characters. The source file ucptest.c should also be updated
-whenever new Unicode script names are added, and adding a few tests for new
-scripts is a good idea.
+number of test characters. It used to be necessary to update the source
+ucptest.c whenever new Unicode scripts were added, but this is no longer
+required because that program now uses the lists in the PCRE2 source. However,
+adding a few tests for new scripts to the files in ucptestdata is a good idea.


Preparing for a PCRE2 release
@@ -437,4 +440,4 @@
Philip Hazel
Email local part: ph10
Email domain: cam.ac.uk
-Last updated: 03 June 2019
+Last updated: 01 April 2020

Modified: code/trunk/maint/ucptest.c
===================================================================
--- code/trunk/maint/ucptest.c    2020-03-25 17:18:33 UTC (rev 1240)
+++ code/trunk/maint/ucptest.c    2020-04-01 17:00:33 UTC (rev 1241)
@@ -16,28 +16,45 @@
 /* This is a hacked-up program for testing the Unicode properties tables of
 PCRE2. It can also be used for finding characters with certain properties.
 I wrote it to help with debugging PCRE, and have added things that I found
-useful, in a rather haphazard way. The code has never been "tidied" or checked
-for robustness.
+useful, in a rather haphazard way. The code has never been seriously tidied or
+checked for robustness, but it shouldn't now give compiler warnings.


-If there are arguments, they are a list of hexadecimal code points whose
-properties are to be output. Otherwise, the program expects to read commands on
-stdin, and it writes output to stdout. There are two commands:
+There is only one option: "-s". If given, it applies only to the "findprop"
+command. It causes the UTF-8 sequence of bytes that encode the character to be
+output between angle brackets at the end of the line. On a UTF-8 terminal, this
+will show the appropriate graphic for the code point.

-"findprop" must be followed by a list of Unicode code points as hex numbers
-(without any prefixes). The output is one line per character, giving its
-Unicode properties followed by its other case if there is one, followed by its
-Script Extension list if it is not just the same as the base script.
+If the command has arguments, they are concatenated into a buffer, separated by
+spaces. If the first argument starts "U+" or consists entirely of hexadecimal
+digits, "findprop" is inserted at the start. The buffer is then processed as a
+single line file, after which the program exits. If there are no arguments, the
+program reads commands line by line on stdin and writes output to stdout. The
+return code is always zero.

-"find" must be followed by a list of property names and their values. This
-finds characters that have those properties. If multiple properties are listed,
-they must all be matched. Currently supported:
+There are three commands:

+"findprop" must be followed by a space-separated list of Unicode code points as
+hex numbers, either without any prefix or starting with "U+". The output is one
+line per character, giving its Unicode properties followed by its other case or 
+cases if one or more exist, followed by its Script Extension list if it is not
+just the same as the base script. This list is in square brackets. The
+properties are:
+
+General type        e.g. Letter
+Specific type       e.g. Upper case letter
+Script              e.g. Medefaidrin
+Grapheme break type e.g. Extend (most common is Other)
+
+"find" must be followed by a list of property names and their values. The 
+values are case-sensitive. This finds characters that have those properties. If
+multiple properties are listed, they must all be matched. Currently supported:
+
   script <name>    The character must have this script property. Only one
                      such script may be given.
   scriptx <name>   This script must be in the character's Script Extension
                      property list. If this is used many times, all the given
                      scripts must be present.
-  type <abbrev>    The character's type (e.g. Lu or Nd) must match.
+  type <abbrev>    The character's specific type (e.g. Lu or Nd) must match.
   gbreak <name>    The grapheme break property must match.


If a <name> or <abbrev> is preceded by !, the value must NOT be present. For
@@ -44,9 +61,14 @@
Script Extensions, there may be a mixture of positive and negative
requirements. All must be satisfied.

-No more than 100 characters are output. If there are more, the list ends with
-... */
+Sequences of two or more characters are shown as ranges, for example
+U+0041..U+004A. No more than 100 lines are are output. If there are more
+characters, the list ends with ...

+"list" must be followed by a property name (script, type, or gbreak). The
+defined values for that property are listed. */
+
+
#ifdef HAVE_CONFIG_H
#include "../src/config.h"
#endif
@@ -91,231 +113,102 @@

/* -------------------------------------------------------------------*/

+static BOOL show_character = FALSE;

-const unsigned char *script_names[] = {
- US"Unknown",
- US"Arabic",
- US"Armenian",
- US"Bengali",
- US"Bopomofo",
- US"Braille",
- US"Buginese",
- US"Buhid",
- US"Canadian_Aboriginal",
- US"Cherokee",
- US"Common",
- US"Coptic",
- US"Cypriot",
- US"Cyrillic",
- US"Deseret",
- US"Devanagari",
- US"Ethiopic",
- US"Georgian",
- US"Glagolitic",
- US"Gothic",
- US"Greek",
- US"Gujarati",
- US"Gurmukhi",
- US"Han",
- US"Hangul",
- US"Hanunoo",
- US"Hebrew",
- US"Hiragana",
- US"Inherited",
- US"Kannada",
- US"Katakana",
- US"Kharoshthi",
- US"Khmer",
- US"Lao",
- US"Latin",
- US"Limbu",
- US"Linear_B",
- US"Malayalam",
- US"Mongolian",
- US"Myanmar",
- US"New_Tai_Lue",
- US"Ogham",
- US"Old_Italic",
- US"Old_Persian",
- US"Oriya",
- US"Osmanya",
- US"Runic",
- US"Shavian",
- US"Sinhala",
- US"Syloti_Nagri",
- US"Syriac",
- US"Tagalog",
- US"Tagbanwa",
- US"Tai_Le",
- US"Tamil",
- US"Telugu",
- US"Thaana",
- US"Thai",
- US"Tibetan",
- US"Tifinagh",
- US"Ugaritic",
- US"Yi",
- /* New for Unicode 5.0: */
- US"Balinese",
- US"Cuneiform",
- US"Nko",
- US"Phags_Pa",
- US"Phoenician",
- /* New for Unicode 5.1: */
- US"Carian",
- US"Cham",
- US"Kayah_Li",
- US"Lepcha",
- US"Lycian",
- US"Lydian",
- US"Ol_Chiki",
- US"Rejang",
- US"Saurashtra",
- US"Sundanese",
- US"Vai",
- /* New for Unicode 5.2: */
- US"Avestan",
- US"Bamum",
- US"Egyptian_Hieroglyphs",
- US"Imperial_Aramaic",
- US"Inscriptional_Pahlavi",
- US"Inscriptional_Parthian",
- US"Javanese",
- US"Kaithi",
- US"Lisu",
- US"Meetei_Mayek",
- US"Old_South_Arabian",
- US"Old_Turkic",
- US"Samaritan",
- US"Tai_Tham",
- US"Tai_Viet",
- /* New for Unicode 6.0.0 */
- US"Batak",
- US"Brahmi",
- US"Mandaic",
- /* New for Unicode 6.1.0 */
- US"Chakma",
- US"Meroitic_Cursive",
- US"Meroitic_Hieroglyphs",
- US"Miao",
- US"Sharada",
- US"Sora Sompent",
- US"Takri",
- /* New for Unicode 7.0.0 */
- US"Bassa_Vah",
- US"Caucasian_Albanian",
- US"Duployan",
- US"Elbasan",
- US"Grantha",
- US"Khojki",
- US"Khudawadi",
- US"Linear_A",
- US"Mahajani",
- US"Manichaean",
- US"Mende_Kikakui",
- US"Modi",
- US"Mro",
- US"Nabataean",
- US"Old_North_Arabian",
- US"Old_Permic",
- US"Pahawh_Hmong",
- US"Palmyrene",
- US"Psalter_Pahlavi",
- US"Pau_Cin_Hau",
- US"Siddham",
- US"Tirhuta",
- US"Warang_Citi",
- /* New for Unicode 8.0.0 */
- US"Ahom",
- US"Anatolian_Hieroglyphs",
- US"Hatran",
- US"Multani",
- US"Old_Hungarian",
- US"SignWriting",
- /* New for Unicode 10.0.0 (no update since 8.0.0) */
- US"Adlam",
- US"Bhaiksuki",
- US"Marchen",
- US"Newa",
- US"Osage",
- US"Tangut",
- US"Masaram_Gondi",
- US"Nushu",
- US"Soyombo",
- US"Zanabazar_Square",
- /* New for Unicode 11.0.0 */
- US"Dogra",
- US"Gunjala_Gondi",
- US"Hanifi_Rohingya",
- US"Makasar",
- US"Medefaidrin",
- US"Old_Sogdian",
- US"Sogdian",
- /* New for Unicode 12.0.0 */
- US"Elymaic",
- US"Nandinagari",
- US"Nyiakeng_Puachue_Hmong",
- US"Wancho",
- /* New for Unicode 13.0.0 */
- US"Chorasmian",
- US"Dives_Akuru",
- US"Khitan_Small_Script",
- US"Yezidi"
+static const unsigned char *type_names[] = {
+ US"Cc", US"Control",
+ US"Cf", US"Format",
+ US"Cn", US"Unassigned",
+ US"Co", US"Private use",
+ US"Cs", US"Surrogate",
+ US"Ll", US"Lower case letter",
+ US"Lm", US"Modifier letter",
+ US"Lo", US"Other letter",
+ US"Lt", US"Title case letter",
+ US"Lu", US"Upper case letter",
+ US"Mc", US"Spacing mark",
+ US"Me", US"Enclosing mark",
+ US"Mn", US"Non-spacing mark",
+ US"Nd", US"Decimal number",
+ US"Nl", US"Letter number",
+ US"No", US"Other number",
+ US"Pc", US"Connector punctuation",
+ US"Pd", US"Dash punctuation",
+ US"Pe", US"Close punctuation",
+ US"Pf", US"Final punctuation",
+ US"Pi", US"Initial punctuation",
+ US"Po", US"Other punctuation",
+ US"Ps", US"Open punctuation",
+ US"Sc", US"Currency symbol",
+ US"Sk", US"Modifier symbol",
+ US"Sm", US"Mathematical symbol",
+ US"So", US"Other symbol",
+ US"Zl", US"Line separator",
+ US"Zp", US"Paragraph separator",
+ US"Zs", US"Space separator"
};

-const unsigned char *type_names[] = {
-  US"Cc",
-  US"Cf",
-  US"Cn",
-  US"Co",
-  US"Cs",
-  US"Ll",
-  US"Lm",
-  US"Lo",
-  US"Lt",
-  US"Lu",
-  US"Mc",
-  US"Me",
-  US"Mn",
-  US"Nd",
-  US"Nl",
-  US"No",
-  US"Pc",
-  US"Pd",
-  US"Pe",
-  US"Pf",
-  US"Pi",
-  US"Po",
-  US"Ps",
-  US"Sc",
-  US"Sk",
-  US"Sm",
-  US"So",
-  US"Zl",
-  US"Zp",
-  US"Zs"
+static const unsigned char *gb_names[] = {
+  US"CR",                    US"carriage return",
+  US"LF",                    US"linefeed",
+  US"Control",               US"",
+  US"Extend",                US"",
+  US"Prepend",               US"",
+  US"SpacingMark",           US"",
+  US"L",                     US"Hangul syllable type L",
+  US"V",                     US"Hangul syllable type V",
+  US"T",                     US"Hangul syllable type T",
+  US"LV",                    US"Hangul syllable type LV",
+  US"LVT",                   US"Hangul syllable type LVT",
+  US"RegionalIndicator",     US"",
+  US"Other",                 US"",
+  US"ZWJ",                   US"zero width joiner",
+  US"Extended_Pictographic", US""
 };


-const unsigned char *gb_names[] = {
- US"CR",
- US"LF",
- US"Control",
- US"Extend",
- US"Prepend",
- US"SpacingMark",
- US"L",
- US"V",
- US"T",
- US"LV",
- US"LVT",
- US"RegionalIndicator",
- US"Other",
- US"ZWJ",
- US"Extended_Pictographic"
-};

+static const unsigned int utf8_table1[] = {
+ 0x0000007f, 0x000007ff, 0x0000ffff, 0x001fffff, 0x03ffffff, 0x7fffffff};

+static const int utf8_table2[] = {
+  0, 0xc0, 0xe0, 0xf0, 0xf8, 0xfc};
+
+
 /*************************************************
+*       Convert character value to UTF-8         *
+*************************************************/
+
+/* This function takes an unsigned long integer value in the range 0 -
+0x7fffffff and encodes it as a UTF-8 character in 1 to 6 bytes.
+
+Arguments:
+  cvalue     the character value
+  buffer     pointer to buffer for result - at least 6 bytes long
+
+Returns:     number of bytes placed in the buffer
+             0 if input code point is too big
+*/
+
+static size_t
+ord2utf8(unsigned int cvalue, unsigned char *buffer)
+{
+size_t i, j;
+for (i = 0; i < sizeof(utf8_table1)/sizeof(int); i++)
+  if (cvalue <= utf8_table1[i]) break;
+if (i >= sizeof(utf8_table1)/sizeof(int)) return 0;
+buffer += i;
+for (j = i; j > 0; j--)
+ {
+ *buffer-- = 0x80 | (cvalue & 0x3f);
+ cvalue >>= 6;
+ }
+*buffer = utf8_table2[i] | cvalue;
+return i + 1;
+}
+
+
+
+/*************************************************
 *             Test for interaction               *
 *************************************************/


@@ -331,11 +224,33 @@


 /*************************************************
+*      Get script name from ucp ident            *
+*************************************************/
+
+static const char *
+get_scriptname(int script)
+{
+size_t i;
+const ucp_type_table *u;
+
+for (i = 0; i < PRIV(utt_size); i++)
+  {
+  u = PRIV(utt) + i; 
+  if (u->type == PT_SC && u->value == script) break;
+  }
+if (i < PRIV(utt_size))
+  return PRIV(utt_names) + u->name_offset;
+  
+return "??";
+}  
+
+
+/*************************************************
 *      Print Unicode property info for a char    *
 *************************************************/


static void
-print_prop(int c)
+print_prop(unsigned int c, BOOL is_just_one)
{
int type = UCD_CATEGORY(c);
int fulltype = UCD_CHARTYPE(c);
@@ -342,17 +257,14 @@
int script = UCD_SCRIPT(c);
int scriptx = UCD_SCRIPTX(c);
int gbprop = UCD_GRAPHBREAK(c);
-int othercase = UCD_OTHERCASE(c);
+unsigned int othercase = UCD_OTHERCASE(c);
int caseset = UCD_CASESET(c);

const unsigned char *fulltypename = US"??";
const unsigned char *typename = US"??";
-const unsigned char *scriptname = US"??";
const unsigned char *graphbreak = US"??";
+const unsigned char *scriptname = CUS get_scriptname(script);

-if (script < sizeof(script_names)/sizeof(char *))
-  scriptname = script_names[script];
-
 switch (type)
   {
   case ucp_C: typename = US"Control"; break;
@@ -420,15 +332,18 @@
   default:                 graphbreak = US"Unknown"; break;
   }


-printf("%04x %s: %s, %s, %s", c, typename, fulltypename, scriptname, graphbreak);
-if (othercase != c)
+printf("U+%04X %s: %s, %s, %s", c, typename, fulltypename, scriptname, graphbreak);
+if (is_just_one && othercase != c)
   {
-  printf(", %04x", othercase);
+  printf(", U+%04X", othercase);
   if (caseset != 0)
     {
     const uint32_t *p = PRIV(ucd_caseless_sets) + caseset - 1;
     while (*(++p) < NOTACHAR)
-      if (*p != othercase && *p != c) printf(", %04x", *p);
+      {
+      unsigned int d = *p;  
+      if (d != othercase && d != c) printf(", U+%04X", d);
+      } 
     }
   }


@@ -436,25 +351,26 @@
   {
   printf(", [");
   if (scriptx >= 0)
-    {
-    scriptname = (scriptx >= sizeof(script_names)/sizeof(char *))?
-      US"??" : script_names[scriptx];
-    printf("%s", scriptname);
-    }
+    printf("%s", get_scriptname(scriptx));
   else
     {
-    char *sep = "";
+    const char *sep = "";
     const uint8_t *p = PRIV(ucd_script_sets) - scriptx;
     while (*p != 0)
       {
-      scriptname = (*p >= sizeof(script_names)/sizeof(char *))?
-        US"??" : script_names[*p++];
-      printf("%s%s", sep, scriptname);
+      printf("%s%s", sep, get_scriptname(*p++));
       sep = ", ";
       }
     }
   printf("]");
   }
+  
+if (show_character && is_just_one)
+  {
+  unsigned char buffer[8];
+  size_t len = ord2utf8(c, buffer);
+  printf(", >%.*s<", (int)len, buffer);  
+  }  


 printf("\n");
 }
@@ -483,7 +399,7 @@
 BOOL gbreak_not = FALSE;
 BOOL hadrange = FALSE;
 const ucd_record *ucd, *next_ucd;
-const char *pad = "      ";
+const char *pad = "        ";


 while (*s != 0)
   {
@@ -508,17 +424,20 @@
       offset = 1;
       }


-    for (i = 0; i < sizeof(script_names)/sizeof(char *); i++)
+    for (i = 0; i < PRIV(utt_size); i++)
       {
-      if (strcmp(CS value + offset, script_names[i]) == 0)
+      const ucp_type_table *u = PRIV(utt) + i; 
+      if (u->type == PT_SC && strcmp(CS(value + offset), 
+            PRIV(utt_names) + u->name_offset) == 0)
         {
+        c = u->value; 
         if (name[6] == 'x')
           {
-          scriptx_list[scriptx_count++] = scriptx_not? (-i):i;
+          scriptx_list[scriptx_count++] = scriptx_not? (-c):c;
           }
         else
           {
-          if (script < 0) script = i; else
+          if (script < 0) script = c; else
             {
             printf("** Only 1 script value allowed\n");
             return;
@@ -528,9 +447,9 @@
         }
       }


-    if (i >= sizeof(script_names)/sizeof(char *))
+    if (i >= PRIV(utt_size))
       {
-      printf("** Unrecognized script name '%s'\n", value);
+      printf("** Unrecognized script name \"%s\"\n", value);
       return;
       }
     }
@@ -550,17 +469,17 @@
         offset = 1;
         }


-      for (i = 0; i < sizeof(type_names)/sizeof(char *); i++)
+      for (i = 0; i < sizeof(type_names)/sizeof(char *); i += 2)
         {
-        if (strcmp(CS (value + offset), type_names[i]) == 0)
+        if (strcmp(CS (value + offset), CS type_names[i]) == 0)
           {
-          type = i;
+          type = i/2;
           break;
           }
         }
       if (i >= sizeof(type_names)/sizeof(char *))
         {
-        printf("** Unrecognized type name '%s'\n", value);
+        printf("** Unrecognized type name \"%s\"\n", value);
         return;
         }
       }
@@ -581,17 +500,17 @@
         offset = 1;
         }


-      for (i = 0; i < sizeof(gb_names)/sizeof(char *); i++)
+      for (i = 0; i < sizeof(gb_names)/sizeof(char *); i += 2)
         {
-        if (strcmp(CS (value + offset), gb_names[i]) == 0)
+        if (strcmp(CS (value + offset), CS gb_names[i]) == 0)
           {
-          gbreak = i;
+          gbreak = i/2;
           break;
           }
         }
       if (i >= sizeof(gb_names)/sizeof(char *))
         {
-        printf("** Unrecognized gbreak name '%s'\n", value);
+        printf("** Unrecognized gbreak name \"%s\"\n", value);
         return;
         }
       }
@@ -599,7 +518,7 @@


   else
     {
-    printf("** Unrecognized property name '%s'\n", name);
+    printf("** Unrecognized property name \"%s\"\n", name);
     return;
     }
   }
@@ -617,7 +536,7 @@
   if (scriptx_count > 0)
     {
     const uint8_t *char_scriptx = NULL;
-    int found = 0;
+    unsigned int found = 0;
     int scriptx = UCD_SCRIPTX(c);


     if (scriptx < 0) char_scriptx = PRIV(ucd_script_sets) - scriptx;
@@ -701,13 +620,13 @@


   if (--i > c)
     {
-    printf("%04x..", c);
+    printf("U+%04X..", c);
     c = i;
     hadrange = TRUE;
     }
   else if (hadrange) printf("%s", pad);


-  print_prop(c);
+  print_prop(c, FALSE);
   if (c >= 0x100000) pad = "        ";
     else if (c >= 0x10000) pad = "       ";
   count++;
@@ -723,6 +642,101 @@



 /*************************************************
+*        Process command line                    *
+*************************************************/
+
+static void
+process_command_line(unsigned char *buffer)
+{
+unsigned char *s, *t;
+unsigned char name[24];
+
+s = buffer;
+while (isspace(*s)) s++;
+if (*s == 0) return;
+
+for (t = name; *s != 0 && !isspace(*s); s++) *t++ = *s;
+*t = 0;
+while (isspace(*s)) s++;
+
+if (strcmp(CS name, "findprop") == 0)
+  {
+  while (*s != 0)
+    {
+    unsigned int c; 
+    unsigned char *endptr;
+    t = s; 
+    if (strncmp(CS t, "U+", 2) == 0) t += 2;
+    c = strtoul(CS t, CSS(&endptr), 16);
+    if (*endptr != 0 && !isspace(*endptr))
+      {
+      while (*endptr != 0 && !isspace(*endptr)) endptr++;
+      printf("** Invalid hex number: ignored \"%.*s\"\n", (int)(endptr-s), s);
+      }
+    else  
+      {
+      if (c > 0x10ffff) 
+        printf("** U+%x is too big for a Unicode code point\n", c);
+      else   
+        print_prop(c, TRUE);
+      } 
+    s = endptr;
+    while (isspace(*s)) s++;
+    }
+  }
+
+else if (strcmp(CS name, "find") == 0)
+  {
+  find_chars(s);
+  }
+  
+else if (strcmp(CS name, "list") == 0)
+  {
+  while (*s != 0)
+    {
+    size_t i;
+    for (t = name; *s != 0 && !isspace(*s); s++) *t++ = *s;
+    *t = 0;
+    while (isspace(*s)) s++;
+    
+    if (strcmp(CS name, "script") == 0 || strcmp(CS name, "scripts") == 0)
+      {
+      for (i = 0; i < PRIV(utt_size); i++) 
+        if (PRIV(utt)[i].type == PT_SC)
+          printf("%s\n", PRIV(utt_names) + PRIV(utt)[i].name_offset);  
+      }
+      
+    else if (strcmp(CS name, "type") == 0 || strcmp(CS name, "types") == 0)
+      {
+      for (i = 0; i < sizeof(type_names)/sizeof(char *); i += 2)
+        printf("%s %s\n", type_names[i], type_names[i+1]); 
+      }  
+      
+    else if (strcmp(CS name, "gbreak") == 0 || strcmp(CS name, "gbreaks") == 0)
+      {
+      for (i = 0; i < sizeof(gb_names)/sizeof(char *); i += 2)
+        {
+        if (gb_names[i+1][0] != 0)  
+          printf("%-3s (%s)\n", gb_names[i], gb_names[i+1]);
+        else   
+          printf("%s\n", gb_names[i]);
+        } 
+      }    
+
+    else 
+      {
+      printf("** Unknown property \"%s\"\n", name);  
+      break;
+      }  
+    }  
+  }  
+
+else printf("** Unknown test command \"%s\"\n", name);
+}
+
+
+
+/*************************************************
 *               Main program                     *
 *************************************************/


@@ -730,19 +744,42 @@
main(int argc, char **argv)
{
BOOL interactive;
+int first_arg = 1;
unsigned char buffer[1024];

-if (argc > 1)
+if (argc > 1 && strcmp(argv[1], "-s") == 0)
   {
+  show_character = TRUE;
+  first_arg++;
+  }   
+
+if (argc > first_arg)
+  {
   int i;
-  for (i = 1; i < argc; i++)
+  BOOL hexfirst = TRUE; 
+  char *arg = argv[first_arg]; 
+  unsigned char *s = buffer;
+  
+  if (strncmp(arg, "U+", 2) != 0 && !isdigit(*arg)) 
     {
-    unsigned char *endptr;
-    int c = strtoul(argv[i], CSS(&endptr), 16);
-    if (*endptr != 0)
-      printf("** Hex number expected; ignored '%s'\n", argv[i]);
-    else print_prop(c);
+    while (*arg != 0) 
+      {
+      if (!isxdigit(*arg++)) { hexfirst = FALSE; break; }  
+      } 
+    } 
+     
+  if (hexfirst)
+    {
+    strcpy(CS s, "findprop ");
+    s += 9;
     }
+    
+  for (i = first_arg; i < argc; i++)
+    {
+    s += sprintf(CS s, "%s ", argv[i]);       
+    }
+
+  process_command_line(buffer);
   return 0;
   }


@@ -754,17 +791,14 @@

 for(;;)
   {
-  unsigned char name[24];
-  unsigned char *s, *t;
-
 #if defined(SUPPORT_LIBREADLINE) || defined(SUPPORT_LIBEDIT)
   if (interactive)
     {
     size_t len;
-    s = readline("> ");
+    unsigned char *s = US readline("> ");
     if (s == NULL) break;
-    len = strlen(s);
-    if (len > 0) add_history(s);
+    len = strlen(CS s);
+    if (len > 0) add_history(CS s);
     memcpy(buffer, s, len);
     buffer[len] = '\n';
     buffer[len+1] = 0;
@@ -778,39 +812,8 @@
     if (fgets(CS buffer, sizeof(buffer), stdin) == NULL) break;
     if (!interactive) printf("%s", buffer);
     }
-
-  s = buffer;
-  while (isspace(*s)) s++;
-  if (*s == 0) continue;
-
-  for (t = name; *s != 0 && !isspace(*s); s++) *t++ = *s;
-  *t = 0;
-  while (isspace(*s)) s++;
-
-  if (strcmp(CS name, "findprop") == 0)
-    {
-    while (*s != 0)
-      {
-      unsigned char *endptr;
-      int c = strtoul(CS s, CSS(&endptr), 16);
-
-      if (*endptr != 0 && !isspace(*endptr))
-        {
-        while (*endptr != 0 && !isspace(*endptr)) endptr++;
-        printf("** Hex number expected; ignored '%.*s'\n", endptr-s, s);
-        }
-      else  print_prop(c);
-      s = endptr;
-      while (isspace(*s)) s++;
-      }
-    }
-
-  else if (strcmp(CS name, "find") == 0)
-    {
-    find_chars(s);
-    }
-
-  else printf("** Unknown test command %s\n", name);
+    
+  process_command_line(buffer);
   }


if (interactive) printf("\n");

Modified: code/trunk/maint/ucptestdata/testinput1
===================================================================
--- code/trunk/maint/ucptestdata/testinput1    2020-03-25 17:18:33 UTC (rev 1240)
+++ code/trunk/maint/ucptestdata/testinput1    2020-04-01 17:00:33 UTC (rev 1241)
@@ -45,4 +45,4 @@


findprop 1f16d

-findprop 10e93 10eaa
+findprop U+10e93 U+10eaa

Added: code/trunk/maint/ucptestdata/testinput2
===================================================================
--- code/trunk/maint/ucptestdata/testinput2                            (rev 0)
+++ code/trunk/maint/ucptestdata/testinput2    2020-04-01 17:00:33 UTC (rev 1241)
@@ -0,0 +1,5 @@
+find script Han
+find type Pe script Common scriptx Hangul
+find type Sk
+find type Pd
+find gbreak LVT


Modified: code/trunk/maint/ucptestdata/testoutput1
===================================================================
--- code/trunk/maint/ucptestdata/testoutput1    2020-03-25 17:18:33 UTC (rev 1240)
+++ code/trunk/maint/ucptestdata/testoutput1    2020-04-01 17:00:33 UTC (rev 1241)
@@ -1,398 +1,398 @@
 findprop 00 01 02 03 04 05 06 07 08 09 0a 0b 0c 0d 0e 0f 
-0000 Control: Control, Common, Control
-0001 Control: Control, Common, Control
-0002 Control: Control, Common, Control
-0003 Control: Control, Common, Control
-0004 Control: Control, Common, Control
-0005 Control: Control, Common, Control
-0006 Control: Control, Common, Control
-0007 Control: Control, Common, Control
-0008 Control: Control, Common, Control
-0009 Control: Control, Common, Control
-000a Control: Control, Common, LF
-000b Control: Control, Common, Control
-000c Control: Control, Common, Control
-000d Control: Control, Common, CR
-000e Control: Control, Common, Control
-000f Control: Control, Common, Control
+U+0000 Control: Control, Common, Control
+U+0001 Control: Control, Common, Control
+U+0002 Control: Control, Common, Control
+U+0003 Control: Control, Common, Control
+U+0004 Control: Control, Common, Control
+U+0005 Control: Control, Common, Control
+U+0006 Control: Control, Common, Control
+U+0007 Control: Control, Common, Control
+U+0008 Control: Control, Common, Control
+U+0009 Control: Control, Common, Control
+U+000A Control: Control, Common, LF
+U+000B Control: Control, Common, Control
+U+000C Control: Control, Common, Control
+U+000D Control: Control, Common, CR
+U+000E Control: Control, Common, Control
+U+000F Control: Control, Common, Control
 findprop 10 11 12 13 14 15 16 17 18 19 1a 1b 1c 1d 1e 1f 
-0010 Control: Control, Common, Control
-0011 Control: Control, Common, Control
-0012 Control: Control, Common, Control
-0013 Control: Control, Common, Control
-0014 Control: Control, Common, Control
-0015 Control: Control, Common, Control
-0016 Control: Control, Common, Control
-0017 Control: Control, Common, Control
-0018 Control: Control, Common, Control
-0019 Control: Control, Common, Control
-001a Control: Control, Common, Control
-001b Control: Control, Common, Control
-001c Control: Control, Common, Control
-001d Control: Control, Common, Control
-001e Control: Control, Common, Control
-001f Control: Control, Common, Control
+U+0010 Control: Control, Common, Control
+U+0011 Control: Control, Common, Control
+U+0012 Control: Control, Common, Control
+U+0013 Control: Control, Common, Control
+U+0014 Control: Control, Common, Control
+U+0015 Control: Control, Common, Control
+U+0016 Control: Control, Common, Control
+U+0017 Control: Control, Common, Control
+U+0018 Control: Control, Common, Control
+U+0019 Control: Control, Common, Control
+U+001A Control: Control, Common, Control
+U+001B Control: Control, Common, Control
+U+001C Control: Control, Common, Control
+U+001D Control: Control, Common, Control
+U+001E Control: Control, Common, Control
+U+001F Control: Control, Common, Control
 findprop 20 21 22 23 24 25 26 27 28 29 2a 2b 2c 2d 2e 2f 
-0020 Separator: Space separator, Common, Other
-0021 Punctuation: Other punctuation, Common, Other
-0022 Punctuation: Other punctuation, Common, Other
-0023 Punctuation: Other punctuation, Common, Other
-0024 Symbol: Currency symbol, Common, Other
-0025 Punctuation: Other punctuation, Common, Other
-0026 Punctuation: Other punctuation, Common, Other
-0027 Punctuation: Other punctuation, Common, Other
-0028 Punctuation: Open punctuation, Common, Other
-0029 Punctuation: Close punctuation, Common, Other
-002a Punctuation: Other punctuation, Common, Other
-002b Symbol: Mathematical symbol, Common, Other
-002c Punctuation: Other punctuation, Common, Other
-002d Punctuation: Dash punctuation, Common, Other
-002e Punctuation: Other punctuation, Common, Other
-002f Punctuation: Other punctuation, Common, Other
+U+0020 Separator: Space separator, Common, Other
+U+0021 Punctuation: Other punctuation, Common, Other
+U+0022 Punctuation: Other punctuation, Common, Other
+U+0023 Punctuation: Other punctuation, Common, Other
+U+0024 Symbol: Currency symbol, Common, Other
+U+0025 Punctuation: Other punctuation, Common, Other
+U+0026 Punctuation: Other punctuation, Common, Other
+U+0027 Punctuation: Other punctuation, Common, Other
+U+0028 Punctuation: Open punctuation, Common, Other
+U+0029 Punctuation: Close punctuation, Common, Other
+U+002A Punctuation: Other punctuation, Common, Other
+U+002B Symbol: Mathematical symbol, Common, Other
+U+002C Punctuation: Other punctuation, Common, Other
+U+002D Punctuation: Dash punctuation, Common, Other
+U+002E Punctuation: Other punctuation, Common, Other
+U+002F Punctuation: Other punctuation, Common, Other
 findprop 30 31 32 33 34 35 36 37 38 39 3a 3b 3c 3d 3e 3f 
-0030 Number: Decimal number, Common, Other
-0031 Number: Decimal number, Common, Other
-0032 Number: Decimal number, Common, Other
-0033 Number: Decimal number, Common, Other
-0034 Number: Decimal number, Common, Other
-0035 Number: Decimal number, Common, Other
-0036 Number: Decimal number, Common, Other
-0037 Number: Decimal number, Common, Other
-0038 Number: Decimal number, Common, Other
-0039 Number: Decimal number, Common, Other
-003a Punctuation: Other punctuation, Common, Other
-003b Punctuation: Other punctuation, Common, Other
-003c Symbol: Mathematical symbol, Common, Other
-003d Symbol: Mathematical symbol, Common, Other
-003e Symbol: Mathematical symbol, Common, Other
-003f Punctuation: Other punctuation, Common, Other
+U+0030 Number: Decimal number, Common, Other
+U+0031 Number: Decimal number, Common, Other
+U+0032 Number: Decimal number, Common, Other
+U+0033 Number: Decimal number, Common, Other
+U+0034 Number: Decimal number, Common, Other
+U+0035 Number: Decimal number, Common, Other
+U+0036 Number: Decimal number, Common, Other
+U+0037 Number: Decimal number, Common, Other
+U+0038 Number: Decimal number, Common, Other
+U+0039 Number: Decimal number, Common, Other
+U+003A Punctuation: Other punctuation, Common, Other
+U+003B Punctuation: Other punctuation, Common, Other
+U+003C Symbol: Mathematical symbol, Common, Other
+U+003D Symbol: Mathematical symbol, Common, Other
+U+003E Symbol: Mathematical symbol, Common, Other
+U+003F Punctuation: Other punctuation, Common, Other
 findprop 40 41 42 43 44 45 46 47 48 49 4a 4b 4c 4d 4e 4f 
-0040 Punctuation: Other punctuation, Common, Other
-0041 Letter: Upper case letter, Latin, Other, 0061
-0042 Letter: Upper case letter, Latin, Other, 0062
-0043 Letter: Upper case letter, Latin, Other, 0063
-0044 Letter: Upper case letter, Latin, Other, 0064
-0045 Letter: Upper case letter, Latin, Other, 0065
-0046 Letter: Upper case letter, Latin, Other, 0066
-0047 Letter: Upper case letter, Latin, Other, 0067
-0048 Letter: Upper case letter, Latin, Other, 0068
-0049 Letter: Upper case letter, Latin, Other, 0069
-004a Letter: Upper case letter, Latin, Other, 006a
-004b Letter: Upper case letter, Latin, Other, 006b, 212a
-004c Letter: Upper case letter, Latin, Other, 006c
-004d Letter: Upper case letter, Latin, Other, 006d
-004e Letter: Upper case letter, Latin, Other, 006e
-004f Letter: Upper case letter, Latin, Other, 006f
+U+0040 Punctuation: Other punctuation, Common, Other
+U+0041 Letter: Upper case letter, Latin, Other, U+0061
+U+0042 Letter: Upper case letter, Latin, Other, U+0062
+U+0043 Letter: Upper case letter, Latin, Other, U+0063
+U+0044 Letter: Upper case letter, Latin, Other, U+0064
+U+0045 Letter: Upper case letter, Latin, Other, U+0065
+U+0046 Letter: Upper case letter, Latin, Other, U+0066
+U+0047 Letter: Upper case letter, Latin, Other, U+0067
+U+0048 Letter: Upper case letter, Latin, Other, U+0068
+U+0049 Letter: Upper case letter, Latin, Other, U+0069
+U+004A Letter: Upper case letter, Latin, Other, U+006A
+U+004B Letter: Upper case letter, Latin, Other, U+006B, U+212A
+U+004C Letter: Upper case letter, Latin, Other, U+006C
+U+004D Letter: Upper case letter, Latin, Other, U+006D
+U+004E Letter: Upper case letter, Latin, Other, U+006E
+U+004F Letter: Upper case letter, Latin, Other, U+006F
 findprop 50 51 52 53 54 55 56 57 58 59 5a 5b 5c 5d 5e 5f 
-0050 Letter: Upper case letter, Latin, Other, 0070
-0051 Letter: Upper case letter, Latin, Other, 0071
-0052 Letter: Upper case letter, Latin, Other, 0072
-0053 Letter: Upper case letter, Latin, Other, 0073, 017f
-0054 Letter: Upper case letter, Latin, Other, 0074
-0055 Letter: Upper case letter, Latin, Other, 0075
-0056 Letter: Upper case letter, Latin, Other, 0076
-0057 Letter: Upper case letter, Latin, Other, 0077
-0058 Letter: Upper case letter, Latin, Other, 0078
-0059 Letter: Upper case letter, Latin, Other, 0079
-005a Letter: Upper case letter, Latin, Other, 007a
-005b Punctuation: Open punctuation, Common, Other
-005c Punctuation: Other punctuation, Common, Other
-005d Punctuation: Close punctuation, Common, Other
-005e Symbol: Modifier symbol, Common, Other
-005f Punctuation: Connector punctuation, Common, Other
+U+0050 Letter: Upper case letter, Latin, Other, U+0070
+U+0051 Letter: Upper case letter, Latin, Other, U+0071
+U+0052 Letter: Upper case letter, Latin, Other, U+0072
+U+0053 Letter: Upper case letter, Latin, Other, U+0073, U+017F
+U+0054 Letter: Upper case letter, Latin, Other, U+0074
+U+0055 Letter: Upper case letter, Latin, Other, U+0075
+U+0056 Letter: Upper case letter, Latin, Other, U+0076
+U+0057 Letter: Upper case letter, Latin, Other, U+0077
+U+0058 Letter: Upper case letter, Latin, Other, U+0078
+U+0059 Letter: Upper case letter, Latin, Other, U+0079
+U+005A Letter: Upper case letter, Latin, Other, U+007A
+U+005B Punctuation: Open punctuation, Common, Other
+U+005C Punctuation: Other punctuation, Common, Other
+U+005D Punctuation: Close punctuation, Common, Other
+U+005E Symbol: Modifier symbol, Common, Other
+U+005F Punctuation: Connector punctuation, Common, Other
 findprop 60 61 62 63 64 65 66 67 68 69 6a 6b 6c 6d 6e 6f 
-0060 Symbol: Modifier symbol, Common, Other
-0061 Letter: Lower case letter, Latin, Other, 0041
-0062 Letter: Lower case letter, Latin, Other, 0042
-0063 Letter: Lower case letter, Latin, Other, 0043
-0064 Letter: Lower case letter, Latin, Other, 0044
-0065 Letter: Lower case letter, Latin, Other, 0045
-0066 Letter: Lower case letter, Latin, Other, 0046
-0067 Letter: Lower case letter, Latin, Other, 0047
-0068 Letter: Lower case letter, Latin, Other, 0048
-0069 Letter: Lower case letter, Latin, Other, 0049
-006a Letter: Lower case letter, Latin, Other, 004a
-006b Letter: Lower case letter, Latin, Other, 004b, 212a
-006c Letter: Lower case letter, Latin, Other, 004c
-006d Letter: Lower case letter, Latin, Other, 004d
-006e Letter: Lower case letter, Latin, Other, 004e
-006f Letter: Lower case letter, Latin, Other, 004f
+U+0060 Symbol: Modifier symbol, Common, Other
+U+0061 Letter: Lower case letter, Latin, Other, U+0041
+U+0062 Letter: Lower case letter, Latin, Other, U+0042
+U+0063 Letter: Lower case letter, Latin, Other, U+0043
+U+0064 Letter: Lower case letter, Latin, Other, U+0044
+U+0065 Letter: Lower case letter, Latin, Other, U+0045
+U+0066 Letter: Lower case letter, Latin, Other, U+0046
+U+0067 Letter: Lower case letter, Latin, Other, U+0047
+U+0068 Letter: Lower case letter, Latin, Other, U+0048
+U+0069 Letter: Lower case letter, Latin, Other, U+0049
+U+006A Letter: Lower case letter, Latin, Other, U+004A
+U+006B Letter: Lower case letter, Latin, Other, U+004B, U+212A
+U+006C Letter: Lower case letter, Latin, Other, U+004C
+U+006D Letter: Lower case letter, Latin, Other, U+004D
+U+006E Letter: Lower case letter, Latin, Other, U+004E
+U+006F Letter: Lower case letter, Latin, Other, U+004F
 findprop 70 71 72 73 74 75 76 77 78 79 7a 7b 7c 7d 7e 7f 
-0070 Letter: Lower case letter, Latin, Other, 0050
-0071 Letter: Lower case letter, Latin, Other, 0051
-0072 Letter: Lower case letter, Latin, Other, 0052
-0073 Letter: Lower case letter, Latin, Other, 0053, 017f
-0074 Letter: Lower case letter, Latin, Other, 0054
-0075 Letter: Lower case letter, Latin, Other, 0055
-0076 Letter: Lower case letter, Latin, Other, 0056
-0077 Letter: Lower case letter, Latin, Other, 0057
-0078 Letter: Lower case letter, Latin, Other, 0058
-0079 Letter: Lower case letter, Latin, Other, 0059
-007a Letter: Lower case letter, Latin, Other, 005a
-007b Punctuation: Open punctuation, Common, Other
-007c Symbol: Mathematical symbol, Common, Other
-007d Punctuation: Close punctuation, Common, Other
-007e Symbol: Mathematical symbol, Common, Other
-007f Control: Control, Common, Control
+U+0070 Letter: Lower case letter, Latin, Other, U+0050
+U+0071 Letter: Lower case letter, Latin, Other, U+0051
+U+0072 Letter: Lower case letter, Latin, Other, U+0052
+U+0073 Letter: Lower case letter, Latin, Other, U+0053, U+017F
+U+0074 Letter: Lower case letter, Latin, Other, U+0054
+U+0075 Letter: Lower case letter, Latin, Other, U+0055
+U+0076 Letter: Lower case letter, Latin, Other, U+0056
+U+0077 Letter: Lower case letter, Latin, Other, U+0057
+U+0078 Letter: Lower case letter, Latin, Other, U+0058
+U+0079 Letter: Lower case letter, Latin, Other, U+0059
+U+007A Letter: Lower case letter, Latin, Other, U+005A
+U+007B Punctuation: Open punctuation, Common, Other
+U+007C Symbol: Mathematical symbol, Common, Other
+U+007D Punctuation: Close punctuation, Common, Other
+U+007E Symbol: Mathematical symbol, Common, Other
+U+007F Control: Control, Common, Control


findprop 80 81 82 83 84 85 86 87 88 89 8a 8b 8c 8d 8e 8f
-0080 Control: Control, Common, Control
-0081 Control: Control, Common, Control
-0082 Control: Control, Common, Control
-0083 Control: Control, Common, Control
-0084 Control: Control, Common, Control
-0085 Control: Control, Common, Control
-0086 Control: Control, Common, Control
-0087 Control: Control, Common, Control
-0088 Control: Control, Common, Control
-0089 Control: Control, Common, Control
-008a Control: Control, Common, Control
-008b Control: Control, Common, Control
-008c Control: Control, Common, Control
-008d Control: Control, Common, Control
-008e Control: Control, Common, Control
-008f Control: Control, Common, Control
+U+0080 Control: Control, Common, Control
+U+0081 Control: Control, Common, Control
+U+0082 Control: Control, Common, Control
+U+0083 Control: Control, Common, Control
+U+0084 Control: Control, Common, Control
+U+0085 Control: Control, Common, Control
+U+0086 Control: Control, Common, Control
+U+0087 Control: Control, Common, Control
+U+0088 Control: Control, Common, Control
+U+0089 Control: Control, Common, Control
+U+008A Control: Control, Common, Control
+U+008B Control: Control, Common, Control
+U+008C Control: Control, Common, Control
+U+008D Control: Control, Common, Control
+U+008E Control: Control, Common, Control
+U+008F Control: Control, Common, Control
findprop 90 91 92 93 94 95 96 97 98 99 9a 9b 9c 9d 9e 9f
-0090 Control: Control, Common, Control
-0091 Control: Control, Common, Control
-0092 Control: Control, Common, Control
-0093 Control: Control, Common, Control
-0094 Control: Control, Common, Control
-0095 Control: Control, Common, Control
-0096 Control: Control, Common, Control
-0097 Control: Control, Common, Control
-0098 Control: Control, Common, Control
-0099 Control: Control, Common, Control
-009a Control: Control, Common, Control
-009b Control: Control, Common, Control
-009c Control: Control, Common, Control
-009d Control: Control, Common, Control
-009e Control: Control, Common, Control
-009f Control: Control, Common, Control
+U+0090 Control: Control, Common, Control
+U+0091 Control: Control, Common, Control
+U+0092 Control: Control, Common, Control
+U+0093 Control: Control, Common, Control
+U+0094 Control: Control, Common, Control
+U+0095 Control: Control, Common, Control
+U+0096 Control: Control, Common, Control
+U+0097 Control: Control, Common, Control
+U+0098 Control: Control, Common, Control
+U+0099 Control: Control, Common, Control
+U+009A Control: Control, Common, Control
+U+009B Control: Control, Common, Control
+U+009C Control: Control, Common, Control
+U+009D Control: Control, Common, Control
+U+009E Control: Control, Common, Control
+U+009F Control: Control, Common, Control
findprop a0 a1 a2 a3 a4 a5 a6 a7 a8 a9 aa ab ac ad ae af
-00a0 Separator: Space separator, Common, Other
-00a1 Punctuation: Other punctuation, Common, Other
-00a2 Symbol: Currency symbol, Common, Other
-00a3 Symbol: Currency symbol, Common, Other
-00a4 Symbol: Currency symbol, Common, Other
-00a5 Symbol: Currency symbol, Common, Other
-00a6 Symbol: Other symbol, Common, Other
-00a7 Punctuation: Other punctuation, Common, Other
-00a8 Symbol: Modifier symbol, Common, Other
-00a9 Symbol: Other symbol, Common, Extended Pictographic
-00aa Letter: Other letter, Latin, Other
-00ab Punctuation: Initial punctuation, Common, Other
-00ac Symbol: Mathematical symbol, Common, Other
-00ad Control: Format, Common, Control
-00ae Symbol: Other symbol, Common, Extended Pictographic
-00af Symbol: Modifier symbol, Common, Other
+U+00A0 Separator: Space separator, Common, Other
+U+00A1 Punctuation: Other punctuation, Common, Other
+U+00A2 Symbol: Currency symbol, Common, Other
+U+00A3 Symbol: Currency symbol, Common, Other
+U+00A4 Symbol: Currency symbol, Common, Other
+U+00A5 Symbol: Currency symbol, Common, Other
+U+00A6 Symbol: Other symbol, Common, Other
+U+00A7 Punctuation: Other punctuation, Common, Other
+U+00A8 Symbol: Modifier symbol, Common, Other
+U+00A9 Symbol: Other symbol, Common, Extended Pictographic
+U+00AA Letter: Other letter, Latin, Other
+U+00AB Punctuation: Initial punctuation, Common, Other
+U+00AC Symbol: Mathematical symbol, Common, Other
+U+00AD Control: Format, Common, Control
+U+00AE Symbol: Other symbol, Common, Extended Pictographic
+U+00AF Symbol: Modifier symbol, Common, Other
findprop b0 b1 b2 b3 b4 b5 b6 b7 b8 b9 ba bb bc bd be bf
-00b0 Symbol: Other symbol, Common, Other
-00b1 Symbol: Mathematical symbol, Common, Other
-00b2 Number: Other number, Common, Other
-00b3 Number: Other number, Common, Other
-00b4 Symbol: Modifier symbol, Common, Other
-00b5 Letter: Lower case letter, Common, Other, 03bc, 039c
-00b6 Punctuation: Other punctuation, Common, Other
-00b7 Punctuation: Other punctuation, Common, Other
-00b8 Symbol: Modifier symbol, Common, Other
-00b9 Number: Other number, Common, Other
-00ba Letter: Other letter, Latin, Other
-00bb Punctuation: Final punctuation, Common, Other
-00bc Number: Other number, Common, Other
-00bd Number: Other number, Common, Other
-00be Number: Other number, Common, Other
-00bf Punctuation: Other punctuation, Common, Other
+U+00B0 Symbol: Other symbol, Common, Other
+U+00B1 Symbol: Mathematical symbol, Common, Other
+U+00B2 Number: Other number, Common, Other
+U+00B3 Number: Other number, Common, Other
+U+00B4 Symbol: Modifier symbol, Common, Other
+U+00B5 Letter: Lower case letter, Common, Other, U+03BC, U+039C
+U+00B6 Punctuation: Other punctuation, Common, Other
+U+00B7 Punctuation: Other punctuation, Common, Other
+U+00B8 Symbol: Modifier symbol, Common, Other
+U+00B9 Number: Other number, Common, Other
+U+00BA Letter: Other letter, Latin, Other
+U+00BB Punctuation: Final punctuation, Common, Other
+U+00BC Number: Other number, Common, Other
+U+00BD Number: Other number, Common, Other
+U+00BE Number: Other number, Common, Other
+U+00BF Punctuation: Other punctuation, Common, Other
findprop c0 c1 c2 c3 c4 c5 c6 c7 c8 c9 ca cb cc cd ce cf
-00c0 Letter: Upper case letter, Latin, Other, 00e0
-00c1 Letter: Upper case letter, Latin, Other, 00e1
-00c2 Letter: Upper case letter, Latin, Other, 00e2
-00c3 Letter: Upper case letter, Latin, Other, 00e3
-00c4 Letter: Upper case letter, Latin, Other, 00e4
-00c5 Letter: Upper case letter, Latin, Other, 00e5, 212b
-00c6 Letter: Upper case letter, Latin, Other, 00e6
-00c7 Letter: Upper case letter, Latin, Other, 00e7
-00c8 Letter: Upper case letter, Latin, Other, 00e8
-00c9 Letter: Upper case letter, Latin, Other, 00e9
-00ca Letter: Upper case letter, Latin, Other, 00ea
-00cb Letter: Upper case letter, Latin, Other, 00eb
-00cc Letter: Upper case letter, Latin, Other, 00ec
-00cd Letter: Upper case letter, Latin, Other, 00ed
-00ce Letter: Upper case letter, Latin, Other, 00ee
-00cf Letter: Upper case letter, Latin, Other, 00ef
+U+00C0 Letter: Upper case letter, Latin, Other, U+00E0
+U+00C1 Letter: Upper case letter, Latin, Other, U+00E1
+U+00C2 Letter: Upper case letter, Latin, Other, U+00E2
+U+00C3 Letter: Upper case letter, Latin, Other, U+00E3
+U+00C4 Letter: Upper case letter, Latin, Other, U+00E4
+U+00C5 Letter: Upper case letter, Latin, Other, U+00E5, U+212B
+U+00C6 Letter: Upper case letter, Latin, Other, U+00E6
+U+00C7 Letter: Upper case letter, Latin, Other, U+00E7
+U+00C8 Letter: Upper case letter, Latin, Other, U+00E8
+U+00C9 Letter: Upper case letter, Latin, Other, U+00E9
+U+00CA Letter: Upper case letter, Latin, Other, U+00EA
+U+00CB Letter: Upper case letter, Latin, Other, U+00EB
+U+00CC Letter: Upper case letter, Latin, Other, U+00EC
+U+00CD Letter: Upper case letter, Latin, Other, U+00ED
+U+00CE Letter: Upper case letter, Latin, Other, U+00EE
+U+00CF Letter: Upper case letter, Latin, Other, U+00EF
findprop d0 d1 d2 d3 d4 d5 d6 d7 d8 d9 da db dc dd de df
-00d0 Letter: Upper case letter, Latin, Other, 00f0
-00d1 Letter: Upper case letter, Latin, Other, 00f1
-00d2 Letter: Upper case letter, Latin, Other, 00f2
-00d3 Letter: Upper case letter, Latin, Other, 00f3
-00d4 Letter: Upper case letter, Latin, Other, 00f4
-00d5 Letter: Upper case letter, Latin, Other, 00f5
-00d6 Letter: Upper case letter, Latin, Other, 00f6
-00d7 Symbol: Mathematical symbol, Common, Other
-00d8 Letter: Upper case letter, Latin, Other, 00f8
-00d9 Letter: Upper case letter, Latin, Other, 00f9
-00da Letter: Upper case letter, Latin, Other, 00fa
-00db Letter: Upper case letter, Latin, Other, 00fb
-00dc Letter: Upper case letter, Latin, Other, 00fc
-00dd Letter: Upper case letter, Latin, Other, 00fd
-00de Letter: Upper case letter, Latin, Other, 00fe
-00df Letter: Lower case letter, Latin, Other, 1e9e
+U+00D0 Letter: Upper case letter, Latin, Other, U+00F0
+U+00D1 Letter: Upper case letter, Latin, Other, U+00F1
+U+00D2 Letter: Upper case letter, Latin, Other, U+00F2
+U+00D3 Letter: Upper case letter, Latin, Other, U+00F3
+U+00D4 Letter: Upper case letter, Latin, Other, U+00F4
+U+00D5 Letter: Upper case letter, Latin, Other, U+00F5
+U+00D6 Letter: Upper case letter, Latin, Other, U+00F6
+U+00D7 Symbol: Mathematical symbol, Common, Other
+U+00D8 Letter: Upper case letter, Latin, Other, U+00F8
+U+00D9 Letter: Upper case letter, Latin, Other, U+00F9
+U+00DA Letter: Upper case letter, Latin, Other, U+00FA
+U+00DB Letter: Upper case letter, Latin, Other, U+00FB
+U+00DC Letter: Upper case letter, Latin, Other, U+00FC
+U+00DD Letter: Upper case letter, Latin, Other, U+00FD
+U+00DE Letter: Upper case letter, Latin, Other, U+00FE
+U+00DF Letter: Lower case letter, Latin, Other, U+1E9E
findprop e0 e1 e2 e3 e4 e5 e6 e7 e8 e9 ea eb ec ed ee ef
-00e0 Letter: Lower case letter, Latin, Other, 00c0
-00e1 Letter: Lower case letter, Latin, Other, 00c1
-00e2 Letter: Lower case letter, Latin, Other, 00c2
-00e3 Letter: Lower case letter, Latin, Other, 00c3
-00e4 Letter: Lower case letter, Latin, Other, 00c4
-00e5 Letter: Lower case letter, Latin, Other, 00c5, 212b
-00e6 Letter: Lower case letter, Latin, Other, 00c6
-00e7 Letter: Lower case letter, Latin, Other, 00c7
-00e8 Letter: Lower case letter, Latin, Other, 00c8
-00e9 Letter: Lower case letter, Latin, Other, 00c9
-00ea Letter: Lower case letter, Latin, Other, 00ca
-00eb Letter: Lower case letter, Latin, Other, 00cb
-00ec Letter: Lower case letter, Latin, Other, 00cc
-00ed Letter: Lower case letter, Latin, Other, 00cd
-00ee Letter: Lower case letter, Latin, Other, 00ce
-00ef Letter: Lower case letter, Latin, Other, 00cf
+U+00E0 Letter: Lower case letter, Latin, Other, U+00C0
+U+00E1 Letter: Lower case letter, Latin, Other, U+00C1
+U+00E2 Letter: Lower case letter, Latin, Other, U+00C2
+U+00E3 Letter: Lower case letter, Latin, Other, U+00C3
+U+00E4 Letter: Lower case letter, Latin, Other, U+00C4
+U+00E5 Letter: Lower case letter, Latin, Other, U+00C5, U+212B
+U+00E6 Letter: Lower case letter, Latin, Other, U+00C6
+U+00E7 Letter: Lower case letter, Latin, Other, U+00C7
+U+00E8 Letter: Lower case letter, Latin, Other, U+00C8
+U+00E9 Letter: Lower case letter, Latin, Other, U+00C9
+U+00EA Letter: Lower case letter, Latin, Other, U+00CA
+U+00EB Letter: Lower case letter, Latin, Other, U+00CB
+U+00EC Letter: Lower case letter, Latin, Other, U+00CC
+U+00ED Letter: Lower case letter, Latin, Other, U+00CD
+U+00EE Letter: Lower case letter, Latin, Other, U+00CE
+U+00EF Letter: Lower case letter, Latin, Other, U+00CF
findprop f0 f1 f2 f3 f4 f5 f6 f7 f8 f9 fa fb fc fd fe ff
-00f0 Letter: Lower case letter, Latin, Other, 00d0
-00f1 Letter: Lower case letter, Latin, Other, 00d1
-00f2 Letter: Lower case letter, Latin, Other, 00d2
-00f3 Letter: Lower case letter, Latin, Other, 00d3
-00f4 Letter: Lower case letter, Latin, Other, 00d4
-00f5 Letter: Lower case letter, Latin, Other, 00d5
-00f6 Letter: Lower case letter, Latin, Other, 00d6
-00f7 Symbol: Mathematical symbol, Common, Other
-00f8 Letter: Lower case letter, Latin, Other, 00d8
-00f9 Letter: Lower case letter, Latin, Other, 00d9
-00fa Letter: Lower case letter, Latin, Other, 00da
-00fb Letter: Lower case letter, Latin, Other, 00db
-00fc Letter: Lower case letter, Latin, Other, 00dc
-00fd Letter: Lower case letter, Latin, Other, 00dd
-00fe Letter: Lower case letter, Latin, Other, 00de
-00ff Letter: Lower case letter, Latin, Other, 0178
+U+00F0 Letter: Lower case letter, Latin, Other, U+00D0
+U+00F1 Letter: Lower case letter, Latin, Other, U+00D1
+U+00F2 Letter: Lower case letter, Latin, Other, U+00D2
+U+00F3 Letter: Lower case letter, Latin, Other, U+00D3
+U+00F4 Letter: Lower case letter, Latin, Other, U+00D4
+U+00F5 Letter: Lower case letter, Latin, Other, U+00D5
+U+00F6 Letter: Lower case letter, Latin, Other, U+00D6
+U+00F7 Symbol: Mathematical symbol, Common, Other
+U+00F8 Letter: Lower case letter, Latin, Other, U+00D8
+U+00F9 Letter: Lower case letter, Latin, Other, U+00D9
+U+00FA Letter: Lower case letter, Latin, Other, U+00DA
+U+00FB Letter: Lower case letter, Latin, Other, U+00DB
+U+00FC Letter: Lower case letter, Latin, Other, U+00DC
+U+00FD Letter: Lower case letter, Latin, Other, U+00DD
+U+00FE Letter: Lower case letter, Latin, Other, U+00DE
+U+00FF Letter: Lower case letter, Latin, Other, U+0178

findprop 0100 0101 0102 0103 0104 0105 0106
-0100 Letter: Upper case letter, Latin, Other, 0101
-0101 Letter: Lower case letter, Latin, Other, 0100
-0102 Letter: Upper case letter, Latin, Other, 0103
-0103 Letter: Lower case letter, Latin, Other, 0102
-0104 Letter: Upper case letter, Latin, Other, 0105
-0105 Letter: Lower case letter, Latin, Other, 0104
-0106 Letter: Upper case letter, Latin, Other, 0107
+U+0100 Letter: Upper case letter, Latin, Other, U+0101
+U+0101 Letter: Lower case letter, Latin, Other, U+0100
+U+0102 Letter: Upper case letter, Latin, Other, U+0103
+U+0103 Letter: Lower case letter, Latin, Other, U+0102
+U+0104 Letter: Upper case letter, Latin, Other, U+0105
+U+0105 Letter: Lower case letter, Latin, Other, U+0104
+U+0106 Letter: Upper case letter, Latin, Other, U+0107

findprop ffe0 ffe1 ffe2 ffe3 ffe4 ffe5 ffe6 ffe7
-ffe0 Symbol: Currency symbol, Common, Other
-ffe1 Symbol: Currency symbol, Common, Other
-ffe2 Symbol: Mathematical symbol, Common, Other
-ffe3 Symbol: Modifier symbol, Common, Other
-ffe4 Symbol: Other symbol, Common, Other
-ffe5 Symbol: Currency symbol, Common, Other
-ffe6 Symbol: Currency symbol, Common, Other
-ffe7 Control: Unassigned, Unknown, Other
+U+FFE0 Symbol: Currency symbol, Common, Other
+U+FFE1 Symbol: Currency symbol, Common, Other
+U+FFE2 Symbol: Mathematical symbol, Common, Other
+U+FFE3 Symbol: Modifier symbol, Common, Other
+U+FFE4 Symbol: Other symbol, Common, Other
+U+FFE5 Symbol: Currency symbol, Common, Other
+U+FFE6 Symbol: Currency symbol, Common, Other
+U+FFE7 Control: Unassigned, Unknown, Other
findprop ffe8 ffe9 ffea ffeb ffec ffed ffee ffef
-ffe8 Symbol: Other symbol, Common, Other
-ffe9 Symbol: Mathematical symbol, Common, Other
-ffea Symbol: Mathematical symbol, Common, Other
-ffeb Symbol: Mathematical symbol, Common, Other
-ffec Symbol: Mathematical symbol, Common, Other
-ffed Symbol: Other symbol, Common, Other
-ffee Symbol: Other symbol, Common, Other
-ffef Control: Unassigned, Unknown, Other
+U+FFE8 Symbol: Other symbol, Common, Other
+U+FFE9 Symbol: Mathematical symbol, Common, Other
+U+FFEA Symbol: Mathematical symbol, Common, Other
+U+FFEB Symbol: Mathematical symbol, Common, Other
+U+FFEC Symbol: Mathematical symbol, Common, Other
+U+FFED Symbol: Other symbol, Common, Other
+U+FFEE Symbol: Other symbol, Common, Other
+U+FFEF Control: Unassigned, Unknown, Other
findprop fff8 fff9 fffa fffb fffc fffd fffe ffff
-fff8 Control: Unassigned, Unknown, Control
-fff9 Control: Format, Common, Control
-fffa Control: Format, Common, Control
-fffb Control: Format, Common, Control
-fffc Symbol: Other symbol, Common, Other
-fffd Symbol: Other symbol, Common, Other
-fffe Control: Unassigned, Unknown, Other
-ffff Control: Unassigned, Unknown, Other
+U+FFF8 Control: Unassigned, Unknown, Control
+U+FFF9 Control: Format, Common, Control
+U+FFFA Control: Format, Common, Control
+U+FFFB Control: Format, Common, Control
+U+FFFC Symbol: Other symbol, Common, Other
+U+FFFD Symbol: Other symbol, Common, Other
+U+FFFE Control: Unassigned, Unknown, Other
+U+FFFF Control: Unassigned, Unknown, Other
findprop 10000 10001 e01ef f0000 100000
-10000 Letter: Other letter, Linear_B, Other
-10001 Letter: Other letter, Linear_B, Other
-e01ef Mark: Non-spacing mark, Inherited, Extend
-f0000 Control: Private use, Unknown, Other
-100000 Control: Private use, Unknown, Other
+U+10000 Letter: Other letter, Linear_B, Other
+U+10001 Letter: Other letter, Linear_B, Other
+U+E01EF Mark: Non-spacing mark, Inherited, Extend
+U+F0000 Control: Private use, Unknown, Other
+U+100000 Control: Private use, Unknown, Other

findprop 1b00 12000 7c0 a840 10900
-1b00 Mark: Non-spacing mark, Balinese, Extend
-12000 Letter: Other letter, Cuneiform, Other
-07c0 Number: Decimal number, Nko, Other
-a840 Letter: Other letter, Phags_Pa, Other
-10900 Letter: Other letter, Phoenician, Other
+U+1B00 Mark: Non-spacing mark, Balinese, Extend
+U+12000 Letter: Other letter, Cuneiform, Other
+U+07C0 Number: Decimal number, Nko, Other
+U+A840 Letter: Other letter, Phags_Pa, Other
+U+10900 Letter: Other letter, Phoenician, Other
findprop 1d79 a77d
-1d79 Letter: Lower case letter, Latin, Other, a77d
-a77d Letter: Upper case letter, Latin, Other, 1d79
+U+1D79 Letter: Lower case letter, Latin, Other, U+A77D
+U+A77D Letter: Upper case letter, Latin, Other, U+1D79

findprop 0800 083e a4d0 a4f7 aa80 aadf
-0800 Letter: Other letter, Samaritan, Other
-083e Punctuation: Other punctuation, Samaritan, Other
-a4d0 Letter: Other letter, Lisu, Other
-a4f7 Letter: Other letter, Lisu, Other
-aa80 Letter: Other letter, Tai_Viet, Other
-aadf Punctuation: Other punctuation, Tai_Viet, Other
+U+0800 Letter: Other letter, Samaritan, Other
+U+083E Punctuation: Other punctuation, Samaritan, Other
+U+A4D0 Letter: Other letter, Lisu, Other
+U+A4F7 Letter: Other letter, Lisu, Other
+U+AA80 Letter: Other letter, Tai_Viet, Other
+U+AADF Punctuation: Other punctuation, Tai_Viet, Other
findprop 10b00 10b35 13000 1342e 10840 10855
-10b00 Letter: Other letter, Avestan, Other
-10b35 Letter: Other letter, Avestan, Other
-13000 Letter: Other letter, Egyptian_Hieroglyphs, Other
-1342e Letter: Other letter, Egyptian_Hieroglyphs, Other
-10840 Letter: Other letter, Imperial_Aramaic, Other
-10855 Letter: Other letter, Imperial_Aramaic, Other
+U+10B00 Letter: Other letter, Avestan, Other
+U+10B35 Letter: Other letter, Avestan, Other
+U+13000 Letter: Other letter, Egyptian_Hieroglyphs, Other
+U+1342E Letter: Other letter, Egyptian_Hieroglyphs, Other
+U+10840 Letter: Other letter, Imperial_Aramaic, Other
+U+10855 Letter: Other letter, Imperial_Aramaic, Other

findprop 11100 1113c 11680 116c0
-11100 Mark: Non-spacing mark, Chakma, Extend
-1113c Number: Decimal number, Chakma, Other
-11680 Letter: Other letter, Takri, Other
-116c0 Number: Decimal number, Takri, Other
+U+11100 Mark: Non-spacing mark, Chakma, Extend
+U+1113C Number: Decimal number, Chakma, Other
+U+11680 Letter: Other letter, Takri, Other
+U+116C0 Number: Decimal number, Takri, Other

findprop 0d 0a 0e 0711 1b04 1111 1169 11fe ae4c ad89
-000d Control: Control, Common, CR
-000a Control: Control, Common, LF
-000e Control: Control, Common, Control
-0711 Mark: Non-spacing mark, Syriac, Extend
-1b04 Mark: Spacing mark, Balinese, SpacingMark
-1111 Letter: Other letter, Hangul, Hangul syllable type L
-1169 Letter: Other letter, Hangul, Hangul syllable type V
-11fe Letter: Other letter, Hangul, Hangul syllable type T
-ae4c Letter: Other letter, Hangul, Hangul syllable type LV
-ad89 Letter: Other letter, Hangul, Hangul syllable type LVT
+U+000D Control: Control, Common, CR
+U+000A Control: Control, Common, LF
+U+000E Control: Control, Common, Control
+U+0711 Mark: Non-spacing mark, Syriac, Extend
+U+1B04 Mark: Spacing mark, Balinese, SpacingMark
+U+1111 Letter: Other letter, Hangul, Hangul syllable type L
+U+1169 Letter: Other letter, Hangul, Hangul syllable type V
+U+11FE Letter: Other letter, Hangul, Hangul syllable type T
+U+AE4C Letter: Other letter, Hangul, Hangul syllable type LV
+U+AD89 Letter: Other letter, Hangul, Hangul syllable type LVT

findprop 118a0 11ac7 16ad0
-118a0 Letter: Upper case letter, Warang_Citi, Other, 118c0
-11ac7 Letter: Other letter, Pau_Cin_Hau, Other
-16ad0 Letter: Other letter, Bassa_Vah, Other
+U+118A0 Letter: Upper case letter, Warang_Citi, Other, U+118C0
+U+11AC7 Letter: Other letter, Pau_Cin_Hau, Other
+U+16AD0 Letter: Other letter, Bassa_Vah, Other

findprop 11700 14400 108e0 11280 1d800
-11700 Letter: Other letter, Ahom, Other
-14400 Letter: Other letter, Anatolian_Hieroglyphs, Other
-108e0 Letter: Other letter, Hatran, Other
-11280 Letter: Other letter, Multani, Other
-1d800 Symbol: Other symbol, SignWriting, Other
+U+11700 Letter: Other letter, Ahom, Other
+U+14400 Letter: Other letter, Anatolian_Hieroglyphs, Other
+U+108E0 Letter: Other letter, Hatran, Other
+U+11280 Letter: Other letter, Multani, Other
+U+1D800 Symbol: Other symbol, SignWriting, Other

findprop 11800 1e903 11da9 10d27 11ee0 16e48 10f27 10f30
-11800 Letter: Other letter, Dogra, Other
-1e903 Letter: Upper case letter, Adlam, Other, 1e925
-11da9 Number: Decimal number, Gunjala_Gondi, Other
-10d27 Mark: Non-spacing mark, Hanifi_Rohingya, Extend
-11ee0 Letter: Other letter, Makasar, Other
-16e48 Letter: Upper case letter, Medefaidrin, Other, 16e68
-10f27 Letter: Other letter, Old_Sogdian, Other
-10f30 Letter: Other letter, Sogdian, Other
+U+11800 Letter: Other letter, Dogra, Other
+U+1E903 Letter: Upper case letter, Adlam, Other, U+1E925
+U+11DA9 Number: Decimal number, Gunjala_Gondi, Other
+U+10D27 Mark: Non-spacing mark, Hanifi_Rohingya, Extend
+U+11EE0 Letter: Other letter, Makasar, Other
+U+16E48 Letter: Upper case letter, Medefaidrin, Other, U+16E68
+U+10F27 Letter: Other letter, Old_Sogdian, Other
+U+10F30 Letter: Other letter, Sogdian, Other

findprop a836 a833 1cf4 20f0 1cd0
-a836 Symbol: Other symbol, Common, Other, [Devanagari, Dogra, Gujarati, Gurmukhi, Khojki, Kaithi, Mahajani, Modi, Khudawadi, Takri, Tirhuta]
-a833 Number: Other number, Common, Other, [Devanagari, Dogra, Gujarati, Gurmukhi, Khojki, Kannada, Kaithi, Mahajani, Modi, Nandinagari, Khudawadi, Takri, Tirhuta]
-1cf4 Mark: Non-spacing mark, Inherited, Extend, [Devanagari, Grantha, Kannada]
-20f0 Mark: Non-spacing mark, Inherited, Extend, [Devanagari, Grantha, Latin]
-1cd0 Mark: Non-spacing mark, Inherited, Extend, [Bengali, Devanagari, Grantha, Kannada]
+U+A836 Symbol: Other symbol, Common, Other, [Devanagari, Dogra, Gujarati, Gurmukhi, Khojki, Kaithi, Mahajani, Modi, Khudawadi, Takri, Tirhuta]
+U+A833 Number: Other number, Common, Other, [Devanagari, Dogra, Gujarati, Gurmukhi, Khojki, Kannada, Kaithi, Mahajani, Modi, Nandinagari, Khudawadi, Takri, Tirhuta]
+U+1CF4 Mark: Non-spacing mark, Inherited, Extend, [Devanagari, Grantha, Kannada]
+U+20F0 Mark: Non-spacing mark, Inherited, Extend, [Devanagari, Grantha, Latin]
+U+1CD0 Mark: Non-spacing mark, Inherited, Extend, [Bengali, Devanagari, Grantha, Kannada]

findprop 32ff
-32ff Symbol: Other symbol, Common, Other, [Han]
+U+32FF Symbol: Other symbol, Common, Other, [Han]

findprop 1f16d
-1f16d Symbol: Other symbol, Common, Extended Pictographic
+U+1F16D Symbol: Other symbol, Common, Extended Pictographic

-findprop 10e93 10eaa
-10e93 Letter: Other letter, Yezidi, Other
-10eaa Control: Unassigned, Unknown, Other
+findprop U+10e93 U+10eaa
+U+10E93 Letter: Other letter, Yezidi, Other
+U+10EAA Control: Unassigned, Unknown, Other

Added: code/trunk/maint/ucptestdata/testoutput2
===================================================================
--- code/trunk/maint/ucptestdata/testoutput2                            (rev 0)
+++ code/trunk/maint/ucptestdata/testoutput2    2020-04-01 17:00:33 UTC (rev 1241)
@@ -0,0 +1,188 @@
+find script Han
+U+2E80..U+2E99 Symbol: Other symbol, Han, Other
+U+2E9B..U+2EF3 Symbol: Other symbol, Han, Other
+U+2F00..U+2FD5 Symbol: Other symbol, Han, Other
+        U+3005 Letter: Modifier letter, Han, Other
+        U+3007 Number: Letter number, Han, Other
+U+3021..U+3029 Number: Letter number, Han, Other
+U+3038..U+303A Number: Letter number, Han, Other
+        U+303B Letter: Modifier letter, Han, Other
+U+3400..U+4DBF Letter: Other letter, Han, Other
+U+4E00..U+9FFC Letter: Other letter, Han, Other
+U+F900..U+FA6D Letter: Other letter, Han, Other
+U+FA70..U+FAD9 Letter: Other letter, Han, Other
+U+16FF0..U+16FF1 Mark: Spacing mark, Han, SpacingMark
+U+20000..U+2A6DD Letter: Other letter, Han, Other
+U+2A700..U+2B734 Letter: Other letter, Han, Other
+U+2B740..U+2B81D Letter: Other letter, Han, Other
+U+2B820..U+2CEA1 Letter: Other letter, Han, Other
+U+2CEB0..U+2EBE0 Letter: Other letter, Han, Other
+U+2F800..U+2FA1D Letter: Other letter, Han, Other
+U+30000..U+3134A Letter: Other letter, Han, Other
+find type Pe script Common scriptx Hangul
+U+3009 Punctuation: Close punctuation, Common, Other, [Bopomofo, Hangul, Han, Hiragana, Katakana, Yi]
+U+300B Punctuation: Close punctuation, Common, Other, [Bopomofo, Hangul, Han, Hiragana, Katakana, Yi]
+U+300D Punctuation: Close punctuation, Common, Other, [Bopomofo, Hangul, Han, Hiragana, Katakana, Yi]
+U+300F Punctuation: Close punctuation, Common, Other, [Bopomofo, Hangul, Han, Hiragana, Katakana, Yi]
+U+3011 Punctuation: Close punctuation, Common, Other, [Bopomofo, Hangul, Han, Hiragana, Katakana, Yi]
+U+3015 Punctuation: Close punctuation, Common, Other, [Bopomofo, Hangul, Han, Hiragana, Katakana, Yi]
+U+3017 Punctuation: Close punctuation, Common, Other, [Bopomofo, Hangul, Han, Hiragana, Katakana, Yi]
+U+3019 Punctuation: Close punctuation, Common, Other, [Bopomofo, Hangul, Han, Hiragana, Katakana, Yi]
+U+301B Punctuation: Close punctuation, Common, Other, [Bopomofo, Hangul, Han, Hiragana, Katakana, Yi]
+U+301E..U+301F Punctuation: Close punctuation, Common, Other, [Bopomofo, Hangul, Han, Hiragana, Katakana]
+        U+FF63 Punctuation: Close punctuation, Common, Other, [Bopomofo, Hangul, Han, Hiragana, Katakana, Yi]
+find type Sk
+U+005E Symbol: Modifier symbol, Common, Other
+U+0060 Symbol: Modifier symbol, Common, Other
+U+00A8 Symbol: Modifier symbol, Common, Other
+U+00AF Symbol: Modifier symbol, Common, Other
+U+00B4 Symbol: Modifier symbol, Common, Other
+U+00B8 Symbol: Modifier symbol, Common, Other
+U+02C2..U+02C5 Symbol: Modifier symbol, Common, Other
+U+02D2..U+02DF Symbol: Modifier symbol, Common, Other
+U+02E5..U+02E9 Symbol: Modifier symbol, Common, Other
+U+02EA..U+02EB Symbol: Modifier symbol, Bopomofo, Other
+        U+02ED Symbol: Modifier symbol, Common, Other
+U+02EF..U+02FF Symbol: Modifier symbol, Common, Other
+        U+0375 Symbol: Modifier symbol, Greek, Other
+        U+0384 Symbol: Modifier symbol, Greek, Other
+        U+0385 Symbol: Modifier symbol, Common, Other
+        U+1FBD Symbol: Modifier symbol, Greek, Other
+U+1FBF..U+1FC1 Symbol: Modifier symbol, Greek, Other
+U+1FCD..U+1FCF Symbol: Modifier symbol, Greek, Other
+U+1FDD..U+1FDF Symbol: Modifier symbol, Greek, Other
+U+1FED..U+1FEF Symbol: Modifier symbol, Greek, Other
+U+1FFD..U+1FFE Symbol: Modifier symbol, Greek, Other
+U+309B..U+309C Symbol: Modifier symbol, Common, Other, [Hiragana, Katakana]
+U+A700..U+A707 Symbol: Modifier symbol, Common, Other, [Han, Latin]
+U+A708..U+A716 Symbol: Modifier symbol, Common, Other
+U+A720..U+A721 Symbol: Modifier symbol, Common, Other
+U+A789..U+A78A Symbol: Modifier symbol, Common, Other
+        U+AB5B Symbol: Modifier symbol, Common, Other
+U+AB6A..U+AB6B Symbol: Modifier symbol, Common, Other
+U+FBB2..U+FBC1 Symbol: Modifier symbol, Arabic, Other
+        U+FF3E Symbol: Modifier symbol, Common, Other
+        U+FF40 Symbol: Modifier symbol, Common, Other
+        U+FFE3 Symbol: Modifier symbol, Common, Other
+U+1F3FB..U+1F3FF Symbol: Modifier symbol, Common, Extend
+find type Pd
+U+002D Punctuation: Dash punctuation, Common, Other
+U+058A Punctuation: Dash punctuation, Armenian, Other
+U+05BE Punctuation: Dash punctuation, Hebrew, Other
+U+1400 Punctuation: Dash punctuation, Canadian_Aboriginal, Other
+U+1806 Punctuation: Dash punctuation, Mongolian, Other
+U+2010..U+2015 Punctuation: Dash punctuation, Common, Other
+        U+2E17 Punctuation: Dash punctuation, Common, Other
+        U+2E1A Punctuation: Dash punctuation, Common, Other
+U+2E3A..U+2E3B Punctuation: Dash punctuation, Common, Other
+        U+2E40 Punctuation: Dash punctuation, Common, Other
+        U+301C Punctuation: Dash punctuation, Common, Other, [Bopomofo, Hangul, Han, Hiragana, Katakana]
+        U+3030 Punctuation: Dash punctuation, Common, Extended Pictographic, [Bopomofo, Hangul, Han, Hiragana, Katakana]
+        U+30A0 Punctuation: Dash punctuation, Common, Other, [Hiragana, Katakana]
+U+FE31..U+FE32 Punctuation: Dash punctuation, Common, Other
+        U+FE58 Punctuation: Dash punctuation, Common, Other
+        U+FE63 Punctuation: Dash punctuation, Common, Other
+        U+FF0D Punctuation: Dash punctuation, Common, Other
+        U+10EAD Punctuation: Dash punctuation, Yezidi, Other
+find gbreak LVT
+U+AC01..U+AC1B Letter: Other letter, Hangul, Hangul syllable type LVT
+U+AC1D..U+AC37 Letter: Other letter, Hangul, Hangul syllable type LVT
+U+AC39..U+AC53 Letter: Other letter, Hangul, Hangul syllable type LVT
+U+AC55..U+AC6F Letter: Other letter, Hangul, Hangul syllable type LVT
+U+AC71..U+AC8B Letter: Other letter, Hangul, Hangul syllable type LVT
+U+AC8D..U+ACA7 Letter: Other letter, Hangul, Hangul syllable type LVT
+U+ACA9..U+ACC3 Letter: Other letter, Hangul, Hangul syllable type LVT
+U+ACC5..U+ACDF Letter: Other letter, Hangul, Hangul syllable type LVT
+U+ACE1..U+ACFB Letter: Other letter, Hangul, Hangul syllable type LVT
+U+ACFD..U+AD17 Letter: Other letter, Hangul, Hangul syllable type LVT
+U+AD19..U+AD33 Letter: Other letter, Hangul, Hangul syllable type LVT
+U+AD35..U+AD4F Letter: Other letter, Hangul, Hangul syllable type LVT
+U+AD51..U+AD6B Letter: Other letter, Hangul, Hangul syllable type LVT
+U+AD6D..U+AD87 Letter: Other letter, Hangul, Hangul syllable type LVT
+U+AD89..U+ADA3 Letter: Other letter, Hangul, Hangul syllable type LVT
+U+ADA5..U+ADBF Letter: Other letter, Hangul, Hangul syllable type LVT
+U+ADC1..U+ADDB Letter: Other letter, Hangul, Hangul syllable type LVT
+U+ADDD..U+ADF7 Letter: Other letter, Hangul, Hangul syllable type LVT
+U+ADF9..U+AE13 Letter: Other letter, Hangul, Hangul syllable type LVT
+U+AE15..U+AE2F Letter: Other letter, Hangul, Hangul syllable type LVT
+U+AE31..U+AE4B Letter: Other letter, Hangul, Hangul syllable type LVT
+U+AE4D..U+AE67 Letter: Other letter, Hangul, Hangul syllable type LVT
+U+AE69..U+AE83 Letter: Other letter, Hangul, Hangul syllable type LVT
+U+AE85..U+AE9F Letter: Other letter, Hangul, Hangul syllable type LVT
+U+AEA1..U+AEBB Letter: Other letter, Hangul, Hangul syllable type LVT
+U+AEBD..U+AED7 Letter: Other letter, Hangul, Hangul syllable type LVT
+U+AED9..U+AEF3 Letter: Other letter, Hangul, Hangul syllable type LVT
+U+AEF5..U+AF0F Letter: Other letter, Hangul, Hangul syllable type LVT
+U+AF11..U+AF2B Letter: Other letter, Hangul, Hangul syllable type LVT
+U+AF2D..U+AF47 Letter: Other letter, Hangul, Hangul syllable type LVT
+U+AF49..U+AF63 Letter: Other letter, Hangul, Hangul syllable type LVT
+U+AF65..U+AF7F Letter: Other letter, Hangul, Hangul syllable type LVT
+U+AF81..U+AF9B Letter: Other letter, Hangul, Hangul syllable type LVT
+U+AF9D..U+AFB7 Letter: Other letter, Hangul, Hangul syllable type LVT
+U+AFB9..U+AFD3 Letter: Other letter, Hangul, Hangul syllable type LVT
+U+AFD5..U+AFEF Letter: Other letter, Hangul, Hangul syllable type LVT
+U+AFF1..U+B00B Letter: Other letter, Hangul, Hangul syllable type LVT
+U+B00D..U+B027 Letter: Other letter, Hangul, Hangul syllable type LVT
+U+B029..U+B043 Letter: Other letter, Hangul, Hangul syllable type LVT
+U+B045..U+B05F Letter: Other letter, Hangul, Hangul syllable type LVT
+U+B061..U+B07B Letter: Other letter, Hangul, Hangul syllable type LVT
+U+B07D..U+B097 Letter: Other letter, Hangul, Hangul syllable type LVT
+U+B099..U+B0B3 Letter: Other letter, Hangul, Hangul syllable type LVT
+U+B0B5..U+B0CF Letter: Other letter, Hangul, Hangul syllable type LVT
+U+B0D1..U+B0EB Letter: Other letter, Hangul, Hangul syllable type LVT
+U+B0ED..U+B107 Letter: Other letter, Hangul, Hangul syllable type LVT
+U+B109..U+B123 Letter: Other letter, Hangul, Hangul syllable type LVT
+U+B125..U+B13F Letter: Other letter, Hangul, Hangul syllable type LVT
+U+B141..U+B15B Letter: Other letter, Hangul, Hangul syllable type LVT
+U+B15D..U+B177 Letter: Other letter, Hangul, Hangul syllable type LVT
+U+B179..U+B193 Letter: Other letter, Hangul, Hangul syllable type LVT
+U+B195..U+B1AF Letter: Other letter, Hangul, Hangul syllable type LVT
+U+B1B1..U+B1CB Letter: Other letter, Hangul, Hangul syllable type LVT
+U+B1CD..U+B1E7 Letter: Other letter, Hangul, Hangul syllable type LVT
+U+B1E9..U+B203 Letter: Other letter, Hangul, Hangul syllable type LVT
+U+B205..U+B21F Letter: Other letter, Hangul, Hangul syllable type LVT
+U+B221..U+B23B Letter: Other letter, Hangul, Hangul syllable type LVT
+U+B23D..U+B257 Letter: Other letter, Hangul, Hangul syllable type LVT
+U+B259..U+B273 Letter: Other letter, Hangul, Hangul syllable type LVT
+U+B275..U+B28F Letter: Other letter, Hangul, Hangul syllable type LVT
+U+B291..U+B2AB Letter: Other letter, Hangul, Hangul syllable type LVT
+U+B2AD..U+B2C7 Letter: Other letter, Hangul, Hangul syllable type LVT
+U+B2C9..U+B2E3 Letter: Other letter, Hangul, Hangul syllable type LVT
+U+B2E5..U+B2FF Letter: Other letter, Hangul, Hangul syllable type LVT
+U+B301..U+B31B Letter: Other letter, Hangul, Hangul syllable type LVT
+U+B31D..U+B337 Letter: Other letter, Hangul, Hangul syllable type LVT
+U+B339..U+B353 Letter: Other letter, Hangul, Hangul syllable type LVT
+U+B355..U+B36F Letter: Other letter, Hangul, Hangul syllable type LVT
+U+B371..U+B38B Letter: Other letter, Hangul, Hangul syllable type LVT
+U+B38D..U+B3A7 Letter: Other letter, Hangul, Hangul syllable type LVT
+U+B3A9..U+B3C3 Letter: Other letter, Hangul, Hangul syllable type LVT
+U+B3C5..U+B3DF Letter: Other letter, Hangul, Hangul syllable type LVT
+U+B3E1..U+B3FB Letter: Other letter, Hangul, Hangul syllable type LVT
+U+B3FD..U+B417 Letter: Other letter, Hangul, Hangul syllable type LVT
+U+B419..U+B433 Letter: Other letter, Hangul, Hangul syllable type LVT
+U+B435..U+B44F Letter: Other letter, Hangul, Hangul syllable type LVT
+U+B451..U+B46B Letter: Other letter, Hangul, Hangul syllable type LVT
+U+B46D..U+B487 Letter: Other letter, Hangul, Hangul syllable type LVT
+U+B489..U+B4A3 Letter: Other letter, Hangul, Hangul syllable type LVT
+U+B4A5..U+B4BF Letter: Other letter, Hangul, Hangul syllable type LVT
+U+B4C1..U+B4DB Letter: Other letter, Hangul, Hangul syllable type LVT
+U+B4DD..U+B4F7 Letter: Other letter, Hangul, Hangul syllable type LVT
+U+B4F9..U+B513 Letter: Other letter, Hangul, Hangul syllable type LVT
+U+B515..U+B52F Letter: Other letter, Hangul, Hangul syllable type LVT
+U+B531..U+B54B Letter: Other letter, Hangul, Hangul syllable type LVT
+U+B54D..U+B567 Letter: Other letter, Hangul, Hangul syllable type LVT
+U+B569..U+B583 Letter: Other letter, Hangul, Hangul syllable type LVT
+U+B585..U+B59F Letter: Other letter, Hangul, Hangul syllable type LVT
+U+B5A1..U+B5BB Letter: Other letter, Hangul, Hangul syllable type LVT
+U+B5BD..U+B5D7 Letter: Other letter, Hangul, Hangul syllable type LVT
+U+B5D9..U+B5F3 Letter: Other letter, Hangul, Hangul syllable type LVT
+U+B5F5..U+B60F Letter: Other letter, Hangul, Hangul syllable type LVT
+U+B611..U+B62B Letter: Other letter, Hangul, Hangul syllable type LVT
+U+B62D..U+B647 Letter: Other letter, Hangul, Hangul syllable type LVT
+U+B649..U+B663 Letter: Other letter, Hangul, Hangul syllable type LVT
+U+B665..U+B67F Letter: Other letter, Hangul, Hangul syllable type LVT
+U+B681..U+B69B Letter: Other letter, Hangul, Hangul syllable type LVT
+U+B69D..U+B6B7 Letter: Other letter, Hangul, Hangul syllable type LVT
+U+B6B9..U+B6D3 Letter: Other letter, Hangul, Hangul syllable type LVT
+U+B6D5..U+B6EF Letter: Other letter, Hangul, Hangul syllable type LVT
+...


Modified: code/trunk/maint/utf8.c
===================================================================
--- code/trunk/maint/utf8.c    2020-03-25 17:18:33 UTC (rev 1240)
+++ code/trunk/maint/utf8.c    2020-04-01 17:00:33 UTC (rev 1241)
@@ -1,30 +1,47 @@
-/* A test program for converting characters to UTF-8 and vice versa. Note that
-this program conforms to the original definition of UTF-8, which allows
-codepoints up to 7fffffff. The more recent definition limits the validity of
-UTF-8 codepoints to a maximum of 10ffffff.
+/****************************************************
+* PCRE maintainers' helper program: UTF-8 converter *
+****************************************************/


-The arguments are either single codepoint values, written as 0xhhhh, for
-conversion to UTF-8, or sequences of hex values, written without 0x and
-optionally including spaces (but such arguments must be quoted), for conversion
+/* This is a test program for converting character code points to UTF-8 and
+vice versa. Note that this program conforms to the original definition of
+UTF-8, which allows codepoints up to 7fffffff. The more recent definition
+limits the validity of Unicode UTF-8 codepoints to a maximum of 10ffffff, and
+forbids the "surrogate" code points. This program now gives warnings for these
+invalid code points.
+
+The arguments are either single code point values written as U+hh.. or 0xhh..
+for conversion to UTF-8, or sequences of hex values, written without 0x and
+optionally including spaces (but such arguments must be quoted), for conversion
from UTF-8 to codepoints. For example:

./utf8 0x1234
-0x00001234 => e1 88 b4
+U+00001234 => e1 88 b4

./utf8 "e1 88 b4"
-0x00001234 <= e1 88 b4
+U+00001234 <= e1 88 b4

-In the second case, a number of characters can be present in one argument:
+In the second case, a number of UTF-8 characters can be present in one
+argument. In other words, each such argument is interpreted (after ignoring
+spaces) as a string of UTF-8 bytes representing a string of characters:

./utf8 "65 e188b4 77"
-0x00000065 <= 65
-0x00001234 <= e1 88 b4
-0x00000077 <= 77
+0x00000065 <= 65
+0x00001234 <= e1 88 b4
+0x00000077 <= 77

-If the option -s is given, the sequence of UTF-bytes is written out between
+If the option -s is given, the sequence of UTF-bytes is written out between
angle brackets at the end of the line. On a UTF-8 terminal, this will show the
-appropriate graphic for the codepoint. */
+appropriate graphic for the code point.

+Errors provoke error messages, but the program carries on with the next
+argument. The return code is always zero.
+
+Philip Hazel
+Original creation data: unknown
+Code extended and tidied to avoid compiler warnings: 26 March 2020
+*/
+
+
#include <stdio.h>
#include <stdlib.h>
#include <ctype.h>
@@ -41,20 +58,14 @@
*/


-static const int utf8_table1[] = {
- 0x0000007f, 0x000007ff, 0x0000ffff, 0x001fffff, 0x03ffffff, 0x7fffffff};
+static const unsigned int utf8_table1[] = {
+ 0x0000007f, 0x000007ff, 0x0000ffff, 0x001fffff, 0x03ffffff, 0x7fffffff};

static const int utf8_table2[] = {
- 0, 0xc0, 0xe0, 0xf0, 0xf8, 0xfc};
-
+ 0, 0xc0, 0xe0, 0xf0, 0xf8, 0xfc};
+
static const int utf8_table3[] = {
- 0xff, 0x1f, 0x0f, 0x07, 0x03, 0x01};
-
-static const unsigned char utf8_table4[] = {
- 1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,
- 1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,
- 2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,
- 3,3,3,3,3,3,3,3,4,4,4,4,5,5,6,6 };
+ 0xff, 0x1f, 0x0f, 0x07, 0x03, 0x01};


 /*************************************************
@@ -61,27 +72,24 @@
 *       Convert character value to UTF-8         *
 *************************************************/


-/* This function takes an integer value in the range 0 - 0x7fffffff
-and encodes it as a UTF-8 character in 1 to 6 bytes.
+/* This function takes an unsigned long integer value in the range 0 -
+0x7fffffff and encodes it as a UTF-8 character in 1 to 6 bytes.

-Arguments:   
-  cvalue     the character value 
+Arguments:
+  cvalue     the character value
   buffer     pointer to buffer for result - at least 6 bytes long
-  
-Returns:     number of characters placed in the buffer
-             -1 if input character is negative  
-             0 if input character is positive but too big (only when
-             int is longer than 32 bits) 
+
+Returns:     number of bytes placed in the buffer
+             0 if input code point is too big
 */


-int
-ord2utf8(int cvalue, unsigned char *buffer)
+static size_t
+ord2utf8(unsigned long int cvalue, unsigned char *buffer)
 {
-register int i, j;
+size_t i, j;
 for (i = 0; i < sizeof(utf8_table1)/sizeof(int); i++)
   if (cvalue <= utf8_table1[i]) break;
 if (i >= sizeof(utf8_table1)/sizeof(int)) return 0;
-if (cvalue < 0) return -1;
 buffer += i;
 for (j = i; j > 0; j--)
  {
@@ -98,33 +106,60 @@
 *            Convert UTF-8 string to value       *
 *************************************************/


-/* This function takes one or more bytes that represents a UTF-8 character,
-and returns the value of the character.
+/* This function takes one or more bytes that represent a UTF-8 character from
+the start of a string of bytes. It returns the value of the character, or the
+offset of a malformation. For an overlong encoding that works but is not the
+correct (shortest) one, the error offset is just after the last byte.

-Argument:  
+Argument:
   buffer   a pointer to the byte vector
-  vptr     a pointer to an int to receive the value 
+  buffend  a pointer to the end of the buffer
+  vptr     a pointer to a variable to receive the value
+  lenptr   a pointer to a variable to receive the offset when error detected


-Returns:   >  0 => the number of bytes consumed
-           -6 to 0 => malformed UTF-8 character at offset = (-return)
+Returns:   > 0 => the number of bytes consumed
+             0 => invalid UTF-8: first byte missing 0x40 bit
+            -1 => invalid UTF-8: first byte has too many high-order 1-bits
+            -2 => incomplete sequence at end of string
+            -3 => incomplete sequence within string
+            -4 => overlong code sequence
 */


-int
-utf82ord(unsigned char *buffer, int *vptr)
+static int
+utf82ord(unsigned char *buffer, unsigned char *buffend,
+ long unsigned int *vptr, int *lenptr)
{
-int c = *buffer++;
-int d = c;
+unsigned int c = *buffer++;
+unsigned int d = c;
int i, j, s;

-for (i = -1; i < 6; i++)               /* i is number of additional bytes */
+/* Check for an ASCII character, or find the number of additional bytes in a
+multibyte character. */
+
+for (i = -1; i < 6; i++)
   {
   if ((d & 0x80) == 0) break;
   d <<= 1;
   }


-if (i == -1) { *vptr = c; return 1; }  /* ascii character */
-if (i == 0 || i == 6) return 0;        /* invalid UTF-8 */
+switch (i)
+  {
+  case -1:      /* ASCII character; first byte does not have 0x80 bit */
+  *vptr = c;
+  return 1;


+  case 0:       /* First byte has 0x80 but is missing 0x40 bit */
+  *lenptr = 0;
+  return 0;
+
+  case 6:
+  *lenptr = 0;  /* Too many high bits */
+  return -1;
+
+  default:
+  break;
+  }
+
 /* i now has a value in the range 1-5 */


s = 6*i;
@@ -132,32 +167,46 @@

 for (j = 0; j < i; j++)
   {
+  if (buffer >= buffend)
+    {
+    *lenptr = j + 1;
+    return -2;
+    }
   c = *buffer++;
-  if ((c & 0xc0) != 0x80) return -(j+1);
+  if ((c & 0xc0) != 0x80)
+    {
+    *lenptr = j + 1;
+    return -3;
+    }
   s -= 6;
   d |= (c & 0x3f) << s;
   }


-/* Check that encoding was the correct unique one */
+/* Valid UTF-8 syntax */

-for (j = 0; j < sizeof(utf8_table1)/sizeof(int); j++)
+*vptr = d;
+
+/* Check that encoding was the correct one, not overlong */
+
+for (j = 0; j < (int)(sizeof(utf8_table1)/sizeof(int)); j++)
if (d <= utf8_table1[j]) break;
-if (j != i) return -(i+1);
+if (j != i)
+ {
+ *lenptr = i + 1;
+ return -4;
+ }

/* Valid value */

-*vptr = d;
-return i+1;
+return i + 1;
}



-
 /*************************************************
 *                 Main Program                   *
 *************************************************/


-
int
main(int argc, char **argv)
{
@@ -169,85 +218,129 @@
{
show = 1;
i = 2;
- }
+ }

 for (; i < argc; i++)
   {
   char *x = argv[i];
-  if (strncmp(x, "0x", 2) == 0)
+  char *endptr;
+  if (strncmp(x, "0x", 2) == 0 || strncmp(x, "U+", 2) == 0)
     {
-    int j; 
-    int d = strtol(x+2, NULL, 16);
-    int rc = ord2utf8(d, buffer);
-    printf("0x%08x => ", d); 
-    if (rc <= 0) printf("*** Error %d ***", rc); else 
+    size_t rc, j;
+    unsigned long int d = strtoul(x+2, &endptr, 16);
+    if (*endptr != 0)
       {
+      printf("** Invalid hex number %s\n", x);
+      continue;   /* With next argument */
+      }
+    rc = ord2utf8(d, buffer);
+    printf("U+%08lx => ", d);
+    if (rc == 0)
+      printf("** Code point greater than 0x7fffffff cannot be encoded");
+    else
+      {
       for (j = 0; j < rc; j++) printf("%02x ", buffer[j]);
       if (show)
         {
         printf(">");
         for (j = 0; j < rc; j++) printf("%c", buffer[j]);
-        printf("<"); 
-        }  
-      } 
-    printf("\n");   
+        printf("< ");
+        }
+      if (d >= 0xd800 && d <= 0xdfff)
+        printf("** Invalid Unicode (surrogate)");
+      else if (d > 0x10ffff)
+        printf("** Invalid Unicode (greater than U+10ffff)");
+      }
+    printf("\n");
     }
   else
     {
-    int d, rc; 
-    int j = 0;
-    int y = 0; 
+    unsigned char *bptr;
+    unsigned char *buffend;
+    int len = 0;
+    int y = 0;
     int z = 0;
-    unsigned char *bptr;
-       
-    for (;;) 
-      { 
-      while (*x == ' ') x++; 
+
+    for (;;)
+      {
+      while (*x == ' ') x++;
       if (*x == 0 && !z) break;
-      if (!isxdigit(*x)) 
+      if (!isxdigit(*x))
         {
-        printf("Malformed hex string: %s\n", argv[i]);
-        j = -1;
-        break;    
-        } 
+        printf("** Malformed hex string: %s\n", argv[i]);
+        len = -1;
+        break;
+        }
       y = y * 16 + tolower(*x) - ((isdigit(*x))? '0' : 'W');
-      x++; 
+      x++;
       if (z)
-        { 
-        buffer[j++] = y;
+        {
+        buffer[len++] = y;
         y = 0;
         }
-      z ^= 1;     
-      } 
-    buffer[j] = 0;
+      z ^= 1;
+      }
+
+    if (len < 0) continue;  /* With next argument after malformation */
+
     bptr = buffer;
+    buffend = buffer + len;


-    while (*bptr != 0)
-      { 
-      rc = utf82ord(bptr, &d);
-      if (rc > 0) 
+    while (bptr < buffend)
+      {
+      unsigned long int d;
+      int j;
+      int offset;
+      int rc = utf82ord(bptr, buffend, &d, &offset);
+
+      if (rc > 0)
         {
-        printf("0x%08x <= ", d);
+        printf("U+%08lx <= ", d);
         for (j = 0; j < rc; j++) printf("%02x ", bptr[j]);
         if (show)
           {
           printf(">");
           for (j = 0; j < rc; j++) printf("%c", bptr[j]);
-          printf("<"); 
-          }  
+          printf("<");
+          }
         printf("\n");
-        bptr += rc; 
-        } 
-      else 
+        bptr += rc;
+        }
+      else if (rc == -4)
         {
-        printf("Malformed UTF-8 at offset %d <= ", -rc);
-        while (*bptr != 0) printf("%02x ", *bptr++);
-        printf("\n"); 
-        break;  
-        } 
-      }   
-    }       
-  } 
+        printf("U+%08lx <= ", d);
+        for (j = 0; j < offset; j++) printf("%02x ", bptr[j]);
+        printf("** Overlong UTF-8 sequence\n");
+        bptr += offset;
+        }
+      else
+        {
+        switch (rc)
+          {
+          case 0:  printf("** First byte missing 0x40 bit");
+          break;
+
+          case -1: printf("** First byte has too many high-order bits");
+          break;
+
+          case -2: printf("** Incomplete UTF-8 sequence at end of string");
+          break;
+
+          case -3: printf("** Incomplete UTF-8 sequence");
+          break;
+
+          default: printf("** Unexpected return %d from utf82ord()", rc);
+          break;
+          }
+        printf(" at offset %d in string ", offset);
+        while (bptr < buffend) printf("%02x ", *bptr++);
+        printf("\n");
+        break;
+        }
+      }
+    }
+  }
+
 return 0;
 }