Revision: 861
http://vcs.pcre.org/viewvc?view=rev&revision=861
Author: ph10
Date: 2012-01-10 14:54:28 +0000 (Tue, 10 Jan 2012)
Log Message:
-----------
Documentation.
Modified Paths:
--------------
code/trunk/doc/pcre.3
code/trunk/doc/pcreprecompile.3
code/trunk/doc/pcresample.3
code/trunk/doc/pcrestack.3
code/trunk/doc/pcresyntax.3
Modified: code/trunk/doc/pcre.3
===================================================================
--- code/trunk/doc/pcre.3 2012-01-09 20:12:58 UTC (rev 860)
+++ code/trunk/doc/pcre.3 2012-01-10 14:54:28 UTC (rev 861)
@@ -26,7 +26,9 @@
.\" HREF
\fBpcre16\fP
.\"
-page.
+page. References to functions or structures of the form \fIpcre[16]_xxx\fP
+should be read as meaning "\fIpcre_xxx\fP when using the 8-bit library and
+\fIpcre16_xxx\fP when using the 16-bit library".
.P
The current implementation of PCRE corresponds approximately with Perl 5.12,
including support for UTF-8/16 encoded strings and Unicode general category
@@ -151,6 +153,6 @@
.rs
.sp
.nf
-Last updated: 01 January 2012
+Last updated: 10 January 2012
Copyright (c) 1997-2012 University of Cambridge.
.fi
Modified: code/trunk/doc/pcreprecompile.3
===================================================================
--- code/trunk/doc/pcreprecompile.3 2012-01-09 20:12:58 UTC (rev 860)
+++ code/trunk/doc/pcreprecompile.3 2012-01-10 14:54:28 UTC (rev 861)
@@ -13,27 +13,30 @@
.\"
documentation), this is relatively straightforward. If you are using private
tables, it is a little bit more complicated. However, if you are using the
-just-in-time optimization feature of \fBpcre_study()\fP, it is not possible to
-save and reload the JIT data.
+just-in-time optimization feature, it is not possible to save and reload the
+JIT data.
.P
If you save compiled patterns to a file, you can copy them to a different host
-and run them there. This works even if the new host has the opposite endianness
-to the one on which the patterns were compiled. There may be a small
-performance penalty, but it should be insignificant. However, compiling regular
-expressions with one version of PCRE for use with a different version is not
-guaranteed to work and may cause crashes, and saving and restoring a compiled
-pattern loses any JIT optimization data.
+and run them there. If the two hosts have different endianness (byte order),
+you should run the \fBpcre[16]_pattern_to_host_byte_order()\fP function on the
+new host before trying to match the pattern. The matching functions return
+PCRE_ERROR_BADENDIANNESS if they detect a pattern with the wrong endianness.
+.P
+Compiling regular expressions with one version of PCRE for use with a different
+version is not guaranteed to work and may cause crashes, and saving and
+restoring a compiled pattern loses any JIT optimization data.
.
.
.SH "SAVING A COMPILED PATTERN"
.rs
.sp
-The value returned by \fBpcre_compile()\fP points to a single block of memory
-that holds the compiled pattern and associated data. You can find the length of
-this block in bytes by calling \fBpcre_fullinfo()\fP with an argument of
-PCRE_INFO_SIZE. You can then save the data in any appropriate manner. Here is
-sample code that compiles a pattern and writes it to a file. It assumes that
-the variable \fIfd\fP refers to a file that is open for output:
+The value returned by \fBpcre[16]_compile()\fP points to a single block of
+memory that holds the compiled pattern and associated data. You can find the
+length of this block in bytes by calling \fBpcre[16]_fullinfo()\fP with an
+argument of PCRE_INFO_SIZE. You can then save the data in any appropriate
+manner. Here is sample code for the 8-bit library that compiles a pattern and
+writes it to a file. It assumes that the variable \fIfd\fP refers to a file
+that is open for output:
.sp
int erroroffset, rc, size;
char *error;
@@ -65,8 +68,8 @@
data in a similar way to the compiled pattern itself. However, if the
PCRE_STUDY_JIT_COMPILE was used, the just-in-time data that is created cannot
be saved because it is too dependent on the current environment. When studying
-generates additional information, \fBpcre_study()\fP returns a pointer to a
-\fBpcre_extra\fP data block. Its format is defined in the
+generates additional information, \fBpcre[16]_study()\fP returns a pointer to a
+\fBpcre[16]_extra\fP data block. Its format is defined in the
.\" HTML <a href="pcreapi.html#extradata">
.\" </a>
section on matching a pattern
@@ -76,26 +79,27 @@
\fBpcreapi\fP
.\"
documentation. The \fIstudy_data\fP field points to the binary study data, and
-this is what you must save (not the \fBpcre_extra\fP block itself). The length
-of the study data can be obtained by calling \fBpcre_fullinfo()\fP with an
-argument of PCRE_INFO_STUDYSIZE. Remember to check that \fBpcre_study()\fP did
-return a non-NULL value before trying to save the study data.
+this is what you must save (not the \fBpcre[16]_extra\fP block itself). The
+length of the study data can be obtained by calling \fBpcre[16]_fullinfo()\fP
+with an argument of PCRE_INFO_STUDYSIZE. Remember to check that
+\fBpcre[16]_study()\fP did return a non-NULL value before trying to save the
+study data.
.
.
.SH "RE-USING A PRECOMPILED PATTERN"
.rs
.sp
Re-using a precompiled pattern is straightforward. Having reloaded it into main
-memory, you pass its pointer to \fBpcre_exec()\fP or \fBpcre_dfa_exec()\fP in
-the usual way. This should work even on another host, and even if that host has
-the opposite endianness to the one where the pattern was compiled.
+memory, called \fBpcre[16]_pattern_to_host_byte_order()\fP if necessary,
+you pass its pointer to \fBpcre[16]_exec()\fP or \fBpcre[16]_dfa_exec()\fP in
+the usual way.
.P
However, if you passed a pointer to custom character tables when the pattern
-was compiled (the \fItableptr\fP argument of \fBpcre_compile()\fP), you must
-now pass a similar pointer to \fBpcre_exec()\fP or \fBpcre_dfa_exec()\fP,
-because the value saved with the compiled pattern will obviously be nonsense. A
-field in a \fBpcre_extra()\fP block is used to pass this data, as described in
-the
+was compiled (the \fItableptr\fP argument of \fBpcre[16]_compile()\fP), you
+must now pass a similar pointer to \fBpcre[16]_exec()\fP or
+\fBpcre[16]_dfa_exec()\fP, because the value saved with the compiled pattern
+will obviously be nonsense. A field in a \fBpcre[16]_extra()\fP block is used
+to pass this data, as described in the
.\" HTML <a href="pcreapi.html#extradata">
.\" </a>
section on matching a pattern
@@ -107,17 +111,17 @@
documentation.
.P
If you did not provide custom character tables when the pattern was compiled,
-the pointer in the compiled pattern is NULL, which causes \fBpcre_exec()\fP to
-use PCRE's internal tables. Thus, you do not need to take any special action at
-run time in this case.
+the pointer in the compiled pattern is NULL, which causes the matching
+functions to use PCRE's internal tables. Thus, you do not need to take any
+special action at run time in this case.
.P
If you saved study data with the compiled pattern, you need to create your own
-\fBpcre_extra\fP data block and set the \fIstudy_data\fP field to point to the
+\fBpcre[16]_extra\fP data block and set the \fIstudy_data\fP field to point to the
reloaded study data. You must also set the PCRE_EXTRA_STUDY_DATA bit in the
\fIflags\fP field to indicate that study data is present. Then pass the
-\fBpcre_extra\fP block to \fBpcre_exec()\fP or \fBpcre_dfa_exec()\fP in the
-usual way. If the pattern was studied for just-in-time optimization, that data
-cannot be saved, and so is lost by a save/restore cycle.
+\fBpcre[16]_extra\fP block to the matching function in the usual way. If the
+pattern was studied for just-in-time optimization, that data cannot be saved,
+and so is lost by a save/restore cycle.
.
.
.SH "COMPATIBILITY WITH DIFFERENT PCRE RELEASES"
@@ -142,6 +146,6 @@
.rs
.sp
.nf
-Last updated: 26 August 2011
-Copyright (c) 1997-2011 University of Cambridge.
+Last updated: 10 January 2012
+Copyright (c) 1997-2012 University of Cambridge.
.fi
Modified: code/trunk/doc/pcresample.3
===================================================================
--- code/trunk/doc/pcresample.3 2012-01-09 20:12:58 UTC (rev 860)
+++ code/trunk/doc/pcresample.3 2012-01-10 14:54:28 UTC (rev 861)
@@ -13,11 +13,12 @@
documentation. If you do not have a copy of the PCRE distribution, you can save
this listing to re-create \fIpcredemo.c\fP.
.P
-The program compiles the regular expression that is its first argument, and
-matches it against the subject string in its second argument. No PCRE options
-are set, and default character tables are used. If matching succeeds, the
-program outputs the portion of the subject that matched, together with the
-contents of any captured substrings.
+The demonstration program, which uses the original PCRE 8-bit library, compiles
+the regular expression that is its first argument, and matches it against the
+subject string in its second argument. No PCRE options are set, and default
+character tables are used. If matching succeeds, the program outputs the
+portion of the subject that matched, together with the contents of any captured
+substrings.
.P
If the -g option is given on the command line, the program then goes on to
check for further matches of the same regular expression in the same subject
@@ -55,8 +56,8 @@
.\" HREF
\fBpcretest\fP,
.\"
-which supports many more facilities for testing regular expressions and the
-PCRE library. The
+which supports many more facilities for testing regular expressions and both
+PCRE libraries. The
.\" HREF
\fBpcredemo\fP
.\"
@@ -93,6 +94,6 @@
.rs
.sp
.nf
-Last updated: 17 November 2010
-Copyright (c) 1997-2010 University of Cambridge.
+Last updated: 10 January 2012
+Copyright (c) 1997-2012 University of Cambridge.
.fi
Modified: code/trunk/doc/pcrestack.3
===================================================================
--- code/trunk/doc/pcrestack.3 2012-01-09 20:12:58 UTC (rev 860)
+++ code/trunk/doc/pcrestack.3 2012-01-10 14:54:28 UTC (rev 861)
@@ -4,11 +4,11 @@
.SH "PCRE DISCUSSION OF STACK USAGE"
.rs
.sp
-When you call \fBpcre_exec()\fP, it makes use of an internal function called
-\fBmatch()\fP. This calls itself recursively at branch points in the pattern,
-in order to remember the state of the match so that it can back up and try a
-different alternative if the first one fails. As matching proceeds deeper and
-deeper into the tree of possibilities, the recursion depth increases. The
+When you call \fBpcre[16]_exec()\fP, it makes use of an internal function
+called \fBmatch()\fP. This calls itself recursively at branch points in the
+pattern, in order to remember the state of the match so that it can back up and
+try a different alternative if the first one fails. As matching proceeds deeper
+and deeper into the tree of possibilities, the recursion depth increases. The
\fBmatch()\fP function is also called in other circumstances, for example,
whenever a parenthesized sub-pattern is entered, and in certain cases of
repetition.
@@ -19,10 +19,10 @@
the recursive call would immediately be passed back as the result of the
current call (a "tail recursion"), the function is just restarted instead.
.P
-The above comments apply when \fBpcre_exec()\fP is run in its normal
+The above comments apply when \fBpcre[16]_exec()\fP is run in its normal
interpretive manner. If the pattern was studied with the
PCRE_STUDY_JIT_COMPILE option, and just-in-time compiling was successful, and
-the options passed to \fBpcre_exec()\fP were not incompatible, the matching
+the options passed to \fBpcre[16]_exec()\fP were not incompatible, the matching
process uses the JIT-compiled code instead of the \fBmatch()\fP function. In
this case, the memory requirements are handled entirely differently. See the
.\" HREF
@@ -30,21 +30,21 @@
.\"
documentation for details.
.P
-The \fBpcre_dfa_exec()\fP function operates in an entirely different way, and
-uses recursion only when there is a regular expression recursion or subroutine
-call in the pattern. This includes the processing of assertion and "once-only"
-subpatterns, which are handled like subroutine calls. Normally, these are never
-very deep, and the limit on the complexity of \fBpcre_dfa_exec()\fP is
-controlled by the amount of workspace it is given. However, it is possible to
-write patterns with runaway infinite recursions; such patterns will cause
-\fBpcre_dfa_exec()\fP to run out of stack. At present, there is no protection
-against this.
+The \fBpcre[16]_dfa_exec()\fP function operates in an entirely different way,
+and uses recursion only when there is a regular expression recursion or
+subroutine call in the pattern. This includes the processing of assertion and
+"once-only" subpatterns, which are handled like subroutine calls. Normally,
+these are never very deep, and the limit on the complexity of
+\fBpcre[16]_dfa_exec()\fP is controlled by the amount of workspace it is given.
+However, it is possible to write patterns with runaway infinite recursions;
+such patterns will cause \fBpcre[16]_dfa_exec()\fP to run out of stack. At
+present, there is no protection against this.
.P
-The comments that follow do NOT apply to \fBpcre_dfa_exec()\fP; they are
-relevant only for \fBpcre_exec()\fP without the JIT optimization.
+The comments that follow do NOT apply to \fBpcre[16]_dfa_exec()\fP; they are
+relevant only for \fBpcre[16]_exec()\fP without the JIT optimization.
.
.
-.SS "Reducing \fBpcre_exec()\fP's stack usage"
+.SS "Reducing \fBpcre[16]_exec()\fP's stack usage"
.rs
.sp
Each time that \fBmatch()\fP is actually called recursively, it uses memory
@@ -79,41 +79,42 @@
than one character whenever possible.
.
.
-.SS "Compiling PCRE to use heap instead of stack for \fBpcre_exec()\fP"
+.SS "Compiling PCRE to use heap instead of stack for \fBpcre[16]_exec()\fP"
.rs
.sp
In environments where stack memory is constrained, you might want to compile
PCRE to use heap memory instead of stack for remembering back-up points when
-\fBpcre_exec()\fP is running. This makes it run a lot more slowly, however.
+\fBpcre[16]_exec()\fP is running. This makes it run a lot more slowly, however.
Details of how to do this are given in the
.\" HREF
\fBpcrebuild\fP
.\"
documentation. When built in this way, instead of using the stack, PCRE obtains
and frees memory by calling the functions that are pointed to by the
-\fBpcre_stack_malloc\fP and \fBpcre_stack_free\fP variables. By default, these
-point to \fBmalloc()\fP and \fBfree()\fP, but you can replace the pointers to
-cause PCRE to use your own functions. Since the block sizes are always the
-same, and are always freed in reverse order, it may be possible to implement
-customized memory handlers that are more efficient than the standard functions.
+\fBpcre[16]_stack_malloc\fP and \fBpcre[16]_stack_free\fP variables. By
+default, these point to \fBmalloc()\fP and \fBfree()\fP, but you can replace
+the pointers to cause PCRE to use your own functions. Since the block sizes are
+always the same, and are always freed in reverse order, it may be possible to
+implement customized memory handlers that are more efficient than the standard
+functions.
.
.
-.SS "Limiting \fBpcre_exec()\fP's stack usage"
+.SS "Limiting \fBpcre[16]_exec()\fP's stack usage"
.rs
.sp
You can set limits on the number of times that \fBmatch()\fP is called, both in
-total and recursively. If a limit is exceeded, \fBpcre_exec()\fP returns an
+total and recursively. If a limit is exceeded, \fBpcre[16]_exec()\fP returns an
error code. Setting suitable limits should prevent it from running out of
stack. The default values of the limits are very large, and unlikely ever to
operate. They can be changed when PCRE is built, and they can also be set when
-\fBpcre_exec()\fP is called. For details of these interfaces, see the
+\fBpcre[16]_exec()\fP is called. For details of these interfaces, see the
.\" HREF
\fBpcrebuild\fP
.\"
documentation and the
.\" HTML <a href="pcreapi.html#extradata">
.\" </a>
-section on extra data for \fBpcre_exec()\fP
+section on extra data for \fBpcre[16]_exec()\fP
.\"
in the
.\" HREF
@@ -130,7 +131,7 @@
option (\fB-S\fP) that can be used to increase the size of its stack. As long
as the stack is large enough, another option (\fB-M\fP) can be used to find the
smallest limits that allow a particular pattern to match a given subject
-string. This is done by calling \fBpcre_exec()\fP repeatedly with different
+string. This is done by calling \fBpcre[16]_exec()\fP repeatedly with different
limits.
.
.
@@ -155,7 +156,7 @@
.sp
This reads the current limits (soft and hard) using \fBgetrlimit()\fP, then
attempts to increase the soft limit to 100Mb using \fBsetrlimit()\fP. You must
-do this before calling \fBpcre_exec()\fP.
+do this before calling \fBpcre[16]_exec()\fP.
.
.
.SS "Changing stack size in Mac OS X"
@@ -184,6 +185,6 @@
.rs
.sp
.nf
-Last updated: 26 August 2011
-Copyright (c) 1997-2011 University of Cambridge.
+Last updated: 10 January 2012
+Copyright (c) 1997-2012 University of Cambridge.
.fi
Modified: code/trunk/doc/pcresyntax.3
===================================================================
--- code/trunk/doc/pcresyntax.3 2012-01-09 20:12:58 UTC (rev 860)
+++ code/trunk/doc/pcresyntax.3 2012-01-10 14:54:28 UTC (rev 861)
@@ -9,8 +9,7 @@
.\" HREF
\fBpcrepattern\fP
.\"
-documentation. This document contains just a quick-reference summary of the
-syntax.
+documentation. This document contains a quick-reference summary of the syntax.
.
.
.SH "QUOTING"
@@ -40,7 +39,7 @@
.sp
. any character except newline;
in dotall mode, any character whatsoever
- \eC one byte, even in UTF-8 mode (best avoided)
+ \eC one data unit, even in UTF mode (best avoided)
\ed a decimal digit
\eD a character that is not a decimal digit
\eh a horizontal whitespace character
@@ -58,7 +57,7 @@
\eX an extended Unicode sequence
.sp
In PCRE, by default, \ed, \eD, \es, \eS, \ew, and \eW recognize only ASCII
-characters, even in UTF-8 mode. However, this can be changed by setting the
+characters, even in a UTF mode. However, this can be changed by setting the
PCRE_UCP option.
.
.
@@ -337,7 +336,8 @@
newline-setting options with similar syntax:
.sp
(*NO_START_OPT) no start-match optimization (PCRE_NO_START_OPTIMIZE)
- (*UTF8) set UTF-8 mode (PCRE_UTF8)
+ (*UTF8) set UTF-8 mode: 8-bit library (PCRE_UTF8)
+ (*UTF16) set UTF-16 mode: 16-bit library (PCRE_UTF16)
(*UCP) set PCRE_UCP (use Unicode properties for \ed etc)
.
.
@@ -411,7 +411,9 @@
.sp
(*ACCEPT) force successful match
(*FAIL) force backtrack; synonym (*F)
+ (*MARK:NAME) set name to be passed back; synonym (*:NAME)
.sp
+
The following act only when a subsequent match failure causes a backtrack to
reach them. They all force a match failure, but they differ in what happens
afterwards. Those that advance the start-of-match point do so only if the
@@ -419,15 +421,19 @@
.sp
(*COMMIT) overall failure, no advance of starting point
(*PRUNE) advance to next starting character
- (*SKIP) advance start to current matching position
+ (*PRUNE:NAME) equivalent to (*MARK:NAME)(*PRUNE)
+ (*SKIP) advance to current matching position
+ (*SKIP:NAME) advance to position corresponding to an earlier
+ (*MARK:NAME); if not found, the (*SKIP) is ignored
(*THEN) local failure, backtrack to next alternation
+ (*THEN:NAME) equivalent to (*MARK:NAME)(*THEN)
.
.
.SH "NEWLINE CONVENTIONS"
.rs
.sp
These are recognized only at the very start of the pattern or after a
-(*BSR_...) or (*UTF8) or (*UCP) option.
+(*BSR_...), (*UTF8), (*UTF16) or (*UCP) option.
.sp
(*CR) carriage return only
(*LF) linefeed only
@@ -440,7 +446,7 @@
.rs
.sp
These are recognized only at the very start of the pattern or after a
-(*...) option that sets the newline convention or UTF-8 or UCP mode.
+(*...) option that sets the newline convention or a UTF or UCP mode.
.sp
(*BSR_ANYCRLF) CR, LF, or CRLF
(*BSR_UNICODE) any Unicode newline sequence
@@ -474,6 +480,6 @@
.rs
.sp
.nf
-Last updated: 21 November 2010
-Copyright (c) 1997-2010 University of Cambridge.
+Last updated: 10 January 2012
+Copyright (c) 1997-2012 University of Cambridge.
.fi
