Revision: 921
http://vcs.pcre.org/viewvc?view=rev&revision=921
Author: ph10
Date: 2012-02-18 18:45:55 +0000 (Sat, 18 Feb 2012)
Log Message:
-----------
Update documentation for partial matching support in JIT.
Modified Paths:
--------------
code/trunk/doc/pcreapi.3
code/trunk/doc/pcrejit.3
code/trunk/doc/pcrepartial.3
Modified: code/trunk/doc/pcreapi.3
===================================================================
--- code/trunk/doc/pcreapi.3 2012-02-18 08:35:01 UTC (rev 920)
+++ code/trunk/doc/pcreapi.3 2012-02-18 18:45:55 UTC (rev 921)
@@ -962,12 +962,18 @@
wants to pass any of the other fields to \fBpcre_exec()\fP or
\fBpcre_dfa_exec()\fP, it must set up its own \fBpcre_extra\fP block.
.P
-The second argument of \fBpcre_study()\fP contains option bits. There is only
-one option: PCRE_STUDY_JIT_COMPILE. If this is set, and the just-in-time
-compiler is available, the pattern is further compiled into machine code that
-executes much faster than the \fBpcre_exec()\fP matching function. If
-the just-in-time compiler is not available, this option is ignored. All other
-bits in the \fIoptions\fP argument must be zero.
+The second argument of \fBpcre_study()\fP contains option bits. There are three
+options:
+.sp
+ PCRE_STUDY_JIT_COMPILE
+ PCRE_STUDY_JIT_PARTIAL_HARD_COMPILE
+ PCRE_STUDY_JIT_PARTIAL_SOFT_COMPILE
+.sp
+If any of these are set, and the just-in-time compiler is available, the
+pattern is further compiled into machine code that executes much faster than
+the \fBpcre_exec()\fP interpretive matching function. If the just-in-time
+compiler is not available, these options are ignored. All other bits in the
+\fIoptions\fP argument must be zero.
.P
JIT compilation is a heavyweight optimization. It can take some time for
patterns to be analyzed, and for one-off matches and simple patterns the
@@ -991,8 +997,8 @@
study data by calling \fBpcre_free_study()\fP. This function was added to the
API for release 8.20. For earlier versions, the memory could be freed with
\fBpcre_free()\fP, just like the pattern itself. This will still work in cases
-where PCRE_STUDY_JIT_COMPILE is not used, but it is advisable to change to the
-new function when convenient.
+where JIT optimization is not used, but it is advisable to change to the new
+function when convenient.
.P
This is a typical way in which \fBpcre_study\fP() is used (except that in a
real application there should be tests for errors):
@@ -1025,14 +1031,12 @@
matching. (In 16-bit mode, the bitmap is used for 16-bit values less than 256.)
.P
These two optimizations apply to both \fBpcre_exec()\fP and
-\fBpcre_dfa_exec()\fP. However, they are not used by \fBpcre_exec()\fP if
-\fBpcre_study()\fP is called with the PCRE_STUDY_JIT_COMPILE option, and
-just-in-time compiling is successful. The optimizations can be disabled by
-setting the PCRE_NO_START_OPTIMIZE option when calling \fBpcre_exec()\fP or
-\fBpcre_dfa_exec()\fP. You might want to do this if your pattern contains
-callouts or (*MARK) (which cannot be handled by the JIT compiler), and you want
-to make use of these facilities in cases where matching fails. See the
-discussion of PCRE_NO_START_OPTIMIZE
+\fBpcre_dfa_exec()\fP, and the information is also used by the JIT compiler.
+The optimizations can be disabled by setting the PCRE_NO_START_OPTIMIZE option
+when calling \fBpcre_exec()\fP or \fBpcre_dfa_exec()\fP, but if this is done,
+JIT execution is also disabled. You might want to do this if your pattern
+contains callouts or (*MARK) and you want to make use of these facilities in
+cases where matching fails. See the discussion of PCRE_NO_START_OPTIMIZE
.\" HTML <a href="#execoptions">
.\" </a>
below.
@@ -1205,12 +1209,11 @@
.sp
PCRE_INFO_JIT
.sp
-Return 1 if the pattern was studied with the PCRE_STUDY_JIT_COMPILE option, and
+Return 1 if the pattern was studied with one of the JIT options, and
just-in-time compiling was successful. The fourth argument should point to an
\fBint\fP variable. A return value of 0 means that JIT support is not available
-in this version of PCRE, or that the pattern was not studied with the
-PCRE_STUDY_JIT_COMPILE option, or that the JIT compiler could not handle this
-particular pattern. See the
+in this version of PCRE, or that the pattern was not studied with a JIT option,
+or that the JIT compiler could not handle this particular pattern. See the
.\" HREF
\fBpcrejit\fP
.\"
@@ -1218,9 +1221,9 @@
.sp
PCRE_INFO_JITSIZE
.sp
-If the pattern was successfully studied with the PCRE_STUDY_JIT_COMPILE option,
-return the size of the JIT compiled code, otherwise return zero. The fourth
-argument should point to a \fBsize_t\fP variable.
+If the pattern was successfully studied with a JIT option, return the size of
+the JIT compiled code, otherwise return zero. The fourth argument should point
+to a \fBsize_t\fP variable.
.sp
PCRE_INFO_LASTLITERAL
.sp
@@ -1492,11 +1495,10 @@
in the subject string.
.P
When \fBpcre_exec()\fP is called with a pattern that was successfully studied
-with the PCRE_STUDY_JIT_COMPILE option, the way that the matching is executed
-is entirely different. However, there is still the possibility of runaway
-matching that goes on for a very long time, and so the \fImatch_limit\fP value
-is also used in this case (but in a different way) to limit how long the
-matching can continue.
+with a JIT option, the way that the matching is executed is entirely different.
+However, there is still the possibility of runaway matching that goes on for a
+very long time, and so the \fImatch_limit\fP value is also used in this case
+(but in a different way) to limit how long the matching can continue.
.P
The default value for the limit can be set when PCRE is built; the default
default is 10 million, which handles all but the most extreme cases. You can
@@ -1514,8 +1516,7 @@
Limiting the recursion depth limits the amount of machine stack that can be
used, or, when PCRE has been compiled to use memory on the heap instead of the
stack, the amount of heap memory that can be used. This limit is not relevant,
-and is ignored, if the pattern was successfully studied with
-PCRE_STUDY_JIT_COMPILE.
+and is ignored, when matching is done using JIT compiled code.
.P
The default value for \fImatch_limit_recursion\fP can be set when PCRE is
built; the default default is the same value as the default for
@@ -1572,15 +1573,15 @@
The unused bits of the \fIoptions\fP argument for \fBpcre_exec()\fP must be
zero. The only bits that may be set are PCRE_ANCHORED, PCRE_NEWLINE_\fIxxx\fP,
PCRE_NOTBOL, PCRE_NOTEOL, PCRE_NOTEMPTY, PCRE_NOTEMPTY_ATSTART,
-PCRE_NO_START_OPTIMIZE, PCRE_NO_UTF8_CHECK, PCRE_PARTIAL_SOFT, and
-PCRE_PARTIAL_HARD.
+PCRE_NO_START_OPTIMIZE, PCRE_NO_UTF8_CHECK, PCRE_PARTIAL_HARD, and
+PCRE_PARTIAL_SOFT.
.P
-If the pattern was successfully studied with the PCRE_STUDY_JIT_COMPILE option,
-the only supported options for JIT execution are PCRE_NO_UTF8_CHECK,
-PCRE_NOTBOL, PCRE_NOTEOL, PCRE_NOTEMPTY, and PCRE_NOTEMPTY_ATSTART. Note in
-particular that partial matching is not supported. If an unsupported option is
-used, JIT execution is disabled and the normal interpretive code in
-\fBpcre_exec()\fP is run.
+If the pattern was successfully studied with one of the just-in-time (JIT)
+compile options, the only supported options for JIT execution are
+PCRE_NO_UTF8_CHECK, PCRE_NOTBOL, PCRE_NOTEOL, PCRE_NOTEMPTY,
+PCRE_NOTEMPTY_ATSTART, PCRE_PARTIAL_HARD, and PCRE_PARTIAL_SOFT. If an
+unsupported option is used, JIT execution is disabled and the normal
+interpretive code in \fBpcre_exec()\fP is run.
.sp
PCRE_ANCHORED
.sp
@@ -1699,7 +1700,8 @@
"no match", the callouts do occur, and that items such as (*COMMIT) and (*MARK)
are considered at every possible starting position in the subject string. If
PCRE_NO_START_OPTIMIZE is set at compile time, it cannot be unset at matching
-time.
+time. The use of PCRE_NO_START_OPTIMIZE disables JIT execution; when it is set,
+matching is always done using interpretively.
.P
Setting PCRE_NO_START_OPTIMIZE can change the outcome of a matching operation.
Consider the pattern
@@ -2644,6 +2646,6 @@
.rs
.sp
.nf
-Last updated: 21 January 2012
+Last updated: 18 February 2012
Copyright (c) 1997-2012 University of Cambridge.
.fi
Modified: code/trunk/doc/pcrejit.3
===================================================================
--- code/trunk/doc/pcrejit.3 2012-02-18 08:35:01 UTC (rev 920)
+++ code/trunk/doc/pcrejit.3 2012-02-18 18:45:55 UTC (rev 921)
@@ -40,15 +40,13 @@
MIPS 32-bit
Power PC 32-bit and 64-bit
.sp
-The Power PC support is designated as experimental because it has not been
-fully tested. If --enable-jit is set on an unsupported platform, compilation
-fails.
+If --enable-jit is set on an unsupported platform, compilation fails.
.P
A program that is linked with PCRE 8.20 or later can tell if JIT support is
available by calling \fBpcre_config()\fP with the PCRE_CONFIG_JIT option. The
result is 1 when JIT is available, and 0 otherwise. However, a simple program
does not need to check this in order to use JIT. The API is implemented in a
-way that falls back to the ordinary PCRE code if JIT is not available.
+way that falls back to the interpretive code if JIT is not available.
.P
If your program may sometimes be linked with versions of PCRE that are older
than 8.20, but you want to use JIT when it is available, you can test
@@ -66,7 +64,7 @@
\fBpcre_exec()\fP.
.sp
(2) Use \fBpcre_free_study()\fP to free the \fBpcre_extra\fP block when it is
- no longer needed instead of just freeing it yourself. This
+ no longer needed, instead of just freeing it yourself. This
ensures that any JIT data is also freed.
.sp
For a program that may be linked with pre-8.20 versions of PCRE, you can insert
@@ -84,6 +82,20 @@
pcre_free(study_ptr);
#endif
.sp
+PCRE_STUDY_JIT_COMPILE requests the JIT compiler to generate code for complete
+matches. If you want to run partial matches using the PCRE_PARTIAL_HARD or
+PCRE_PARTIAL_SOFT options of \fBpcre_exec()\fP, you should set one or both of
+the following options in addition to, or instead of, PCRE_STUDY_JIT_COMPILE
+when you call \fBpcre_study()\fP:
+.sp
+ PCRE_STUDY_JIT_PARTIAL_HARD_COMPILE
+ PCRE_STUDY_JIT_PARTIAL_SOFT_COMPILE
+.sp
+The JIT compiler generates different optimized code for each of the three
+modes (normal, soft partial, hard partial). When \fBpcre_exec()\fP is called,
+the appropriate code is run if it is available. Otherwise, the pattern is
+matched using interpretive code.
+.P
In some circumstances you may need to call additional functions. These are
described in the section entitled
.\" HTML <a href="#stackcontrol">
@@ -92,12 +104,13 @@
.\"
below.
.P
-If JIT support is not available, PCRE_STUDY_JIT_COMPILE is ignored, and no JIT
-data is set up. Otherwise, the compiled pattern is passed to the JIT compiler,
-which turns it into machine code that executes much faster than the normal
-interpretive code. When \fBpcre_exec()\fP is passed a \fBpcre_extra\fP block
-containing a pointer to JIT code, it obeys that instead of the normal code. The
-result is identical, but the code runs much faster.
+If JIT support is not available, PCRE_STUDY_JIT_COMPILE etc. are ignored, and
+no JIT data is created. Otherwise, the compiled pattern is passed to the JIT
+compiler, which turns it into machine code that executes much faster than the
+normal interpretive code. When \fBpcre_exec()\fP is passed a \fBpcre_extra\fP
+block containing a pointer to JIT code of the appropriate mode (normal or
+hard/soft partial), it obeys that code instead of running the interpreter. The
+result is identical, but the compiled JIT code runs much faster.
.P
There are some \fBpcre_exec()\fP options that are not supported for JIT
execution. There are also some pattern items that JIT cannot handle. Details
@@ -108,8 +121,8 @@
can find out if JIT execution is available after studying a pattern by calling
\fBpcre_fullinfo()\fP with the PCRE_INFO_JIT option. A result of 1 means that
JIT compilation was successful. A result of 0 means that JIT support is not
-available, or the pattern was not studied with PCRE_STUDY_JIT_COMPILE, or the
-JIT compiler was not able to handle the pattern.
+available, or the pattern was not studied with PCRE_STUDY_JIT_COMPILE etc., or
+the JIT compiler was not able to handle the pattern.
.P
Once a pattern has been studied, with or without JIT, it can be used as many
times as you like for matching different subject strings.
@@ -119,9 +132,8 @@
.rs
.sp
The only \fBpcre_exec()\fP options that are supported for JIT execution are
-PCRE_NO_UTF8_CHECK, PCRE_NOTBOL, PCRE_NOTEOL, PCRE_NOTEMPTY, and
-PCRE_NOTEMPTY_ATSTART. Note in particular that partial matching is not
-supported.
+PCRE_NO_UTF8_CHECK, PCRE_NOTBOL, PCRE_NOTEOL, PCRE_NOTEMPTY,
+PCRE_NOTEMPTY_ATSTART, PCRE_PARTIAL_HARD, and PCRE_PARTIAL_SOFT.
.P
The unsupported pattern items are:
.sp
@@ -210,8 +222,8 @@
void *data
.sp
The \fIextra\fP argument must be the result of studying a pattern with
-PCRE_STUDY_JIT_COMPILE. There are three cases for the values of the other two
-options:
+PCRE_STUDY_JIT_COMPILE etc. There are three cases for the values of the other
+two options:
.sp
(1) If \fIcallback\fP is NULL and \fIdata\fP is NULL, an internal 32K block
on the machine stack is used.
@@ -250,7 +262,7 @@
All the functions described in this section do nothing if JIT is not available,
and \fBpcre_assign_jit_stack()\fP does nothing unless the \fBextra\fP argument
is non-NULL and points to a \fBpcre_extra\fP block that is the result of a
-successful study with PCRE_STUDY_JIT_COMPILE.
+successful study with PCRE_STUDY_JIT_COMPILE etc.
.
.
.\" HTML <a name="stackfaq"></a>
@@ -367,6 +379,6 @@
.rs
.sp
.nf
-Last updated: 08 January 2012
+Last updated: 18 February 2012
Copyright (c) 1997-2012 University of Cambridge.
.fi
Modified: code/trunk/doc/pcrepartial.3
===================================================================
--- code/trunk/doc/pcrepartial.3 2012-02-18 08:35:01 UTC (rev 920)
+++ code/trunk/doc/pcrepartial.3 2012-02-18 18:45:55 UTC (rev 921)
@@ -32,9 +32,18 @@
the details differ between the two types of matching function. If both options
are set, PCRE_PARTIAL_HARD takes precedence.
.P
-Setting a partial matching option disables the use of any just-in-time code
-that was set up by studying the compiled pattern with the
-PCRE_STUDY_JIT_COMPILE option. It also disables two of PCRE's standard
+If you want to use partial matching with just-in-time optimized code, you must
+call \fBpcre_study()\fP or \fBpcre16_study()\fP with one or both of these
+options:
+.sp
+ PCRE_STUDY_JIT_PARTIAL_SOFT_COMPILE
+ PCRE_STUDY_JIT_PARTIAL_HARD_COMPILE
+.sp
+PCRE_STUDY_JIT_COMPILE should also be set if you are going to run non-partial
+matches on the same pattern. If the appropriate JIT study mode has not been set
+for a match, the interpretive matching code is used.
+.P
+Setting a partial matching option disables two of PCRE's standard
optimizations. PCRE remembers the last literal data unit in a pattern, and
abandons matching immediately if it is not present in the subject string. This
optimization cannot be used for a subject string that might match only
@@ -412,6 +421,6 @@
.rs
.sp
.nf
-Last updated: 21 January 2012
+Last updated: 18 February 2012
Copyright (c) 1997-2012 University of Cambridge.
.fi
