Revision: 926
http://vcs.pcre.org/viewvc?view=rev&revision=926
Author: ph10
Date: 2012-02-22 15:01:32 +0000 (Wed, 22 Feb 2012)
Log Message:
-----------
Re-implement /S++ and -s++ in pcretest in a thread-safe way, using JIT
callback. Removed the PCRE_EXTRA_USED_JIT flag.
Modified Paths:
--------------
code/trunk/ChangeLog
code/trunk/doc/pcreapi.3
code/trunk/doc/pcrejit.3
code/trunk/pcre.h.in
code/trunk/pcre_dfa_exec.c
code/trunk/pcre_exec.c
code/trunk/pcretest.c
Modified: code/trunk/ChangeLog
===================================================================
--- code/trunk/ChangeLog 2012-02-22 14:24:56 UTC (rev 925)
+++ code/trunk/ChangeLog 2012-02-22 15:01:32 UTC (rev 926)
@@ -33,11 +33,9 @@
(e) When newline was set to CRLF, the metacharacter "." did not recognize
a partial match for a CR character at the end of the subject string.
-6. When matching is performed using JIT-compiled code, the flag bit
- PCRE_EXTRA_USED_JIT is set in the pcre_extra block and can be tested by the
- caller. This bit is checked by pcretest if JIT is requested by /S++ or -s++
- (instead of just /S+ or -s+) and the text "(JIT)" added to the output if
- the bit is set.
+6. If JIT is requested using /S++ or -s++ (instead of just /S+ or -s+) when
+ running pcretest, the text "(JIT)" added to the output whenever JIT is
+ actually used to run the match.
7. Individual JIT compile options can be set in pcretest by following -s+[+]
or /S+[+] with a digit between 1 and 7.
Modified: code/trunk/doc/pcreapi.3
===================================================================
--- code/trunk/doc/pcreapi.3 2012-02-22 14:24:56 UTC (rev 925)
+++ code/trunk/doc/pcreapi.3 2012-02-22 15:01:32 UTC (rev 926)
@@ -1465,8 +1465,8 @@
In the 16-bit version of this structure, the \fImark\fP field has type
"PCRE_UCHAR16 **".
.P
-The \fIflags\fP field is mostly used to specify which of the other fields are
-set. The flag bits are:
+The \fIflags\fP field is used to specify which of the other fields are set. The
+flag bits are:
.sp
PCRE_EXTRA_CALLOUT_DATA
PCRE_EXTRA_EXECUTABLE_JIT
@@ -1475,7 +1475,6 @@
PCRE_EXTRA_MATCH_LIMIT_RECURSION
PCRE_EXTRA_STUDY_DATA
PCRE_EXTRA_TABLES
- PCRE_EXTRA_USED_JIT
.sp
Other flag bits should be set to zero. The \fIstudy_data\fP field and sometimes
the \fIexecutable_jit\fP field are set in the \fBpcre_extra\fP block that is
@@ -1483,11 +1482,6 @@
should not set these yourself, but you may add to the block by setting other
fields and their corresponding flag bits.
.P
-The PCRE_EXTRA_USED_JIT flag bit is different to the other bits. It does not
-indicate the presence of data in another field. Instead, it is set by
-\fBpcre_exec()\fP if matching was done using JIT-compiled code, and cleared if
-the interpreter was used. You can inspect this bit after a match.
-.P
The \fImatch_limit\fP field provides a means of preventing PCRE from using up a
vast amount of resources when running patterns that are not going to match,
but which have a very large number of possibilities in their search trees. The
@@ -2652,6 +2646,6 @@
.rs
.sp
.nf
-Last updated: 20 February 2012
+Last updated: 22 February 2012
Copyright (c) 1997-2012 University of Cambridge.
.fi
Modified: code/trunk/doc/pcrejit.3
===================================================================
--- code/trunk/doc/pcrejit.3 2012-02-22 14:24:56 UTC (rev 925)
+++ code/trunk/doc/pcrejit.3 2012-02-22 15:01:32 UTC (rev 926)
@@ -115,7 +115,17 @@
There are some \fBpcre_exec()\fP options that are not supported for JIT
execution. There are also some pattern items that JIT cannot handle. Details
are given below. In both cases, execution automatically falls back to the
-interpretive code.
+interpretive code. If you want to know whether JIT was actually used for a
+particular match, you should arrange for a JIT callback function to be set up
+as described in the section entitled
+.\" HTML <a href="#stackcontrol">
+.\" </a>
+"Controlling the JIT stack"
+.\"
+below, even if you do not need to supply a non-default JIT stack. Such a
+callback function is called whenever JIT code is about to be obeyed. If the
+execution options are not right for JIT execution, the callback function is not
+obeyed.
.P
If the JIT compiler finds an unsupported item, no JIT data is generated. You
can find out if JIT execution is available after studying a pattern by calling
@@ -231,12 +241,18 @@
(2) If \fIcallback\fP is NULL and \fIdata\fP is not NULL, \fIdata\fP must be
a valid JIT stack, the result of calling \fBpcre_jit_stack_alloc()\fP.
.sp
- (3) If \fIcallback\fP not NULL, it must point to a function that is called
- with \fIdata\fP as an argument at the start of matching, in order to
- set up a JIT stack. If the result is NULL, the internal 32K stack
- is used; otherwise the return value must be a valid JIT stack,
- the result of calling \fBpcre_jit_stack_alloc()\fP.
+ (3) If \fIcallback\fP is not NULL, it must point to a function that is
+ called with \fIdata\fP as an argument at the start of matching, in
+ order to set up a JIT stack. If the return from the callback
+ function is NULL, the internal 32K stack is used; otherwise the
+ return value must be a valid JIT stack, the result of calling
+ \fBpcre_jit_stack_alloc()\fP.
.sp
+A callback function is obeyed whenever JIT code is about to be run; it is not
+obeyed when \fBpcre_exec()\fP is called with options that are incompatible for
+JIT execution. A callback function can therefore be used to determine whether a
+match operation was executed by JIT or by the interpreter.
+.P
You may safely assign the same JIT stack to more than one pattern, as long as
they are all matched sequentially in the same thread. In a multithread
application, each thread must use its own JIT stack.
@@ -379,6 +395,6 @@
.rs
.sp
.nf
-Last updated: 18 February 2012
+Last updated: 22 February 2012
Copyright (c) 1997-2012 University of Cambridge.
.fi
Modified: code/trunk/pcre.h.in
===================================================================
--- code/trunk/pcre.h.in 2012-02-22 14:24:56 UTC (rev 925)
+++ code/trunk/pcre.h.in 2012-02-22 15:01:32 UTC (rev 926)
@@ -268,7 +268,6 @@
#define PCRE_EXTRA_MATCH_LIMIT_RECURSION 0x0010
#define PCRE_EXTRA_MARK 0x0020
#define PCRE_EXTRA_EXECUTABLE_JIT 0x0040
-#define PCRE_EXTRA_USED_JIT 0x0080
/* Types */
Modified: code/trunk/pcre_dfa_exec.c
===================================================================
--- code/trunk/pcre_dfa_exec.c 2012-02-22 14:24:56 UTC (rev 925)
+++ code/trunk/pcre_dfa_exec.c 2012-02-22 15:01:32 UTC (rev 926)
@@ -3182,7 +3182,6 @@
md->callout_data = extra_data->callout_data;
if ((flags & PCRE_EXTRA_TABLES) != 0)
md->tables = extra_data->tables;
- ((pcre_extra *)extra_data)->flags &= ~PCRE_EXTRA_USED_JIT; /* No JIT support here */
}
/* Check that the first field in the block is the magic number. If it is not,
Modified: code/trunk/pcre_exec.c
===================================================================
--- code/trunk/pcre_exec.c 2012-02-22 14:24:56 UTC (rev 925)
+++ code/trunk/pcre_exec.c 2012-02-22 15:01:32 UTC (rev 926)
@@ -6435,11 +6435,7 @@
/* PCRE_ERROR_NULL means that the selected normal or partial matching
mode is not compiled. In this case we simply fallback to interpreter. */
- if (rc != PCRE_ERROR_NULL) /* JIT was used */
- {
- ((pcre_extra *)extra_data)->flags |= PCRE_EXTRA_USED_JIT;
- return rc;
- }
+ if (rc != PCRE_ERROR_NULL) return rc;
}
#endif
@@ -6474,7 +6470,6 @@
if ((flags & PCRE_EXTRA_CALLOUT_DATA) != 0)
md->callout_data = extra_data->callout_data;
if ((flags & PCRE_EXTRA_TABLES) != 0) tables = extra_data->tables;
- ((pcre_extra *)extra_data)->flags &= ~PCRE_EXTRA_USED_JIT; /* JIT not used */
}
/* If the exec call supplied NULL for tables, use the inbuilt ones. This
Modified: code/trunk/pcretest.c
===================================================================
--- code/trunk/pcretest.c 2012-02-22 14:24:56 UTC (rev 925)
+++ code/trunk/pcretest.c 2012-02-22 15:01:32 UTC (rev 926)
@@ -613,6 +613,7 @@
static int callout_fail_id;
static int debug_lengths;
static int first_callout;
+static int jit_was_used;
static int locale_set = 0;
static int show_malloc;
static int use_utf;
@@ -1077,6 +1078,7 @@
static pcre_jit_stack* jit_callback(void *arg)
{
+jit_was_used = TRUE;
return (pcre_jit_stack *)arg;
}
@@ -3411,7 +3413,6 @@
int start_offset_sign = 1;
int g_notempty = 0;
int use_dfa = 0;
- int jit_was_used = 0;
*copynames = 0;
*getnames = 0;
@@ -3891,9 +3892,16 @@
}
#endif
+ /* Ensure that there is a JIT callback if we want to verify that JIT was
+ actually used. If jit_stack == NULL, no stack has yet been assigned. */
+
+ if (verify_jit && jit_stack == NULL && extra != NULL)
+ { PCRE_ASSIGN_JIT_STACK(extra, jit_callback, jit_stack); }
+
for (;; gmatched++) /* Loop for /g or /G */
{
markptr = NULL;
+ jit_was_used = FALSE;
if (timeitm > 0)
{
@@ -3998,9 +4006,6 @@
}
}
- if (extra != NULL && (extra->flags & PCRE_EXTRA_USED_JIT) != 0)
- jit_was_used = TRUE;
-
/* Matched */
if (count >= 0)
