Revision: 922
http://vcs.pcre.org/viewvc?view=rev&revision=922
Author: ph10
Date: 2012-02-20 18:44:42 +0000 (Mon, 20 Feb 2012)
Log Message:
-----------
Set PCRE_EXTRA_USED_JIT when JIT was actually used at runtime. Add /S++ and
-s++ to pcretest to show whether JIT was used or not.
Modified Paths:
--------------
code/trunk/ChangeLog
code/trunk/doc/pcreapi.3
code/trunk/doc/pcretest.1
code/trunk/pcre.h.in
code/trunk/pcre_dfa_exec.c
code/trunk/pcre_exec.c
code/trunk/pcretest.c
code/trunk/testdata/testinput12
code/trunk/testdata/testoutput12
Modified: code/trunk/ChangeLog
===================================================================
--- code/trunk/ChangeLog 2012-02-18 18:45:55 UTC (rev 921)
+++ code/trunk/ChangeLog 2012-02-20 18:44:42 UTC (rev 922)
@@ -32,6 +32,12 @@
(e) When newline was set to CRLF, the metacharacter "." did not recognize
a partial match for a CR character at the end of the subject string.
+
+6. When matching is performed using JIT-compiled code, the flag bit
+ PCRE_EXTRA_USED_JIT is set in the pcre_extra block and can be tested by the
+ caller. This bit is checked by pcretest if JIT is requested by /S++ or -s++
+ (instead of just /S+ or -s+) and the text "(JIT)" added to the output if
+ the bit is set.
Version 8.30 04-February-2012
Modified: code/trunk/doc/pcreapi.3
===================================================================
--- code/trunk/doc/pcreapi.3 2012-02-18 18:45:55 UTC (rev 921)
+++ code/trunk/doc/pcreapi.3 2012-02-20 18:44:42 UTC (rev 922)
@@ -1465,23 +1465,29 @@
In the 16-bit version of this structure, the \fImark\fP field has type
"PCRE_UCHAR16 **".
.P
-The \fIflags\fP field is a bitmap that specifies which of the other fields
-are set. The flag bits are:
+The \fIflags\fP field is mostly used to specify which of the other fields are
+set. The flag bits are:
.sp
- PCRE_EXTRA_STUDY_DATA
+ PCRE_EXTRA_CALLOUT_DATA
PCRE_EXTRA_EXECUTABLE_JIT
+ PCRE_EXTRA_MARK
PCRE_EXTRA_MATCH_LIMIT
PCRE_EXTRA_MATCH_LIMIT_RECURSION
- PCRE_EXTRA_CALLOUT_DATA
+ PCRE_EXTRA_STUDY_DATA
PCRE_EXTRA_TABLES
- PCRE_EXTRA_MARK
+ PCRE_EXTRA_USED_JIT
.sp
Other flag bits should be set to zero. The \fIstudy_data\fP field and sometimes
the \fIexecutable_jit\fP field are set in the \fBpcre_extra\fP block that is
returned by \fBpcre_study()\fP, together with the appropriate flag bits. You
-should not set these yourself, but you may add to the block by setting the
-other fields and their corresponding flag bits.
+should not set these yourself, but you may add to the block by setting other
+fields and their corresponding flag bits.
.P
+The PCRE_EXTRA_USED_JIT flag bit is different to the other bits. It does not
+indicate the presence of data in another field. Instead, it is set by
+\fBpcre_exec()\fP if matching was done using JIT-compiled code, and cleared if
+the interpreter was used. You can inspect this bit after a match.
+.P
The \fImatch_limit\fP field provides a means of preventing PCRE from using up a
vast amount of resources when running patterns that are not going to match,
but which have a very large number of possibilities in their search trees. The
@@ -2084,9 +2090,9 @@
.sp
PCRE_ERROR_JIT_STACKLIMIT (-27)
.sp
-This error is returned when a pattern that was successfully studied using the
-PCRE_STUDY_JIT_COMPILE option is being matched, but the memory available for
-the just-in-time processing stack is not large enough. See the
+This error is returned when a pattern that was successfully studied using a
+JIT compile option is being matched, but the memory available for the
+just-in-time processing stack is not large enough. See the
.\" HREF
\fBpcrejit\fP
.\"
@@ -2646,6 +2652,6 @@
.rs
.sp
.nf
-Last updated: 18 February 2012
+Last updated: 20 February 2012
Copyright (c) 1997-2012 University of Cambridge.
.fi
Modified: code/trunk/doc/pcretest.1
===================================================================
--- code/trunk/doc/pcretest.1 2012-02-18 18:45:55 UTC (rev 921)
+++ code/trunk/doc/pcretest.1 2012-02-20 18:44:42 UTC (rev 922)
@@ -129,15 +129,19 @@
.TP 10
\fB-s\fP or \fB-s+\fP
Behave as if each pattern has the \fB/S\fP modifier; in other words, force each
-pattern to be studied. If \fB-s+\fP is used, the PCRE_STUDY_JIT_COMPILE flag is
+pattern to be studied. If \fB-s+\fP is used, all the JIT compile options are
passed to \fBpcre[16]_study()\fP, causing just-in-time optimization to be set
-up if it is available. If the \fB/I\fP or \fB/D\fP option is present on a
-pattern (requesting output about the compiled pattern), information about the
-result of studying is not included when studying is caused only by \fB-s\fP and
-neither \fB-i\fP nor \fB-d\fP is present on the command line. This behaviour
-means that the output from tests that are run with and without \fB-s\fP should
-be identical, except when options that output information about the actual
-running of a match are set.
+up if it is available, for both full and partial matching. If \fB-s++\fP is
+used, the text "(JIT)" is added to the first output line after a match or no
+match when JIT-compiled code was actually used.
+.P
+If the \fB/I\fP or \fB/D\fP option is present on a pattern (requesting output
+about the compiled pattern), information about the result of studying is not
+included when studying is caused only by \fB-s\fP and neither \fB-i\fP nor
+\fB-d\fP is present on the command line. This behaviour means that the output
+from tests that are run with and without \fB-s\fP should be identical, except
+when options that output information about the actual running of a match are
+set.
.sp
The \fB-M\fP, \fB-t\fP, and \fB-tm\fP options, which give information about
resources used, are likely to produce different output with and without
@@ -316,7 +320,7 @@
twice, the same action is taken for captured substrings. In each case the
remainder is output on the following line with a plus character following the
capture number. Note that this modifier must not immediately follow the /S
-modifier because /S+ has another meaning.
+modifier because /S+ and \S++ have other meanings.
.P
The \fB/=\fP modifier requests that the values of all potential captured
parentheses be output after a match. By default, only those up to the highest
@@ -936,6 +940,6 @@
.rs
.sp
.nf
-Last updated: 14 January 2012
+Last updated: 20 February 2012
Copyright (c) 1997-2012 University of Cambridge.
.fi
Modified: code/trunk/pcre.h.in
===================================================================
--- code/trunk/pcre.h.in 2012-02-18 18:45:55 UTC (rev 921)
+++ code/trunk/pcre.h.in 2012-02-20 18:44:42 UTC (rev 922)
@@ -268,6 +268,7 @@
#define PCRE_EXTRA_MATCH_LIMIT_RECURSION 0x0010
#define PCRE_EXTRA_MARK 0x0020
#define PCRE_EXTRA_EXECUTABLE_JIT 0x0040
+#define PCRE_EXTRA_USED_JIT 0x0080
/* Types */
Modified: code/trunk/pcre_dfa_exec.c
===================================================================
--- code/trunk/pcre_dfa_exec.c 2012-02-18 18:45:55 UTC (rev 921)
+++ code/trunk/pcre_dfa_exec.c 2012-02-20 18:44:42 UTC (rev 922)
@@ -3172,6 +3172,7 @@
md->callout_data = extra_data->callout_data;
if ((flags & PCRE_EXTRA_TABLES) != 0)
md->tables = extra_data->tables;
+ ((pcre_extra *)extra_data)->flags &= ~PCRE_EXTRA_USED_JIT; /* No JIT support here */
}
/* Check that the first field in the block is the magic number. If it is not,
Modified: code/trunk/pcre_exec.c
===================================================================
--- code/trunk/pcre_exec.c 2012-02-18 18:45:55 UTC (rev 921)
+++ code/trunk/pcre_exec.c 2012-02-20 18:44:42 UTC (rev 922)
@@ -6432,7 +6432,11 @@
/* PCRE_ERROR_NULL means that the selected normal or partial matching
mode is not compiled. In this case we simply fallback to interpreter. */
- if (rc != PCRE_ERROR_NULL) return rc;
+ if (rc != PCRE_ERROR_NULL) /* JIT was used */
+ {
+ ((pcre_extra *)extra_data)->flags |= PCRE_EXTRA_USED_JIT;
+ return rc;
+ }
}
#endif
@@ -6467,6 +6471,7 @@
if ((flags & PCRE_EXTRA_CALLOUT_DATA) != 0)
md->callout_data = extra_data->callout_data;
if ((flags & PCRE_EXTRA_TABLES) != 0) tables = extra_data->tables;
+ ((pcre_extra *)extra_data)->flags &= ~PCRE_EXTRA_USED_JIT; /* JIT not used */
}
/* If the exec call supplied NULL for tables, use the inbuilt ones. This
Modified: code/trunk/pcretest.c
===================================================================
--- code/trunk/pcretest.c 2012-02-18 18:45:55 UTC (rev 921)
+++ code/trunk/pcretest.c 2012-02-20 18:44:42 UTC (rev 922)
@@ -1,4 +1,4 @@
-/*************************************************
+/*.************************************************
* PCRE testing program *
*************************************************/
@@ -2132,6 +2132,7 @@
printf(" -S <n> set stack size to <n> megabytes\n");
printf(" -s force each pattern to be studied at basic level\n"
" -s+ force each pattern to be studied, using JIT if available\n"
+ " -s++ ditto, verifying when JIT was actually used\n"
" -t time compilation and execution\n");
printf(" -t <n> time compilation and execution, repeating <n> times\n");
printf(" -tm time execution (matching) only\n");
@@ -2172,6 +2173,7 @@
int debug = 0;
int done = 0;
int all_use_dfa = 0;
+int verify_jit = 0;
int yield = 0;
int stack_size;
@@ -2233,17 +2235,24 @@
while (argc > 1 && argv[op][0] == '-')
{
pcre_uint8 *endptr;
+ char *arg = argv[op];
- if (strcmp(argv[op], "-m") == 0) showstore = 1;
- else if (strcmp(argv[op], "-s") == 0) force_study = 0;
- else if (strcmp(argv[op], "-s+") == 0)
+ if (strcmp(arg, "-m") == 0) showstore = 1;
+ else if (strcmp(arg, "-s") == 0) force_study = 0;
+
+ else if (strncmp(arg, "-s+", 3) == 0)
{
+ arg += 3;
+ if (*arg == '+') { arg++; verify_jit = TRUE; }
+
+ if (*arg != 0) goto BAD_ARG;
+
force_study = 1;
force_study_options = PCRE_STUDY_JIT_COMPILE
| PCRE_STUDY_JIT_PARTIAL_SOFT_COMPILE
| PCRE_STUDY_JIT_PARTIAL_HARD_COMPILE;
}
- else if (strcmp(argv[op], "-16") == 0)
+ else if (strcmp(arg, "-16") == 0)
{
#ifdef SUPPORT_PCRE16
use_pcre16 = 1;
@@ -2252,24 +2261,24 @@
exit(1);
#endif
}
- else if (strcmp(argv[op], "-q") == 0) quiet = 1;
- else if (strcmp(argv[op], "-b") == 0) debug = 1;
- else if (strcmp(argv[op], "-i") == 0) showinfo = 1;
- else if (strcmp(argv[op], "-d") == 0) showinfo = debug = 1;
- else if (strcmp(argv[op], "-M") == 0) default_find_match_limit = TRUE;
+ else if (strcmp(arg, "-q") == 0) quiet = 1;
+ else if (strcmp(arg, "-b") == 0) debug = 1;
+ else if (strcmp(arg, "-i") == 0) showinfo = 1;
+ else if (strcmp(arg, "-d") == 0) showinfo = debug = 1;
+ else if (strcmp(arg, "-M") == 0) default_find_match_limit = TRUE;
#if !defined NODFA
- else if (strcmp(argv[op], "-dfa") == 0) all_use_dfa = 1;
+ else if (strcmp(arg, "-dfa") == 0) all_use_dfa = 1;
#endif
- else if (strcmp(argv[op], "-o") == 0 && argc > 2 &&
+ else if (strcmp(arg, "-o") == 0 && argc > 2 &&
((size_offsets = get_value((pcre_uint8 *)argv[op+1], &endptr)),
*endptr == 0))
{
op++;
argc--;
}
- else if (strcmp(argv[op], "-t") == 0 || strcmp(argv[op], "-tm") == 0)
+ else if (strcmp(arg, "-t") == 0 || strcmp(arg, "-tm") == 0)
{
- int both = argv[op][2] == 0;
+ int both = arg[2] == 0;
int temp;
if (argc > 2 && (temp = get_value((pcre_uint8 *)argv[op+1], &endptr),
*endptr == 0))
@@ -2281,7 +2290,7 @@
else timeitm = LOOPREPEAT;
if (both) timeit = timeitm;
}
- else if (strcmp(argv[op], "-S") == 0 && argc > 2 &&
+ else if (strcmp(arg, "-S") == 0 && argc > 2 &&
((stack_size = get_value((pcre_uint8 *)argv[op+1], &endptr)),
*endptr == 0))
{
@@ -2304,9 +2313,9 @@
#endif
}
#if !defined NOPOSIX
- else if (strcmp(argv[op], "-p") == 0) posix = 1;
+ else if (strcmp(arg, "-p") == 0) posix = 1;
#endif
- else if (strcmp(argv[op], "-C") == 0)
+ else if (strcmp(arg, "-C") == 0)
{
int rc;
unsigned long int lrc;
@@ -2446,15 +2455,16 @@
printf("\n");
goto EXIT;
}
- else if (strcmp(argv[op], "-help") == 0 ||
- strcmp(argv[op], "--help") == 0)
+ else if (strcmp(arg, "-help") == 0 ||
+ strcmp(arg, "--help") == 0)
{
usage();
goto EXIT;
}
else
{
- printf("** Unknown or malformed option %s\n", argv[op]);
+ BAD_ARG:
+ printf("** Unknown or malformed option %s\n", arg);
usage();
yield = 1;
goto EXIT;
@@ -2770,10 +2780,14 @@
do_study = 1;
if (*pp == '+')
{
+ if (*(++pp) == '+')
+ {
+ verify_jit = TRUE;
+ pp++;
+ }
study_options |= PCRE_STUDY_JIT_COMPILE
| PCRE_STUDY_JIT_PARTIAL_SOFT_COMPILE
| PCRE_STUDY_JIT_PARTIAL_HARD_COMPILE;
- pp++;
}
}
else
@@ -3380,6 +3394,7 @@
int start_offset_sign = 1;
int g_notempty = 0;
int use_dfa = 0;
+ int jit_was_used = 0;
*copynames = 0;
*getnames = 0;
@@ -3965,6 +3980,9 @@
count = use_size_offsets/3;
}
}
+
+ if (extra != NULL && (extra->flags & PCRE_EXTRA_USED_JIT) != 0)
+ jit_was_used = TRUE;
/* Matched */
@@ -4023,6 +4041,7 @@
fprintf(outfile, "%2d: ", i/2);
PCHARSV(bptr, use_offsets[i],
use_offsets[i+1] - use_offsets[i], outfile);
+ if (verify_jit && jit_was_used) fprintf(outfile, " (JIT)");
fprintf(outfile, "\n");
if (do_showcaprest || (i == 0 && do_showrest))
{
@@ -4268,14 +4287,15 @@
{
if (markptr == NULL)
{
- fprintf(outfile, "No match\n");
+ fprintf(outfile, "No match");
}
else
{
fprintf(outfile, "No match, mark = ");
PCHARSV(markptr, 0, -1, outfile);
- putc('\n', outfile);
}
+ if (verify_jit && jit_was_used) fprintf(outfile, " (JIT)");
+ putc('\n', outfile);
}
break;
Modified: code/trunk/testdata/testinput12
===================================================================
--- code/trunk/testdata/testinput12 2012-02-18 18:45:55 UTC (rev 921)
+++ code/trunk/testdata/testinput12 2012-02-20 18:44:42 UTC (rev 922)
@@ -15,5 +15,9 @@
/(?(R)a*(?1)|((?R))b)/S+
aaaabcde
+
+/abcd/S++
+ abcd
+ xyz
/-- End of testinput12 --/
Modified: code/trunk/testdata/testoutput12
===================================================================
--- code/trunk/testdata/testoutput12 2012-02-18 18:45:55 UTC (rev 921)
+++ code/trunk/testdata/testoutput12 2012-02-20 18:44:42 UTC (rev 922)
@@ -47,5 +47,11 @@
/(?(R)a*(?1)|((?R))b)/S+
aaaabcde
Error -27 (JIT stack limit reached)
+
+/abcd/S++
+ abcd
+ 0: abcd (JIT)
+ xyz
+No match (JIT)
/-- End of testinput12 --/
