[Pcre-svn] [922] code/trunk: Set PCRE_EXTRA_USED_JIT when JI…

Top Page

Reply to this message
Author: Subversion repository
Date:  
To: pcre-svn
Subject: [Pcre-svn] [922] code/trunk: Set PCRE_EXTRA_USED_JIT when JIT was actually used at runtime.
Revision: 922
          http://vcs.pcre.org/viewvc?view=rev&revision=922
Author:   ph10
Date:     2012-02-20 18:44:42 +0000 (Mon, 20 Feb 2012)


Log Message:
-----------
Set PCRE_EXTRA_USED_JIT when JIT was actually used at runtime. Add /S++ and
-s++ to pcretest to show whether JIT was used or not.

Modified Paths:
--------------
    code/trunk/ChangeLog
    code/trunk/doc/pcreapi.3
    code/trunk/doc/pcretest.1
    code/trunk/pcre.h.in
    code/trunk/pcre_dfa_exec.c
    code/trunk/pcre_exec.c
    code/trunk/pcretest.c
    code/trunk/testdata/testinput12
    code/trunk/testdata/testoutput12


Modified: code/trunk/ChangeLog
===================================================================
--- code/trunk/ChangeLog    2012-02-18 18:45:55 UTC (rev 921)
+++ code/trunk/ChangeLog    2012-02-20 18:44:42 UTC (rev 922)
@@ -32,6 +32,12 @@


     (e) When newline was set to CRLF, the metacharacter "." did not recognize
         a partial match for a CR character at the end of the subject string.
+        
+6.  When matching is performed using JIT-compiled code, the flag bit 
+    PCRE_EXTRA_USED_JIT is set in the pcre_extra block and can be tested by the 
+    caller. This bit is checked by pcretest if JIT is requested by /S++ or -s++ 
+    (instead of just /S+ or -s+) and the text "(JIT)" added to the output if 
+    the bit is set.   



Version 8.30 04-February-2012

Modified: code/trunk/doc/pcreapi.3
===================================================================
--- code/trunk/doc/pcreapi.3    2012-02-18 18:45:55 UTC (rev 921)
+++ code/trunk/doc/pcreapi.3    2012-02-20 18:44:42 UTC (rev 922)
@@ -1465,23 +1465,29 @@
 In the 16-bit version of this structure, the \fImark\fP field has type
 "PCRE_UCHAR16 **".
 .P
-The \fIflags\fP field is a bitmap that specifies which of the other fields
-are set. The flag bits are:
+The \fIflags\fP field is mostly used to specify which of the other fields are
+set. The flag bits are:
 .sp
-  PCRE_EXTRA_STUDY_DATA
+  PCRE_EXTRA_CALLOUT_DATA
   PCRE_EXTRA_EXECUTABLE_JIT
+  PCRE_EXTRA_MARK
   PCRE_EXTRA_MATCH_LIMIT
   PCRE_EXTRA_MATCH_LIMIT_RECURSION
-  PCRE_EXTRA_CALLOUT_DATA
+  PCRE_EXTRA_STUDY_DATA
   PCRE_EXTRA_TABLES
-  PCRE_EXTRA_MARK
+  PCRE_EXTRA_USED_JIT 
 .sp
 Other flag bits should be set to zero. The \fIstudy_data\fP field and sometimes
 the \fIexecutable_jit\fP field are set in the \fBpcre_extra\fP block that is
 returned by \fBpcre_study()\fP, together with the appropriate flag bits. You
-should not set these yourself, but you may add to the block by setting the
-other fields and their corresponding flag bits.
+should not set these yourself, but you may add to the block by setting other
+fields and their corresponding flag bits.
 .P
+The PCRE_EXTRA_USED_JIT flag bit is different to the other bits. It does not 
+indicate the presence of data in another field. Instead, it is set by
+\fBpcre_exec()\fP if matching was done using JIT-compiled code, and cleared if 
+the interpreter was used. You can inspect this bit after a match.
+.P
 The \fImatch_limit\fP field provides a means of preventing PCRE from using up a
 vast amount of resources when running patterns that are not going to match,
 but which have a very large number of possibilities in their search trees. The
@@ -2084,9 +2090,9 @@
 .sp
   PCRE_ERROR_JIT_STACKLIMIT (-27)
 .sp
-This error is returned when a pattern that was successfully studied using the
-PCRE_STUDY_JIT_COMPILE option is being matched, but the memory available for
-the just-in-time processing stack is not large enough. See the
+This error is returned when a pattern that was successfully studied using a
+JIT compile option is being matched, but the memory available for the
+just-in-time processing stack is not large enough. See the
 .\" HREF
 \fBpcrejit\fP
 .\"
@@ -2646,6 +2652,6 @@
 .rs
 .sp
 .nf
-Last updated: 18 February 2012
+Last updated: 20 February 2012
 Copyright (c) 1997-2012 University of Cambridge.
 .fi


Modified: code/trunk/doc/pcretest.1
===================================================================
--- code/trunk/doc/pcretest.1    2012-02-18 18:45:55 UTC (rev 921)
+++ code/trunk/doc/pcretest.1    2012-02-20 18:44:42 UTC (rev 922)
@@ -129,15 +129,19 @@
 .TP 10
 \fB-s\fP or \fB-s+\fP
 Behave as if each pattern has the \fB/S\fP modifier; in other words, force each
-pattern to be studied. If \fB-s+\fP is used, the PCRE_STUDY_JIT_COMPILE flag is
+pattern to be studied. If \fB-s+\fP is used, all the JIT compile options are
 passed to \fBpcre[16]_study()\fP, causing just-in-time optimization to be set
-up if it is available. If the \fB/I\fP or \fB/D\fP option is present on a
-pattern (requesting output about the compiled pattern), information about the
-result of studying is not included when studying is caused only by \fB-s\fP and
-neither \fB-i\fP nor \fB-d\fP is present on the command line. This behaviour
-means that the output from tests that are run with and without \fB-s\fP should
-be identical, except when options that output information about the actual
-running of a match are set.
+up if it is available, for both full and partial matching. If \fB-s++\fP is 
+used, the text "(JIT)" is added to the first output line after a match or no 
+match when JIT-compiled code was actually used.
+.P
+If the \fB/I\fP or \fB/D\fP option is present on a pattern (requesting output
+about the compiled pattern), information about the result of studying is not
+included when studying is caused only by \fB-s\fP and neither \fB-i\fP nor
+\fB-d\fP is present on the command line. This behaviour means that the output
+from tests that are run with and without \fB-s\fP should be identical, except
+when options that output information about the actual running of a match are
+set.
 .sp
 The \fB-M\fP, \fB-t\fP, and \fB-tm\fP options, which give information about
 resources used, are likely to produce different output with and without
@@ -316,7 +320,7 @@
 twice, the same action is taken for captured substrings. In each case the
 remainder is output on the following line with a plus character following the
 capture number. Note that this modifier must not immediately follow the /S
-modifier because /S+ has another meaning.
+modifier because /S+ and \S++ have other meanings.
 .P
 The \fB/=\fP modifier requests that the values of all potential captured
 parentheses be output after a match. By default, only those up to the highest
@@ -936,6 +940,6 @@
 .rs
 .sp
 .nf
-Last updated: 14 January 2012
+Last updated: 20 February 2012
 Copyright (c) 1997-2012 University of Cambridge.
 .fi


Modified: code/trunk/pcre.h.in
===================================================================
--- code/trunk/pcre.h.in    2012-02-18 18:45:55 UTC (rev 921)
+++ code/trunk/pcre.h.in    2012-02-20 18:44:42 UTC (rev 922)
@@ -268,6 +268,7 @@
 #define PCRE_EXTRA_MATCH_LIMIT_RECURSION  0x0010
 #define PCRE_EXTRA_MARK                   0x0020
 #define PCRE_EXTRA_EXECUTABLE_JIT         0x0040
+#define PCRE_EXTRA_USED_JIT               0x0080


/* Types */


Modified: code/trunk/pcre_dfa_exec.c
===================================================================
--- code/trunk/pcre_dfa_exec.c    2012-02-18 18:45:55 UTC (rev 921)
+++ code/trunk/pcre_dfa_exec.c    2012-02-20 18:44:42 UTC (rev 922)
@@ -3172,6 +3172,7 @@
     md->callout_data = extra_data->callout_data;
   if ((flags & PCRE_EXTRA_TABLES) != 0)
     md->tables = extra_data->tables;
+  ((pcre_extra *)extra_data)->flags &= ~PCRE_EXTRA_USED_JIT;  /* No JIT support here */   
   }


/* Check that the first field in the block is the magic number. If it is not,

Modified: code/trunk/pcre_exec.c
===================================================================
--- code/trunk/pcre_exec.c    2012-02-18 18:45:55 UTC (rev 921)
+++ code/trunk/pcre_exec.c    2012-02-20 18:44:42 UTC (rev 922)
@@ -6432,7 +6432,11 @@
   /* PCRE_ERROR_NULL means that the selected normal or partial matching
   mode is not compiled. In this case we simply fallback to interpreter. */


-  if (rc != PCRE_ERROR_NULL) return rc;
+  if (rc != PCRE_ERROR_NULL)     /* JIT was used */
+    {
+    ((pcre_extra *)extra_data)->flags |= PCRE_EXTRA_USED_JIT;
+    return rc;
+    } 
   }
 #endif


@@ -6467,6 +6471,7 @@
   if ((flags & PCRE_EXTRA_CALLOUT_DATA) != 0)
     md->callout_data = extra_data->callout_data;
   if ((flags & PCRE_EXTRA_TABLES) != 0) tables = extra_data->tables;
+  ((pcre_extra *)extra_data)->flags &= ~PCRE_EXTRA_USED_JIT;   /* JIT not used */
   }


/* If the exec call supplied NULL for tables, use the inbuilt ones. This

Modified: code/trunk/pcretest.c
===================================================================
--- code/trunk/pcretest.c    2012-02-18 18:45:55 UTC (rev 921)
+++ code/trunk/pcretest.c    2012-02-20 18:44:42 UTC (rev 922)
@@ -1,4 +1,4 @@
-/*************************************************
+/*.************************************************
 *             PCRE testing program               *
 *************************************************/


@@ -2132,6 +2132,7 @@
 printf("  -S <n>   set stack size to <n> megabytes\n");
 printf("  -s       force each pattern to be studied at basic level\n"
        "  -s+      force each pattern to be studied, using JIT if available\n"
+       "  -s++     ditto, verifying when JIT was actually used\n" 
        "  -t       time compilation and execution\n");
 printf("  -t <n>   time compilation and execution, repeating <n> times\n");
 printf("  -tm      time execution (matching) only\n");
@@ -2172,6 +2173,7 @@
 int debug = 0;
 int done = 0;
 int all_use_dfa = 0;
+int verify_jit = 0;
 int yield = 0;
 int stack_size;


@@ -2233,17 +2235,24 @@
while (argc > 1 && argv[op][0] == '-')
{
pcre_uint8 *endptr;
+ char *arg = argv[op];

-  if (strcmp(argv[op], "-m") == 0) showstore = 1;
-  else if (strcmp(argv[op], "-s") == 0) force_study = 0;
-  else if (strcmp(argv[op], "-s+") == 0)
+  if (strcmp(arg, "-m") == 0) showstore = 1;
+  else if (strcmp(arg, "-s") == 0) force_study = 0;
+   
+  else if (strncmp(arg, "-s+", 3) == 0)
     {
+    arg += 3;
+    if (*arg == '+') { arg++; verify_jit = TRUE; }
+
+    if (*arg != 0) goto BAD_ARG;
+ 
     force_study = 1;
     force_study_options = PCRE_STUDY_JIT_COMPILE
                         | PCRE_STUDY_JIT_PARTIAL_SOFT_COMPILE
                         | PCRE_STUDY_JIT_PARTIAL_HARD_COMPILE;
     }
-  else if (strcmp(argv[op], "-16") == 0)
+  else if (strcmp(arg, "-16") == 0)
     {
 #ifdef SUPPORT_PCRE16
     use_pcre16 = 1;
@@ -2252,24 +2261,24 @@
     exit(1);
 #endif
     }
-  else if (strcmp(argv[op], "-q") == 0) quiet = 1;
-  else if (strcmp(argv[op], "-b") == 0) debug = 1;
-  else if (strcmp(argv[op], "-i") == 0) showinfo = 1;
-  else if (strcmp(argv[op], "-d") == 0) showinfo = debug = 1;
-  else if (strcmp(argv[op], "-M") == 0) default_find_match_limit = TRUE;
+  else if (strcmp(arg, "-q") == 0) quiet = 1;
+  else if (strcmp(arg, "-b") == 0) debug = 1;
+  else if (strcmp(arg, "-i") == 0) showinfo = 1;
+  else if (strcmp(arg, "-d") == 0) showinfo = debug = 1;
+  else if (strcmp(arg, "-M") == 0) default_find_match_limit = TRUE;
 #if !defined NODFA
-  else if (strcmp(argv[op], "-dfa") == 0) all_use_dfa = 1;
+  else if (strcmp(arg, "-dfa") == 0) all_use_dfa = 1;
 #endif
-  else if (strcmp(argv[op], "-o") == 0 && argc > 2 &&
+  else if (strcmp(arg, "-o") == 0 && argc > 2 &&
       ((size_offsets = get_value((pcre_uint8 *)argv[op+1], &endptr)),
         *endptr == 0))
     {
     op++;
     argc--;
     }
-  else if (strcmp(argv[op], "-t") == 0 || strcmp(argv[op], "-tm") == 0)
+  else if (strcmp(arg, "-t") == 0 || strcmp(arg, "-tm") == 0)
     {
-    int both = argv[op][2] == 0;
+    int both = arg[2] == 0;
     int temp;
     if (argc > 2 && (temp = get_value((pcre_uint8 *)argv[op+1], &endptr),
                      *endptr == 0))
@@ -2281,7 +2290,7 @@
     else timeitm = LOOPREPEAT;
     if (both) timeit = timeitm;
     }
-  else if (strcmp(argv[op], "-S") == 0 && argc > 2 &&
+  else if (strcmp(arg, "-S") == 0 && argc > 2 &&
       ((stack_size = get_value((pcre_uint8 *)argv[op+1], &endptr)),
         *endptr == 0))
     {
@@ -2304,9 +2313,9 @@
 #endif
     }
 #if !defined NOPOSIX
-  else if (strcmp(argv[op], "-p") == 0) posix = 1;
+  else if (strcmp(arg, "-p") == 0) posix = 1;
 #endif
-  else if (strcmp(argv[op], "-C") == 0)
+  else if (strcmp(arg, "-C") == 0)
     {
     int rc;
     unsigned long int lrc;
@@ -2446,15 +2455,16 @@
     printf("\n");
     goto EXIT;
     }
-  else if (strcmp(argv[op], "-help") == 0 ||
-           strcmp(argv[op], "--help") == 0)
+  else if (strcmp(arg, "-help") == 0 ||
+           strcmp(arg, "--help") == 0)
     {
     usage();
     goto EXIT;
     }
   else
     {
-    printf("** Unknown or malformed option %s\n", argv[op]);
+    BAD_ARG: 
+    printf("** Unknown or malformed option %s\n", arg);
     usage();
     yield = 1;
     goto EXIT;
@@ -2770,10 +2780,14 @@
         do_study = 1;
         if (*pp == '+')
           {
+          if (*(++pp) == '+')
+            {
+            verify_jit = TRUE;
+            pp++;  
+            }  
           study_options |= PCRE_STUDY_JIT_COMPILE
                         | PCRE_STUDY_JIT_PARTIAL_SOFT_COMPILE
                         | PCRE_STUDY_JIT_PARTIAL_HARD_COMPILE;
-          pp++;
           }
         }
       else
@@ -3380,6 +3394,7 @@
     int start_offset_sign = 1;
     int g_notempty = 0;
     int use_dfa = 0;
+    int jit_was_used = 0; 


     *copynames = 0;
     *getnames = 0;
@@ -3965,6 +3980,9 @@
           count = use_size_offsets/3;
           }
         }
+        
+      if (extra != NULL && (extra->flags & PCRE_EXTRA_USED_JIT) != 0)
+        jit_was_used = TRUE; 


       /* Matched */


@@ -4023,6 +4041,7 @@
             fprintf(outfile, "%2d: ", i/2);
             PCHARSV(bptr, use_offsets[i],
               use_offsets[i+1] - use_offsets[i], outfile);
+            if (verify_jit && jit_was_used) fprintf(outfile, " (JIT)");  
             fprintf(outfile, "\n");
             if (do_showcaprest || (i == 0 && do_showrest))
               {
@@ -4268,14 +4287,15 @@
               {
               if (markptr == NULL)
                 {
-                fprintf(outfile, "No match\n");
+                fprintf(outfile, "No match");
                 }
               else
                 {
                 fprintf(outfile, "No match, mark = ");
                 PCHARSV(markptr, 0, -1, outfile);
-                putc('\n', outfile);
                 }
+              if (verify_jit && jit_was_used) fprintf(outfile, " (JIT)");  
+              putc('\n', outfile);
               }
             break;



Modified: code/trunk/testdata/testinput12
===================================================================
--- code/trunk/testdata/testinput12    2012-02-18 18:45:55 UTC (rev 921)
+++ code/trunk/testdata/testinput12    2012-02-20 18:44:42 UTC (rev 922)
@@ -15,5 +15,9 @@


 /(?(R)a*(?1)|((?R))b)/S+
     aaaabcde
+    
+/abcd/S++
+    abcd
+    xyz  


/-- End of testinput12 --/

Modified: code/trunk/testdata/testoutput12
===================================================================
--- code/trunk/testdata/testoutput12    2012-02-18 18:45:55 UTC (rev 921)
+++ code/trunk/testdata/testoutput12    2012-02-20 18:44:42 UTC (rev 922)
@@ -47,5 +47,11 @@
 /(?(R)a*(?1)|((?R))b)/S+
     aaaabcde
 Error -27 (JIT stack limit reached)
+    
+/abcd/S++
+    abcd
+ 0: abcd (JIT)
+    xyz  
+No match (JIT)


/-- End of testinput12 --/