Revision: 708
http://vcs.pcre.org/viewvc?view=rev&revision=708
Author: ph10
Date: 2011-09-23 12:03:03 +0100 (Fri, 23 Sep 2011)
Log Message:
-----------
File tidies for 8.20-RC2 release.
Modified Paths:
--------------
code/trunk/ChangeLog
code/trunk/RunTest
code/trunk/RunTest.bat
code/trunk/configure.ac
code/trunk/doc/html/pcreapi.html
code/trunk/doc/html/pcrejit.html
code/trunk/doc/pcre.txt
code/trunk/doc/pcrejit.3
code/trunk/pcre_compile.c
code/trunk/pcre_exec.c
code/trunk/pcre_internal.h
code/trunk/pcre_jit_compile.c
code/trunk/pcre_study.c
Modified: code/trunk/ChangeLog
===================================================================
--- code/trunk/ChangeLog 2011-09-23 10:32:53 UTC (rev 707)
+++ code/trunk/ChangeLog 2011-09-23 11:03:03 UTC (rev 708)
@@ -1,7 +1,7 @@
ChangeLog for PCRE
------------------
-Version 8.20 12-Sep-2011
+Version 8.20 23-Sep-2011
------------------------
1. Change 37 of 8.13 broke patterns like [:a]...[b:] because it thought it had
@@ -9,7 +9,7 @@
Perl has bugs and confusions, a closing square bracket is no longer allowed
in a POSIX name. This bug also affected patterns with classes that started
with full stops.
-
+
2. If a pattern such as /(a)b|ac/ is matched against "ac", there is no
captured substring, but while checking the failing first alternative,
substring 1 is temporarily captured. If the output vector supplied to
@@ -20,7 +20,7 @@
such as /(a)(b)x|abc/ matched against "abc", where the return code
indicates that fewer than the maximum number of slots in the ovector have
been set.
-
+
3. Related to (2) above: when there are more back references in a pattern than
slots in the output vector, pcre_exec() uses temporary memory during
matching, and copies in the captures as far as possible afterwards. It was
@@ -28,42 +28,42 @@
that only 2/3 is used for passing back captured substrings. Now it uses
only the first 2/3, for compatibility. This is, of course, another edge
case.
-
+
4. Zoltan Herczeg's just-in-time compiler support has been integrated into the
main code base, and can be used by building with --enable-jit. When this is
done, pcregrep automatically uses it unless --disable-pcregrep-jit or the
runtime --no-jit option is given.
-
+
5. When the number of matches in a pcre_dfa_exec() run exactly filled the
ovector, the return from the function was zero, implying that there were
other matches that did not fit. The correct "exactly full" value is now
returned.
-
+
6. If a subpattern that was called recursively or as a subroutine contained
(*PRUNE) or any other control that caused it to give a non-standard return,
invalid errors such as "Error -26 (nested recursion at the same subject
position)" or even infinite loops could occur.
-
+
7. If a pattern such as /a(*SKIP)c|b(*ACCEPT)|/ was studied, it stopped
computing the minimum length on reaching *ACCEPT, and so ended up with the
wrong value of 1 rather than 0. Further investigation indicates that
computing a minimum subject length in the presence of *ACCEPT is difficult
(think back references, subroutine calls), and so I have changed the code
so that no minimum is registered for a pattern that contains *ACCEPT.
-
+
8. If (*THEN) was present in the first (true) branch of a conditional group,
it was not handled as intended.
-
+
9. Replaced RunTest.bat with the much improved version provided by Sheri
Pierce.
-
+
10. A pathological pattern such as /(*ACCEPT)a/ was miscompiled, thinking that
the first byte in a match must be "a".
-
+
11. Change 17 for 8.13 increased the recursion depth for patterns like
/a(?:.)*?a/ drastically. I've improved things by remembering whether a
- pattern contains any instances of (*THEN). If it does not, the old
- optimizations are restored. It would be nice to do this on a per-group
+ pattern contains any instances of (*THEN). If it does not, the old
+ optimizations are restored. It would be nice to do this on a per-group
basis, but at the moment that is not feasible.
Modified: code/trunk/RunTest
===================================================================
--- code/trunk/RunTest 2011-09-23 10:32:53 UTC (rev 707)
+++ code/trunk/RunTest 2011-09-23 11:03:03 UTC (rev 708)
@@ -85,7 +85,7 @@
testdata=../testdata
else
echo "Cannot find the testdata directory"
- exit 1
+ exit 1
fi
# Find which optional facilities are available
@@ -268,7 +268,7 @@
sed 's/fr_FR/fr/' $testdata/wintestoutput3 >test3output
else
locale=
- fi
+ fi
fi
fi
Modified: code/trunk/RunTest.bat
===================================================================
--- code/trunk/RunTest.bat 2011-09-23 10:32:53 UTC (rev 707)
+++ code/trunk/RunTest.bat 2011-09-23 11:03:03 UTC (rev 708)
@@ -1,297 +1,297 @@
-@echo off
-@rem This file must use CRLF linebreaks to function properly
-@rem and user must have external findstr command
-@rem This file was contributed by Ralf Junker, and touched up by
-@rem Daniel Richard G. Tests 10-12 added by Philip H.
-@rem Philip H also changed test 3 to use "wintest" files.
-@rem
-@rem Updated by Tom Fortmann to support explicit test numbers on the command line.
-@rem Added argument validation and added error reporting.
-@rem
-@rem MS Windows batch file to run pcretest on testfiles with the correct
-@rem options.
-@rem
-@rem Sheri Pierce added logic to skip feature dependent tests
-@rem tests 4 5 8 and 12 require utf8 support
-@rem tests 6 9 13 require ucp support
-@rem 10 requires ucp and link size 2
-@rem 14 requires presense of jit support
-@rem 15 requires absence of jit support
-
-@rem Sheri P also added override tests for study and jit testing
-@rem Output is written to newly created subfolders named testout testoutstudy and testoutjit.
-@rem Current dir should be the build dir. The testdata folder should exist in the current dir or in ..
-@rem Copy RunTest.bat to the build dir manually if necessary.
-@rem
-
-setlocal enabledelayedexpansion
-
-if exist testdata set srcdir=.
-if [%srcdir%]==[] set srcdir=..
-if NOT exist %srcdir%\testdata (
-echo distribution testdata folder not found.
-exit /b 1
-goto :eof
-)
-if [%pcretest%]==[] set pcretest=pcretest
-
-%pcretest% -C |findstr /C:"No UTF-8 support" >NUL
-set utf8=%ERRORLEVEL%
-%pcretest% -C |findstr /C:"No Unicode properties support" >NUL
-set ucp=%ERRORLEVEL%
-%pcretest% -C |findstr /C:"No just-in-time compiler support" >NUL
-set jit=%ERRORLEVEL%
-%pcretest% -C |findstr /C:"Internal link size = 2" >NUL
-set link2=%ERRORLEVEL%
-
-set ucpandlink2=0
-if %ucp% EQU 1 (
- if %link2% EQU 0 set ucpandlink2=1
-)
-
-if not exist testout md testout
-if not exist testoutstudy md testoutstudy
-if not exist testoutjit md testoutjit
-
-set do1=no
-set do2=no
-set do3=no
-set do4=no
-set do5=no
-set do6=no
-set do7=no
-set do8=no
-set do9=no
-set do10=no
-set do11=no
-set do12=no
-set do13=no
-set do14=no
-set do15=no
-set all=yes
-
-for %%a in (%*) do (
- set valid=no
- for %%v in (1 2 3 4 5 6 7 8 9 10 11 12 13 14 15) do if %%v == %%a set valid=yes
- if "!valid!" == "yes" (
- set do%%a=yes
- set all=no
-) else (
- echo Invalid test number - %%a!
- echo Usage %0 [ test_number ] ...
- echo Where test_number is one or more optional test numbers 1 through 15, default is all tests.
- exit /b 1
-)
-)
-if "%all%" == "yes" (
- set do1=yes
- set do2=yes
- set do3=yes
- set do4=yes
- set do5=yes
- set do6=yes
- set do7=yes
- set do8=yes
- set do9=yes
- set do10=yes
- set do11=yes
- set do12=yes
- set do13=yes
- set do14=yes
- set do15=yes
-)
-
-if "%do1%" == "yes" call :do1
-if "%do2%" == "yes" call :do2
-if "%do3%" == "yes" call :do3
-if "%do4%" == "yes" call :do4
-if "%do5%" == "yes" call :do5
-if "%do6%" == "yes" call :do6
-if "%do7%" == "yes" call :do7
-if "%do8%" == "yes" call :do8
-if "%do9%" == "yes" call :do9
-if "%do10%" == "yes" call :do10
-if "%do11%" == "yes" call :do11
-if "%do12%" == "yes" call :do12
-if "%do13%" == "yes" call :do13
-if "%do14%" == "yes" call :do14
-if "%do15%" == "yes" call :do15
-goto :eof
-
-:runsub
-@rem Function to execute pcretest and compare the output
-@rem Arguments are as follows:
-@rem
-@rem 1 = test number
-@rem 2 = outputdir
-@rem 3 = test name use double quotes
-@rem 4 - 9 = pcretest options
-
-if [%1] == [] (
- echo Missing test number argument!
- exit /b 1
-)
-
-if [%2] == [] (
- echo Missing outputdir!
- exit /b 1
-)
-
-if [%3] == [] (
- echo Missing test name argument!
- exit /b 1
-)
-
-set testinput=testinput%1
-set testoutput=testoutput%1
-if exist %srcdir%\testdata\win%testinput% (
- set testinput=wintestinput%1
- set testoutput=wintestoutput%1
-)
-
-echo.
-echo Test %1: %3
-%pcretest% %4 %5 %6 %7 %8 %9 %srcdir%\testdata\%testinput%>%2\%testoutput%
-if errorlevel 1 (
- echo Test %1: pcretest failed!
- goto :eof
-)
-
-fc /n %srcdir%\testdata\%testoutput% %2\%testoutput%
-if errorlevel 1 (
- echo Test %1: file compare failed!
- goto :eof
-)
-
-echo Test %1: Passed.
-echo.
-goto :eof
-
-:do1
-call :runsub 1 testout "Main functionality - Compatible with Perl 5.8 and above" -q
-call :runsub 1 testoutstudy "Test with Study Override" -q -s
-if %jit% EQU 1 call :runsub 1 testoutjit "Test with JIT Override" -q -s+
-goto :eof
-
-:do2
- call :runsub 2 testout "API, errors, internals, and non-Perl stuff" -q
- call :runsub 2 testoutstudy "Test with Study Override" -q -s
- if %jit% EQU 1 call :runsub 2 testoutjit "Test with JIT Override" -q -s+
-goto :eof
-
-:do3
- call :runsub 3 testout "Locale-specific features" -q
- call :runsub 3 testoutstudy "Test with Study Override" -q -s
- if %jit% EQU 1 call :runsub 3 testoutjit "Test with JIT Override" -q -s+
-goto :eof
-
-:do4
- if %utf8% EQU 0 (
- echo Test 4 Skipped due to absence of UTF-8 support.
- goto :eof
-)
- call :runsub 4 testout "UTF-8 support - Compatible with Perl 5.8 and above" -q
- call :runsub 4 testoutstudy "Test with Study Override" -q -s
- if %jit% EQU 1 call :runsub 4 testoutjit "Test with JIT Override" -q -s+
-goto :eof
-
-:do5
- if %utf8% EQU 0 (
- echo Test 5 Skipped due to absence of UTF-8 support.
- goto :eof
-)
- call :runsub 5 testout "API, internals, and non-Perl stuff for UTF-8 support" -q
- call :runsub 5 testoutstudy "Test with Study Override" -q -s
- if %jit% EQU 1 call :runsub 5 testoutjit "Test with JIT Override" -q -s+
-goto :eof
-
-:do6
-if %ucp% EQU 0 (
- echo Test 6 Skipped due to absence of ucp support.
- goto :eof
-)
- call :runsub 6 testout "Unicode property support - Compatible with Perl 5.10 and above" -q
- call :runsub 6 testoutstudy "Test with Study Override" -q -s
- if %jit% EQU 1 call :runsub 6 testoutjit "Test with JIT Override" -q -s+
-goto :eof
-
-:do7
- call :runsub 7 testout "DFA matching" -q -dfa
- call :runsub 7 testoutstudy "Test with Study Override" -q -dfa -s
- if %jit% EQU 1 call :runsub 7 testoutjit "Test with JIT Override" -q -dfa -s+
-goto :eof
-
-:do8
- if %utf8% EQU 0 (
- echo Test 8 Skipped due to absence of UTF-8 support.
- goto :eof
-)
- call :runsub 8 testout "DFA matching with UTF-8" -q -dfa
- call :runsub 8 testoutstudy "Test with Study Override" -q -dfa -s
- if %jit% EQU 1 call :runsub 8 testoutjit "Test with JIT Override" -q -dfa -s+
- goto :eof
-
-:do9
- if %ucp% EQU 0 (
- echo Test 9 Skipped due to absence of ucp support.
- goto :eof
-)
- call :runsub 9 testout "DFA matching with Unicode properties" -q -dfa
- call :runsub 9 testoutstudy "Test with Study Override" -q -dfa -s
- if %jit% EQU 1 call :runsub 9 testoutjit "Test with JIT Override" -q -dfa -s+
-goto :eof
-
-:do10
- if %ucpandlink2% EQU 0 (
- echo Test 10 Skipped due to requirements of ucp support AND link size 2.
- goto :eof
-)
- call :runsub 10 testout "Internal offsets and code size tests" -q
- call :runsub 10 testoutstudy "Test with Study Override" -q -s
- if %jit% EQU 1 call :runsub 10 testoutjit "Test with JIT Override" -q -s+
-goto :eof
-
-:do11
- call :runsub 11 testout "Features from Perl 5.10 and above" -q
- call :runsub 11 testoutstudy "Test with Study Override" -q -s
- if %jit% EQU 1 call :runsub 11 testoutjit "Test with JIT Override" -q -s+
-goto :eof
-
-:do12
- if %utf8% EQU 0 (
- echo Test 12 Skipped due to absence of UTF-8 support.
- goto :eof
-)
- call :runsub 12 testout "Features from Perl 5.10 and above w UTF-8" -q
- call :runsub 12 testoutstudy "Test with Study Override" -q -s
- if %jit% EQU 1 call :runsub 12 testoutjit "Test with JIT Override" -q -s+
-goto :eof
-
-:do13
- if %ucp% EQU 0 (
- echo Test 13 Skipped due to absence of ucp support.
- goto :eof
-)
-call :runsub 13 testout "API internals and non-Perl stuff for Unicode property support" -q
-call :runsub 13 testoutstudy "Test with Study Override" -q -s
-if %jit% EQU 1 call :runsub 13 testoutjit "Test with JIT Override" -q -s+
-goto :eof
-
-:do14
-if %jit% EQU 0 (
- echo Test 14 Skipped due to absence of JIT support.
- goto :eof
-)
- call :runsub 14 testout "JIT-specific features - have JIT" -q
- call :runsub 14 testoutstudy "Test with Study Override" -q -s
- call :runsub 14 testoutjit "Test with JIT Override" -q -s+
-goto :eof
-
-:do15
- if %jit% EQU 1 (
- echo Test 15 Skipped due to presence of JIT support.
- goto :eof
-)
- call :runsub 15 testout "JIT-specific features - no JIT" -q
- call :runsub 15 testoutstudy "Test with Study Override" -q -s
-goto :eof
+@echo off
+@rem This file must use CRLF linebreaks to function properly
+@rem and user must have external findstr command
+@rem This file was contributed by Ralf Junker, and touched up by
+@rem Daniel Richard G. Tests 10-12 added by Philip H.
+@rem Philip H also changed test 3 to use "wintest" files.
+@rem
+@rem Updated by Tom Fortmann to support explicit test numbers on the command line.
+@rem Added argument validation and added error reporting.
+@rem
+@rem MS Windows batch file to run pcretest on testfiles with the correct
+@rem options.
+@rem
+@rem Sheri Pierce added logic to skip feature dependent tests
+@rem tests 4 5 8 and 12 require utf8 support
+@rem tests 6 9 13 require ucp support
+@rem 10 requires ucp and link size 2
+@rem 14 requires presense of jit support
+@rem 15 requires absence of jit support
+
+@rem Sheri P also added override tests for study and jit testing
+@rem Output is written to newly created subfolders named testout testoutstudy and testoutjit.
+@rem Current dir should be the build dir. The testdata folder should exist in the current dir or in ..
+@rem Copy RunTest.bat to the build dir manually if necessary.
+@rem
+
+setlocal enabledelayedexpansion
+
+if exist testdata set srcdir=.
+if [%srcdir%]==[] set srcdir=..
+if NOT exist %srcdir%\testdata (
+echo distribution testdata folder not found.
+exit /b 1
+goto :eof
+)
+if [%pcretest%]==[] set pcretest=pcretest
+
+%pcretest% -C |findstr /C:"No UTF-8 support" >NUL
+set utf8=%ERRORLEVEL%
+%pcretest% -C |findstr /C:"No Unicode properties support" >NUL
+set ucp=%ERRORLEVEL%
+%pcretest% -C |findstr /C:"No just-in-time compiler support" >NUL
+set jit=%ERRORLEVEL%
+%pcretest% -C |findstr /C:"Internal link size = 2" >NUL
+set link2=%ERRORLEVEL%
+
+set ucpandlink2=0
+if %ucp% EQU 1 (
+ if %link2% EQU 0 set ucpandlink2=1
+)
+
+if not exist testout md testout
+if not exist testoutstudy md testoutstudy
+if not exist testoutjit md testoutjit
+
+set do1=no
+set do2=no
+set do3=no
+set do4=no
+set do5=no
+set do6=no
+set do7=no
+set do8=no
+set do9=no
+set do10=no
+set do11=no
+set do12=no
+set do13=no
+set do14=no
+set do15=no
+set all=yes
+
+for %%a in (%*) do (
+ set valid=no
+ for %%v in (1 2 3 4 5 6 7 8 9 10 11 12 13 14 15) do if %%v == %%a set valid=yes
+ if "!valid!" == "yes" (
+ set do%%a=yes
+ set all=no
+) else (
+ echo Invalid test number - %%a!
+ echo Usage %0 [ test_number ] ...
+ echo Where test_number is one or more optional test numbers 1 through 15, default is all tests.
+ exit /b 1
+)
+)
+if "%all%" == "yes" (
+ set do1=yes
+ set do2=yes
+ set do3=yes
+ set do4=yes
+ set do5=yes
+ set do6=yes
+ set do7=yes
+ set do8=yes
+ set do9=yes
+ set do10=yes
+ set do11=yes
+ set do12=yes
+ set do13=yes
+ set do14=yes
+ set do15=yes
+)
+
+if "%do1%" == "yes" call :do1
+if "%do2%" == "yes" call :do2
+if "%do3%" == "yes" call :do3
+if "%do4%" == "yes" call :do4
+if "%do5%" == "yes" call :do5
+if "%do6%" == "yes" call :do6
+if "%do7%" == "yes" call :do7
+if "%do8%" == "yes" call :do8
+if "%do9%" == "yes" call :do9
+if "%do10%" == "yes" call :do10
+if "%do11%" == "yes" call :do11
+if "%do12%" == "yes" call :do12
+if "%do13%" == "yes" call :do13
+if "%do14%" == "yes" call :do14
+if "%do15%" == "yes" call :do15
+goto :eof
+
+:runsub
+@rem Function to execute pcretest and compare the output
+@rem Arguments are as follows:
+@rem
+@rem 1 = test number
+@rem 2 = outputdir
+@rem 3 = test name use double quotes
+@rem 4 - 9 = pcretest options
+
+if [%1] == [] (
+ echo Missing test number argument!
+ exit /b 1
+)
+
+if [%2] == [] (
+ echo Missing outputdir!
+ exit /b 1
+)
+
+if [%3] == [] (
+ echo Missing test name argument!
+ exit /b 1
+)
+
+set testinput=testinput%1
+set testoutput=testoutput%1
+if exist %srcdir%\testdata\win%testinput% (
+ set testinput=wintestinput%1
+ set testoutput=wintestoutput%1
+)
+
+echo.
+echo Test %1: %3
+%pcretest% %4 %5 %6 %7 %8 %9 %srcdir%\testdata\%testinput%>%2\%testoutput%
+if errorlevel 1 (
+ echo Test %1: pcretest failed!
+ goto :eof
+)
+
+fc /n %srcdir%\testdata\%testoutput% %2\%testoutput%
+if errorlevel 1 (
+ echo Test %1: file compare failed!
+ goto :eof
+)
+
+echo Test %1: Passed.
+echo.
+goto :eof
+
+:do1
+call :runsub 1 testout "Main functionality - Compatible with Perl 5.8 and above" -q
+call :runsub 1 testoutstudy "Test with Study Override" -q -s
+if %jit% EQU 1 call :runsub 1 testoutjit "Test with JIT Override" -q -s+
+goto :eof
+
+:do2
+ call :runsub 2 testout "API, errors, internals, and non-Perl stuff" -q
+ call :runsub 2 testoutstudy "Test with Study Override" -q -s
+ if %jit% EQU 1 call :runsub 2 testoutjit "Test with JIT Override" -q -s+
+goto :eof
+
+:do3
+ call :runsub 3 testout "Locale-specific features" -q
+ call :runsub 3 testoutstudy "Test with Study Override" -q -s
+ if %jit% EQU 1 call :runsub 3 testoutjit "Test with JIT Override" -q -s+
+goto :eof
+
+:do4
+ if %utf8% EQU 0 (
+ echo Test 4 Skipped due to absence of UTF-8 support.
+ goto :eof
+)
+ call :runsub 4 testout "UTF-8 support - Compatible with Perl 5.8 and above" -q
+ call :runsub 4 testoutstudy "Test with Study Override" -q -s
+ if %jit% EQU 1 call :runsub 4 testoutjit "Test with JIT Override" -q -s+
+goto :eof
+
+:do5
+ if %utf8% EQU 0 (
+ echo Test 5 Skipped due to absence of UTF-8 support.
+ goto :eof
+)
+ call :runsub 5 testout "API, internals, and non-Perl stuff for UTF-8 support" -q
+ call :runsub 5 testoutstudy "Test with Study Override" -q -s
+ if %jit% EQU 1 call :runsub 5 testoutjit "Test with JIT Override" -q -s+
+goto :eof
+
+:do6
+if %ucp% EQU 0 (
+ echo Test 6 Skipped due to absence of ucp support.
+ goto :eof
+)
+ call :runsub 6 testout "Unicode property support - Compatible with Perl 5.10 and above" -q
+ call :runsub 6 testoutstudy "Test with Study Override" -q -s
+ if %jit% EQU 1 call :runsub 6 testoutjit "Test with JIT Override" -q -s+
+goto :eof
+
+:do7
+ call :runsub 7 testout "DFA matching" -q -dfa
+ call :runsub 7 testoutstudy "Test with Study Override" -q -dfa -s
+ if %jit% EQU 1 call :runsub 7 testoutjit "Test with JIT Override" -q -dfa -s+
+goto :eof
+
+:do8
+ if %utf8% EQU 0 (
+ echo Test 8 Skipped due to absence of UTF-8 support.
+ goto :eof
+)
+ call :runsub 8 testout "DFA matching with UTF-8" -q -dfa
+ call :runsub 8 testoutstudy "Test with Study Override" -q -dfa -s
+ if %jit% EQU 1 call :runsub 8 testoutjit "Test with JIT Override" -q -dfa -s+
+ goto :eof
+
+:do9
+ if %ucp% EQU 0 (
+ echo Test 9 Skipped due to absence of ucp support.
+ goto :eof
+)
+ call :runsub 9 testout "DFA matching with Unicode properties" -q -dfa
+ call :runsub 9 testoutstudy "Test with Study Override" -q -dfa -s
+ if %jit% EQU 1 call :runsub 9 testoutjit "Test with JIT Override" -q -dfa -s+
+goto :eof
+
+:do10
+ if %ucpandlink2% EQU 0 (
+ echo Test 10 Skipped due to requirements of ucp support AND link size 2.
+ goto :eof
+)
+ call :runsub 10 testout "Internal offsets and code size tests" -q
+ call :runsub 10 testoutstudy "Test with Study Override" -q -s
+ if %jit% EQU 1 call :runsub 10 testoutjit "Test with JIT Override" -q -s+
+goto :eof
+
+:do11
+ call :runsub 11 testout "Features from Perl 5.10 and above" -q
+ call :runsub 11 testoutstudy "Test with Study Override" -q -s
+ if %jit% EQU 1 call :runsub 11 testoutjit "Test with JIT Override" -q -s+
+goto :eof
+
+:do12
+ if %utf8% EQU 0 (
+ echo Test 12 Skipped due to absence of UTF-8 support.
+ goto :eof
+)
+ call :runsub 12 testout "Features from Perl 5.10 and above w UTF-8" -q
+ call :runsub 12 testoutstudy "Test with Study Override" -q -s
+ if %jit% EQU 1 call :runsub 12 testoutjit "Test with JIT Override" -q -s+
+goto :eof
+
+:do13
+ if %ucp% EQU 0 (
+ echo Test 13 Skipped due to absence of ucp support.
+ goto :eof
+)
+call :runsub 13 testout "API internals and non-Perl stuff for Unicode property support" -q
+call :runsub 13 testoutstudy "Test with Study Override" -q -s
+if %jit% EQU 1 call :runsub 13 testoutjit "Test with JIT Override" -q -s+
+goto :eof
+
+:do14
+if %jit% EQU 0 (
+ echo Test 14 Skipped due to absence of JIT support.
+ goto :eof
+)
+ call :runsub 14 testout "JIT-specific features - have JIT" -q
+ call :runsub 14 testoutstudy "Test with Study Override" -q -s
+ call :runsub 14 testoutjit "Test with JIT Override" -q -s+
+goto :eof
+
+:do15
+ if %jit% EQU 1 (
+ echo Test 15 Skipped due to presence of JIT support.
+ goto :eof
+)
+ call :runsub 15 testout "JIT-specific features - no JIT" -q
+ call :runsub 15 testoutstudy "Test with Study Override" -q -s
+goto :eof
Modified: code/trunk/configure.ac
===================================================================
--- code/trunk/configure.ac 2011-09-23 10:32:53 UTC (rev 707)
+++ code/trunk/configure.ac 2011-09-23 11:03:03 UTC (rev 708)
@@ -11,7 +11,7 @@
m4_define(pcre_major, [8])
m4_define(pcre_minor, [20])
m4_define(pcre_prerelease, [-RC2])
-m4_define(pcre_date, [2011-09-18])
+m4_define(pcre_date, [2011-09-23])
# Libtool shared library interface versions (current:revision:age)
m4_define(libpcre_version, [0:1:0])
Modified: code/trunk/doc/html/pcreapi.html
===================================================================
--- code/trunk/doc/html/pcreapi.html 2011-09-23 10:32:53 UTC (rev 707)
+++ code/trunk/doc/html/pcreapi.html 2011-09-23 11:03:03 UTC (rev 708)
@@ -1311,9 +1311,14 @@
The function <b>pcre_exec()</b> is called to match a subject string against a
compiled pattern, which is passed in the <i>code</i> argument. If the
pattern was studied, the result of the study should be passed in the
-<i>extra</i> argument. This function is the main matching facility of the
-library, and it operates in a Perl-like manner. For specialist use there is
-also an alternative matching function, which is described
+<i>extra</i> argument. You can call <b>pcre_exec()</b> with the same <i>code</i>
+and <i>extra</i> arguments as many times as you like, in order to match
+different subject strings with the same pattern.
+</P>
+<P>
+This function is the main matching facility of the library, and it operates in
+a Perl-like manner. For specialist use there is also an alternative matching
+function, which is described
<a href="#dfamatch">below</a>
in the section about the <b>pcre_dfa_exec()</b> function.
</P>
@@ -2481,7 +2486,7 @@
</P>
<br><a name="SEC24" href="#TOC1">REVISION</a><br>
<P>
-Last updated: 06 September 2011
+Last updated: 23 September 2011
<br>
Copyright © 1997-2011 University of Cambridge.
<br>
Modified: code/trunk/doc/html/pcrejit.html
===================================================================
--- code/trunk/doc/html/pcrejit.html 2011-09-23 10:32:53 UTC (rev 707)
+++ code/trunk/doc/html/pcrejit.html 2011-09-23 11:03:03 UTC (rev 708)
@@ -96,10 +96,14 @@
If the JIT compiler finds an unsupported item, no JIT data is generated. You
can find out if JIT execution is available after studying a pattern by calling
<b>pcre_fullinfo()</b> with the PCRE_INFO_JIT option. A result of 1 means that
-JIT compilationw was successful. A result of 0 means that JIT support is not
+JIT compilation was successful. A result of 0 means that JIT support is not
available, or the pattern was not studied with PCRE_STUDY_JIT_COMPILE, or the
JIT compiler was not able to handle the pattern.
</P>
+<P>
+Once a pattern has been studied, with or without JIT, it can be used as many
+times as you like for matching different subject strings.
+</P>
<br><a name="SEC4" href="#TOC1">UNSUPPORTED OPTIONS AND PATTERN ITEMS</a><br>
<P>
The only <b>pcre_exec()</b> options that are supported for JIT execution are
@@ -145,11 +149,15 @@
<br><a name="SEC6" href="#TOC1">SAVING AND RESTORING COMPILED PATTERNS</a><br>
<P>
The code that is generated by the JIT compiler is architecture-specific, and is
-also position dependent. For those reasons it cannot be saved and restored like
-the bytecode and other data of a compiled pattern. You should be able run
-<b>pcre_study()</b> on a saved and restored pattern, and thereby recreate the
-JIT data, but because JIT compilation uses significant resources, it is
-probably not worth doing this.
+also position dependent. For those reasons it cannot be saved (in a file or
+database) and restored later like the bytecode and other data of a compiled
+pattern. Saving and restoring compiled patterns is not something many people
+do. More detail about this facility is given in the
+<a href="pcreprecompile.html"><b>pcreprecompile</b></a>
+documentation. It should be possible to run <b>pcre_study()</b> on a saved and
+restored pattern, and thereby recreate the JIT data, but because JIT
+compilation uses significant resources, it is probably not worth doing this;
+you might as well recompile the original pattern.
<a name="stackcontrol"></a></P>
<br><a name="SEC7" href="#TOC1">CONTROLLING THE JIT STACK</a><br>
<P>
@@ -265,7 +273,7 @@
</P>
<br><a name="SEC11" href="#TOC1">REVISION</a><br>
<P>
-Last updated: 06 September 2011
+Last updated: 23 September 2011
<br>
Copyright © 1997-2011 University of Cambridge.
<br>
Modified: code/trunk/doc/pcre.txt
===================================================================
--- code/trunk/doc/pcre.txt 2011-09-23 10:32:53 UTC (rev 707)
+++ code/trunk/doc/pcre.txt 2011-09-23 11:03:03 UTC (rev 708)
@@ -1888,11 +1888,15 @@
The function pcre_exec() is called to match a subject string against a
compiled pattern, which is passed in the code argument. If the pattern
was studied, the result of the study should be passed in the extra
- argument. This function is the main matching facility of the library,
- and it operates in a Perl-like manner. For specialist use there is also
- an alternative matching function, which is described below in the sec-
- tion about the pcre_dfa_exec() function.
+ argument. You can call pcre_exec() with the same code and extra argu-
+ ments as many times as you like, in order to match different subject
+ strings with the same pattern.
+ This function is the main matching facility of the library, and it
+ operates in a Perl-like manner. For specialist use there is also an
+ alternative matching function, which is described below in the section
+ about the pcre_dfa_exec() function.
+
In most applications, the pattern will have been compiled (and option-
ally studied) in the same process that calls pcre_exec(). However, it
is possible to save compiled patterns and study data, and then use them
@@ -2976,7 +2980,7 @@
REVISION
- Last updated: 06 September 2011
+ Last updated: 23 September 2011
Copyright (c) 1997-2011 University of Cambridge.
------------------------------------------------------------------------------
@@ -6450,17 +6454,20 @@
If the JIT compiler finds an unsupported item, no JIT data is gener-
ated. You can find out if JIT execution is available after studying a
pattern by calling pcre_fullinfo() with the PCRE_INFO_JIT option. A
- result of 1 means that JIT compilationw was successful. A result of 0
+ result of 1 means that JIT compilation was successful. A result of 0
means that JIT support is not available, or the pattern was not studied
with PCRE_STUDY_JIT_COMPILE, or the JIT compiler was not able to handle
the pattern.
+ Once a pattern has been studied, with or without JIT, it can be used as
+ many times as you like for matching different subject strings.
+
UNSUPPORTED OPTIONS AND PATTERN ITEMS
- The only pcre_exec() options that are supported for JIT execution are
- PCRE_NO_UTF8_CHECK, PCRE_NOTBOL, PCRE_NOTEOL, PCRE_NOTEMPTY, and
- PCRE_NOTEMPTY_ATSTART. Note in particular that partial matching is not
+ The only pcre_exec() options that are supported for JIT execution are
+ PCRE_NO_UTF8_CHECK, PCRE_NOTBOL, PCRE_NOTEOL, PCRE_NOTEMPTY, and
+ PCRE_NOTEMPTY_ATSTART. Note in particular that partial matching is not
supported.
The unsupported pattern items are:
@@ -6482,61 +6489,65 @@
RETURN VALUES FROM JIT EXECUTION
- When a pattern is matched using JIT execution, the return values are
- the same as those given by the interpretive pcre_exec() code, with the
- addition of one new error code: PCRE_ERROR_JIT_STACKLIMIT. This means
- that the memory used for the JIT stack was insufficient. See "Control-
+ When a pattern is matched using JIT execution, the return values are
+ the same as those given by the interpretive pcre_exec() code, with the
+ addition of one new error code: PCRE_ERROR_JIT_STACKLIMIT. This means
+ that the memory used for the JIT stack was insufficient. See "Control-
ling the JIT stack" below for a discussion of JIT stack usage. For com-
- patibility with the interpretive pcre_exec() code, no more than two-
- thirds of the ovector argument is used for passing back captured sub-
+ patibility with the interpretive pcre_exec() code, no more than two-
+ thirds of the ovector argument is used for passing back captured sub-
strings.
- The error code PCRE_ERROR_MATCHLIMIT is returned by the JIT code if
- searching a very large pattern tree goes on for too long, as it is in
- the same circumstance when JIT is not used, but the details of exactly
- what is counted are not the same. The PCRE_ERROR_RECURSIONLIMIT error
+ The error code PCRE_ERROR_MATCHLIMIT is returned by the JIT code if
+ searching a very large pattern tree goes on for too long, as it is in
+ the same circumstance when JIT is not used, but the details of exactly
+ what is counted are not the same. The PCRE_ERROR_RECURSIONLIMIT error
code is never returned by JIT execution.
SAVING AND RESTORING COMPILED PATTERNS
- The code that is generated by the JIT compiler is architecture-spe-
- cific, and is also position dependent. For those reasons it cannot be
- saved and restored like the bytecode and other data of a compiled pat-
- tern. You should be able run pcre_study() on a saved and restored pat-
- tern, and thereby recreate the JIT data, but because JIT compilation
- uses significant resources, it is probably not worth doing this.
+ The code that is generated by the JIT compiler is architecture-spe-
+ cific, and is also position dependent. For those reasons it cannot be
+ saved (in a file or database) and restored later like the bytecode and
+ other data of a compiled pattern. Saving and restoring compiled pat-
+ terns is not something many people do. More detail about this facility
+ is given in the pcreprecompile documentation. It should be possible to
+ run pcre_study() on a saved and restored pattern, and thereby recreate
+ the JIT data, but because JIT compilation uses significant resources,
+ it is probably not worth doing this; you might as well recompile the
+ original pattern.
CONTROLLING THE JIT STACK
When the compiled JIT code runs, it needs a block of memory to use as a
- stack. By default, it uses 32K on the machine stack. However, some
- large or complicated patterns need more than this. The error
- PCRE_ERROR_JIT_STACKLIMIT is given when there is not enough stack.
- Three functions are provided for managing blocks of memory for use as
+ stack. By default, it uses 32K on the machine stack. However, some
+ large or complicated patterns need more than this. The error
+ PCRE_ERROR_JIT_STACKLIMIT is given when there is not enough stack.
+ Three functions are provided for managing blocks of memory for use as
JIT stacks.
- The pcre_jit_stack_alloc() function creates a JIT stack. Its arguments
- are a starting size and a maximum size, and it returns a pointer to an
- opaque structure of type pcre_jit_stack, or NULL if there is an error.
- The pcre_jit_stack_free() function can be used to free a stack that is
- no longer needed. (For the technically minded: the address space is
+ The pcre_jit_stack_alloc() function creates a JIT stack. Its arguments
+ are a starting size and a maximum size, and it returns a pointer to an
+ opaque structure of type pcre_jit_stack, or NULL if there is an error.
+ The pcre_jit_stack_free() function can be used to free a stack that is
+ no longer needed. (For the technically minded: the address space is
allocated by mmap or VirtualAlloc.)
- JIT uses far less memory for recursion than the interpretive code, and
- a maximum stack size of 512K to 1M should be more than enough for any
+ JIT uses far less memory for recursion than the interpretive code, and
+ a maximum stack size of 512K to 1M should be more than enough for any
pattern.
- The pcre_assign_jit_stack() function specifies which stack JIT code
+ The pcre_assign_jit_stack() function specifies which stack JIT code
should use. Its arguments are as follows:
pcre_extra *extra
pcre_jit_callback callback
void *data
- The extra argument must be the result of studying a pattern with
- PCRE_STUDY_JIT_COMPILE. There are three cases for the values of the
+ The extra argument must be the result of studying a pattern with
+ PCRE_STUDY_JIT_COMPILE. There are three cases for the values of the
other two options:
(1) If callback is NULL and data is NULL, an internal 32K block
@@ -6551,18 +6562,18 @@
is used; otherwise the return value must be a valid JIT stack,
the result of calling pcre_jit_stack_alloc().
- You may safely assign the same JIT stack to more than one pattern, as
+ You may safely assign the same JIT stack to more than one pattern, as
long as they are all matched sequentially in the same thread. In a mul-
tithread application, each thread must use its own JIT stack.
- Strictly speaking, even more is allowed. You can assign the same stack
- to any number of patterns as long as they are not used for matching by
+ Strictly speaking, even more is allowed. You can assign the same stack
+ to any number of patterns as long as they are not used for matching by
multiple threads at the same time. For example, you can assign the same
- stack to all compiled patterns, and use a global mutex in the callback
+ stack to all compiled patterns, and use a global mutex in the callback
to wait until the stack is available for use. However, this is an inef-
ficient solution, and not recommended.
- This is a suggestion for how a typical multithreaded program might
+ This is a suggestion for how a typical multithreaded program might
operate:
During thread initalization
@@ -6574,15 +6585,15 @@
Use a one-line callback function
return thread_local_var
- All the functions described in this section do nothing if JIT is not
- available, and pcre_assign_jit_stack() does nothing unless the extra
- argument is non-NULL and points to a pcre_extra block that is the
+ All the functions described in this section do nothing if JIT is not
+ available, and pcre_assign_jit_stack() does nothing unless the extra
+ argument is non-NULL and points to a pcre_extra block that is the
result of a successful study with PCRE_STUDY_JIT_COMPILE.
EXAMPLE CODE
- This is a single-threaded example that specifies a JIT stack without
+ This is a single-threaded example that specifies a JIT stack without
using a callback.
int rc;
@@ -6618,7 +6629,7 @@
REVISION
- Last updated: 06 September 2011
+ Last updated: 23 September 2011
Copyright (c) 1997-2011 University of Cambridge.
------------------------------------------------------------------------------
Modified: code/trunk/doc/pcrejit.3
===================================================================
--- code/trunk/doc/pcrejit.3 2011-09-23 10:32:53 UTC (rev 707)
+++ code/trunk/doc/pcrejit.3 2011-09-23 11:03:03 UTC (rev 708)
@@ -79,7 +79,7 @@
available, or the pattern was not studied with PCRE_STUDY_JIT_COMPILE, or the
JIT compiler was not able to handle the pattern.
.P
-Once a pattern has been studied, with or without JIT, it can be used as many
+Once a pattern has been studied, with or without JIT, it can be used as many
times as you like for matching different subject strings.
.
.
@@ -134,16 +134,16 @@
.rs
.sp
The code that is generated by the JIT compiler is architecture-specific, and is
-also position dependent. For those reasons it cannot be saved (in a file or
+also position dependent. For those reasons it cannot be saved (in a file or
database) and restored later like the bytecode and other data of a compiled
-pattern. Saving and restoring compiled patterns is not something many people
+pattern. Saving and restoring compiled patterns is not something many people
do. More detail about this facility is given in the
.\" HREF
\fBpcreprecompile\fP
.\"
documentation. It should be possible to run \fBpcre_study()\fP on a saved and
restored pattern, and thereby recreate the JIT data, but because JIT
-compilation uses significant resources, it is probably not worth doing this;
+compilation uses significant resources, it is probably not worth doing this;
you might as well recompile the original pattern.
.
.
Modified: code/trunk/pcre_compile.c
===================================================================
--- code/trunk/pcre_compile.c 2011-09-23 10:32:53 UTC (rev 707)
+++ code/trunk/pcre_compile.c 2011-09-23 11:03:03 UTC (rev 708)
@@ -5045,7 +5045,7 @@
PUT2INC(code, 0, oc->number);
}
*code++ = (cd->assert_depth > 0)? OP_ASSERT_ACCEPT : OP_ACCEPT;
-
+
/* Do not set firstbyte after *ACCEPT */
if (firstbyte == REQ_UNSET) firstbyte = REQ_NONE;
}
@@ -5064,7 +5064,7 @@
{
PUT(code, 0, code - bcptr->current_branch - 1);
code += LINK_SIZE;
- cd->external_flags |= PCRE_HASTHEN;
+ cd->external_flags |= PCRE_HASTHEN;
}
}
@@ -6327,7 +6327,7 @@
byte, set it from this character, but revert to none on a zero repeat.
Otherwise, leave the firstbyte value alone, and don't change it on a zero
repeat. */
-
+
if (firstbyte == REQ_UNSET)
{
zerofirstbyte = REQ_NONE;
Modified: code/trunk/pcre_exec.c
===================================================================
--- code/trunk/pcre_exec.c 2011-09-23 10:32:53 UTC (rev 707)
+++ code/trunk/pcre_exec.c 2011-09-23 11:03:03 UTC (rev 708)
@@ -870,18 +870,18 @@
/* VVVVVVVVVVVVVVVVVVVVVVVVV */
/* Non-capturing or atomic group, except for possessive with unlimited
- repeat. Loop for all the alternatives.
-
+ repeat. Loop for all the alternatives.
+
When we get to the final alternative within the brackets, we used to return
the result of a recursive call to match() whatever happened so it was
possible to reduce stack usage by turning this into a tail recursion,
except in the case of a possibly empty group. However, now that there is
the possiblity of (*THEN) occurring in the final alternative, this
optimization is no longer always possible.
-
- We can optimize if we know there are no (*THEN)s in the pattern; at present
- this is the best that can be done.
+ We can optimize if we know there are no (*THEN)s in the pattern; at present
+ this is the best that can be done.
+
MATCH_ONCE is returned when the end of an atomic group is successfully
reached, but subsequent matching fails. It passes back up the tree (causing
captured values to be reset) until the original atomic group level is
@@ -899,18 +899,18 @@
if (op >= OP_SBRA || op == OP_ONCE) md->match_function_type = MATCH_CBEGROUP;
/* If this is not a possibly empty group, and there are no (*THEN)s in
- the pattern, and this is the final alternative, optimize as described
+ the pattern, and this is the final alternative, optimize as described
above. */
else if (!md->hasthen && ecode[GET(ecode, 1)] != OP_ALT)
{
ecode += _pcre_OP_lengths[*ecode];
goto TAIL_RECURSE;
- }
+ }
/* In all other cases, we have to make another call to match(). */
- RMATCH(eptr, ecode + _pcre_OP_lengths[*ecode], offset_top, md, eptrb,
+ RMATCH(eptr, ecode + _pcre_OP_lengths[*ecode], offset_top, md, eptrb,
RM2);
if (rrc != MATCH_NOMATCH &&
(rrc != MATCH_THEN || md->start_match_ptr != ecode))
@@ -1297,31 +1297,31 @@
{
ecode += 1 + LINK_SIZE;
goto TAIL_RECURSE;
- }
+ }
/* A call to match() is required. */
-
+
RMATCH(eptr, ecode + 1 + LINK_SIZE, offset_top, md, eptrb, RM49);
-
+
/* If the result is THEN from within the "true" branch of the condition,
- md->start_match_ptr will point to the original OP_COND, not to the start
- of the branch, so we have do work to see if it matches. If THEN comes
+ md->start_match_ptr will point to the original OP_COND, not to the start
+ of the branch, so we have do work to see if it matches. If THEN comes
from the "false" branch, md->start_match_ptr does point to OP_ALT. */
if (rrc == MATCH_THEN)
{
if (*ecode != OP_ALT)
- {
+ {
do ecode += GET(ecode, 1); while (*ecode == OP_ALT);
ecode -= GET(ecode, 1);
- }
- if (md->start_match_ptr == ecode) rrc = MATCH_NOMATCH;
- }
+ }
+ if (md->start_match_ptr == ecode) rrc = MATCH_NOMATCH;
+ }
RRETURN(rrc);
}
-
+
/* Condition false & no alternative; continue after the group. */
-
+
else
{
ecode += 1 + LINK_SIZE;
Modified: code/trunk/pcre_internal.h
===================================================================
--- code/trunk/pcre_internal.h 2011-09-23 10:32:53 UTC (rev 707)
+++ code/trunk/pcre_internal.h 2011-09-23 11:03:03 UTC (rev 708)
@@ -1821,7 +1821,7 @@
BOOL notempty_atstart; /* Empty string match at start not wanted */
BOOL hitend; /* Hit the end of the subject at some point */
BOOL bsr_anycrlf; /* \R is just any CRLF, not full Unicode */
- BOOL hasthen; /* Pattern contains (*THEN) */
+ BOOL hasthen; /* Pattern contains (*THEN) */
const uschar *start_code; /* For use when recursing */
USPTR start_subject; /* Start of the subject string */
USPTR end_subject; /* End of the subject string */
Modified: code/trunk/pcre_jit_compile.c
===================================================================
--- code/trunk/pcre_jit_compile.c 2011-09-23 10:32:53 UTC (rev 707)
+++ code/trunk/pcre_jit_compile.c 2011-09-23 11:03:03 UTC (rev 708)
@@ -5943,7 +5943,7 @@
common->nltype = NLTYPE_FIXED;
switch(re->options & PCRE_NEWLINE_BITS)
{
- case 0:
+ case 0:
/* Compile-time default */
switch (NEWLINE)
{
Modified: code/trunk/pcre_study.c
===================================================================
--- code/trunk/pcre_study.c 2011-09-23 10:32:53 UTC (rev 707)
+++ code/trunk/pcre_study.c 2011-09-23 11:03:03 UTC (rev 708)
@@ -138,14 +138,14 @@
case OP_ACCEPT:
case OP_ASSERT_ACCEPT:
- return -1;
+ return -1;
/* Reached end of a branch; if it's a ket it is the end of a nested
call. If it's ALT it is an alternation in a nested call. If it is END it's
- the end of the outer call. All can be handled by the same code. If an
- ACCEPT was previously encountered, use the length that was in force at that
+ the end of the outer call. All can be handled by the same code. If an
+ ACCEPT was previously encountered, use the length that was in force at that
time, and pass back the shortest ACCEPT length. */
-
+
case OP_ALT:
case OP_KET:
case OP_KETRMAX:
@@ -1291,7 +1291,7 @@
case -3: *errorptr = "internal error: opcode not recognized"; return NULL;
default: break;
}
-
+
/* If a set of starting bytes has been identified, or if the minimum length is
greater than zero, or if JIT optimization has been requested, get a pcre_extra
block and a pcre_study_data block. The study data is put in the latter, which
@@ -1333,12 +1333,12 @@
zero - the interpretive pcre_exec() and pcre_dfa_exec() needn't waste time
checking the zero case. */
- if (min > 0)
+ if (min > 0)
{
study->flags |= PCRE_STUDY_MINLEN;
study->minlength = min;
}
- else study->minlength = 0;
+ else study->minlength = 0;
/* If JIT support was compiled and requested, attempt the JIT compilation.
If no starting bytes were found, and the minimum length is zero, and JIT
