Re: [pcre-dev] [Bug 1174] allow passing of pcre_{malloc, fre…

On Sat, 12 Nov 2011, Philip Hazel wrote:

> My understanding is that a shared library shares the *code* across
> multiple programs, but each address space (in Unix, each process) gets
> its own copy of any static data. I also assume that multiple threads
> within one process share heap data, but each thread has its own stack.

That makes sense, thanks. Since each program gets its own copy of
library data then I would not see a problem with the library having
storage variables. The values of those variables would be unique to
each program so one would not blow out the data seen by another one.


> The issue of other libraries using PCRE is a new one that has only just
> come to my attention. It means that, even within one thread, different
> parts of the application may need to use different memory management
> functions.

Then the problem is that a program using PCRE might not be aware that
some other libary supporting the application is also using PCRE. Yes,
I see now that could be a conflict and could cause problems if that
was allowed to happen. Sorry it took so long to wrap my head around
what the problem was.

I'd blame the other library for not using a static build of its PCRE
functionality. If linkage of that other library enables the memory
allocation vectors to be made public or common with other PCRE usage
outside of that other library then something seems wrong with that.

It leaves me scratching my head over any differences established in
PCRE's config.h in situations like that. Maybe only one of them
wanted to have SUPPORT_UCP, SUPPORT_UTF8 or SUPPORT_JIT. And perhaps
one of them intended to have MATCH_LIMIT 20000. I don't see how
issues could be resolved by relying upon a program's linkage if there
could be different conflicting PCRE compilations at play.

On the other hand if there is one single shared PCRE libary instance
built which is used by everything, including being used by other
generic linked-in libraries, then that PCRE instance would need to
contain and deploy a maximum of what everything MIGHT want to use.
I wouldn't want to do that, but I can understand others might want to
assuming they never tailor any of PCRE's config.h options to the needs
of any particular applications.

My notes indicate that in order for Windows to run pcretest on test
case #2, pcretest needs to be custom-built with a designated stack
size of at least 2.6 Meg (the default for Windows programs is 1 Meg)
and the library must have been compiled with config.h having
MATCH_LIMIT_RECURSION 7200 (or higher). Yet if a PCRE library's
MATCH_LIMIT_RECURSION were to remain anywhere near 7200 then other
programs using it could blow up having a default stack. Or else the
programs need to be coded to set their match_limit_recursion in an
extra structure (create one if no study was done or it was not helpful)
and pass that in to every exec(). Am I in the minority in defining
options and building PCRE according to the tasks at hand vs. a single
one-for-everything? I feel so ... old-school.

As you tried to tell me, supporting those environments boils down to
PCRE not being allowed to have any global read/write variables.
I might favor having those variables and not supporting the kind of
deployment that induces the problems. I see the causes as being
muddled by questionable build or deployment practices, and so my view
is most likely incorrect (not to mention unpopular).

Your solution of a pointer to storage administered by the caller seems
like a good one.


> > Our threads that use PCRE are created using only 32K
> > total stack size. That's probably your worst-case scenario but PCRE
> > has been able to do the job. Please pat yourself on the back.
>
> Wow. That's nice to know. Thanks.

I've been corrected. Windows manages stack size using 64K allocation
increments. Even though the code specifies 32K and throttles PCRE as
if there was only 32K, the threads are acually getting 64K stack.


Regards,
Graycode