Re: [pcre-dev] [Bug 1174] allow passing of pcre_{malloc, fre…

Top Page
Delete this message
Author: Zoltán Herczeg
Date:  
To: Graycode
CC: pcre-dev
Subject: Re: [pcre-dev] [Bug 1174] allow passing of pcre_{malloc, free, stack_malloc, stack_free, callout} as parameters
Hi Graycode,

thanks for checking the JIT code.

Graycode <xgcode@???> írta:
>In my very limited understanding of JIT, it utilizes two kinds of
> memory allocations. There is SLJIT_MALLOC with a corresponding
> SLJIT_FREE, and I think those could be (or should be) within the
> scope of this discussion.>


True. And there is an SLJIT_CALLOC somewhere, where the data must be zero initialized.

> But there is also utilization of executable memory that enables
> its operation in DEP (Data Execution Prevention) environments.
> There is SLJIT_MALLOC_EXEC / SLJIT_EXECUTABLE_ALLOCATOR, and its
> Windows implementation deploys VirtualAlloc() requesting>
> PAGE_EXECUTE_READWRITE.


There is no way to avoid writing an own allocator nowadays. Mine was simple (I am not an allocator expert) but hopefully it is enough for the JIT. Btw do you know about a static mutex under Windows? If the mutex handle alloction fail now, it might cause trouble...

> In this case JIT has a specific memory requirement. Exposing that
> to general application programs may not be desirable. Doing so may
> lower the probability of successful implementations. That portion
> of memory management should probably remain internalized to the JIT
> implementation and not made accessible.


Agreed. Absolutely not intended to be a general allocator.

> Also it should be mentioned that global read/write variables exist in JIT.
>
> For example sljitLir.c contains:
>    static int compiler_initialized = 0;

>
> And sljitExecAllocator.c contains:
> static struct free_block* free_blocks;
> static sljit_uw allocated_size;
> static sljit_uw total_size;
>


These are protected by mutexes as you figured out.

> There are others such as a couple of important mutexes. These globals
> seem to be used in ways that do not detract from your overall goal.
> In reviewing limited portions of the code, I don't think there would
> be conflict with multiple disparate threads using a single shared
> library instance.


True since they belong to the PCRE lib (at least for now).

> In sljitLir.c line# 272 it seems possible that a timing sequence would
> allow more than 1 thread to invoke init_compiler(), but to me that
> doesn't appear critical. If it is then that portion could be wrapped
> with something like sljit_grab_lock() and sljit_release_lock().


This is intended. init_compiler() must be that simle that it should not cause any trouble if it runs multiple times by multiple threads. This is also true for the floating point unit detection code on those architectures which does not necessary support it.

Regards,
Zoltan