[pcre-dev] [Bug 2618] JIT crash in macOS 11 with ARM64 hardw…

Top Page

Reply to this message
Author: admin
To: pcre-dev
Old-Topics: [pcre-dev] [Bug 2618] New: Patch with JIT support for macOS ARM64 hardware
Subject: [pcre-dev] [Bug 2618] JIT crash in macOS 11 with ARM64 hardware

Saagar Jha <saagar@???> changed:

           What    |Removed                     |Added
                 CC|                            |saagar@???

--- Comment #13 from Saagar Jha <saagar@???> ---
Hi, just stopping by with some additional details you might find interesting.
The issue here is that Apple has added a new, proprietary page permission
mechanism called APRR to their recent ARM chips. A few years back macOS had few
restrictions on JIT code generation, all you had to do was mmap a region
PROT_READ | PROT_WRITE | PROT_EXEC and you could just use it as you would on
any platform. Mojave introduced hardened runtime, an optional feature which
programs could opt-in to and has additional requirements: the mmap call must be
done with the MAP_JIT flag, and you must sign your code with the
com.apple.security.cs.allow-jit entitlement. If you fail to do this (AFAIK)
your process will be killed for a codesigning violation once you try to execute
the code you've generated.

macOS now supports running on Apple's custom ARM chips, which have the
additional APRR feature. This applies additively on top of the other changes I
just mentioned and is enabled for every process regardless of hardened runtime
or codesigning flags. It attempts to prevent having RWX memory (MAP_JIT or not)
that is simultaneously writable and executable from the same thread. Memory
allocated RWX will effectively be made R-X; Apple's intended solution is that
you use the pthread_jit_write_protect_np(0) call in before writing code to this
region (changing effective permissions to RW-) and then using the call again
(pthread_jit_write_protect_np(1)) to set permissions back to R-X so you can
execute it–and then alternating between the two calls when necessary. This has
been a thing on iOS for a few years already–from devices using the A11 chip or
newer, I believe, although it has largely not been of concern to third-party
applications due to the difficulty in mmaping a RWX region in the first place.

With that said, here is what will no longer work:

* Allocating a RWX region with no special flags and attempting to write code to
it and execute it. This will fail because you will not have write permission
due to APRR on ARM chips. Will not work on Intel chips running under hardened
runtime when the code is attempted to be executed due to codesigning, but will
work otherwise.
* Allocating a RWX region with MAP_JIT and the com.apple.security.cs.allow-jit
entitlement: will fail for the same reason above on ARM. Will succeed on
x86_64, even with runtime hardening.
* Mirror mapping/remapping, at least when I tried it this does not work. YMMV.

What will work on ARM, if the memory is correctly allocated based on the
runtime hardening setting:

* Using two threads to work with the RWX region; one with the original
effective permissions (R-X); the second which runs
pthread_jit_write_protect_np(0) to change its permissions to RW- and can now
write code.
* Using one thread: interleaving calls to pthread_jit_write_protect_np when
switching between writing to the JIT region and executing it.
* Using one thread: modifying a proprietary register before you try to write
code into this region, which will disable this APRR feature altogether for the

unsigned long long mask;
__asm__ volatile("mrs %0, s3_4_c15_c2_7" : "=r"(mask): :);
__asm__ volatile("msr s3_4_c15_c2_7, %0" : : "r"(mask & 0xfffffffff0ffffff) :);

This has the upside that you only have to do it once early at program startup
and never have to deal with it again, but the downside is that I don't think
Apple would particularly approve of this.

While I cannot provide remote access, I would be glad to help
test/debug/validate patches to either pcre2 or sljit. And by the way, here is a
"real" application crashing due to this change:

$ lldb ag -- 'Entitlements\.mm' .
(lldb) target create "ag"
Current executable set to 'ag' (arm64).
(lldb) settings set -- target.run-args  "Entitlements\\.mm" "."
(lldb) r
Process 18149 launched: '/opt/local/bin/ag' (arm64)
Process 18149 stopped
* thread #1, queue = 'com.apple.main-thread', stop reason = EXC_BAD_ACCESS
(code=2, address=0x10021c008)
    frame #0: 0x0000000100158798 libpcre.1.dylib`_pcre_jit_compile + 27132
->  0x100158798 <+27132>: str    xzr, [x20, #0x8]
    0x10015879c <+27136>: add    x11, x22, #0x40           ; =0x40 
    0x1001587a0 <+27140>: nop    
    0x1001587a4 <+27144>: ldr    x10, [x9, #0x160]
Target 0: (ag) stopped.
(lldb) bt
* thread #1, queue = 'com.apple.main-thread', stop reason = EXC_BAD_ACCESS
(code=2, address=0x10021c008)
  * frame #0: 0x0000000100158798 libpcre.1.dylib`_pcre_jit_compile + 27132
    frame #1: 0x000000010017465c libpcre.1.dylib`pcre_study + 688
    frame #2: 0x0000000100007784 ag`compile_study + 92
    frame #3: 0x0000000100008448 ag`main + 708
    frame #4: 0x00000001d6f2a844 libdyld.dylib`start + 4

You are receiving this mail because:
You are on the CC list for the bug.