LuaJIT
LuaJIT 2.0 is a just-in-time compiler for the Lua scripting language; it is the preferred way to embed Lua in a plugin. This technote describes the integration issues between X-Plane 10.20b3 OS X/64-bit and LuaJIT. These problems are specific to OS X and 64-bit; 32-bit plugins and Windows/Linux 64-bit plugins are not affected.
LuaJIT 2.0 and Address Space
LuaJIT 2.0 requires that all allocations used by JIT code be in the bottom 2 GB of address space. (This requirement comes from the use of signed 32-bit relative addressing for constants in the generated code.) To meet this requirement, LuaJIT has two modifications from a 'standard' Lua run-time:
- The built-in Lua allocator attempts to grab memory from the lower 2 GB of address space using various OS-specific techniques. The allocator is a copy of dl-malloc sitting on top of this custom bottom-2-gb scheme.
- lua_newstate is inoperative in the 64-bit version of LuaJIT 2.0. This stops client code from providing a custom allocator that ignores the < 2 GB rule (which would cause JIT-code crashes).
LuaJIT 2.0 and X-Plane
LuaJIT 2.0 requires that Mac OS X 64-bit apps have their base addresses customized, as the default OS X memory map sets the entire bottom 4 GB of address space to zero page. X-Plane 10.20 has this fix applied, providing at least 1.3 GB of address space to LuaJIT 2.0 (of the 1.8 GB maximum the custom LuaJIT allocator could use).
The problem: during normal operation, OS X's system allocator and OpenGL stack will use free memory in the bottom 2 GB, potentially exhausting it under high scenery engine load. Plugins utilizing Lua that are loaded late (e.g. in an airplane load) or need more memory mid-run may fail allocation since this critical "bottom 2 GB" of address space is taken.
X-Plane 64-bit for OS X solves this problem by pre-grabbing the entire LuaJIT address space for itself on startup and then providing this address space to Lua-based plugins on demand. A special set of messages can be sent from plugins to X-Plane to allocate and release memory.
You will need to modify LuaJIT to allow the custom allocator API on 64-bit builds. Contact Ben if you need a pre-compiled static library that has the custom allocator API re-enabled.
Sample Code to Use X-Plane's LuaJIT Allocator
/* Include XPLM headers, etc. */ struct lua_alloc_request_t { void * ud; void * ptr; size_t osize; size_t nsize; }; #define ALLOC_OPEN 0x00A110C1 #define ALLOC_REALLOC 0x00A110C2 #define ALLOC_CLOSE 0x00A110C3 /* lua interpreter */ lua_State* l; static void *lj_alloc_create(void) { struct lua_alloc_request_t r = { 0 }; XPLMSendMessageToPlugin(XPLM_PLUGIN_XPLANE, ALLOC_OPEN,&r); return r.ud; } static void lj_alloc_destroy(void *msp) { struct lua_alloc_request_t r = { 0 }; r.ud = msp; XPLMSendMessageToPlugin(XPLM_PLUGIN_XPLANE, ALLOC_CLOSE,&r); } static void *lj_alloc_f(void *msp, void *ptr, size_t osize, size_t nsize) { struct lua_alloc_request_t r = { 0 }; r.ud = msp; r.ptr = ptr; r.osize = osize; r.nsize = nsize; XPLMSendMessageToPlugin(XPLM_PLUGIN_XPLANE, ALLOC_REALLOC,&r); return r.ptr; } /* when you need to init LUA... */ XPLMDataRef lua_alloc_ref = XPLMFindDataRef("sim/operation/prefs/misc/has_lua_alloc"); if(lua_alloc_ref && XPLMGetDatai(lua_alloc_ref)) { /* X-Plane has an allocator for us - we _must_ use it. */ void *ud = lj_alloc_create(); if (ud == NULL) { /* handle error */ } l = lua_newstate(lj_alloc_f, ud); } else { /* use the default allocator. */ l = luaL_newstate(); }