feat(luau): expose native breakpoint and step debug API#705
Conversation
Wrap Luau's debug C API so embedders can build line-precise debuggers: lua_breakpoint, lua_singlestep, lua_getlocal/lua_setlocal, and the debugbreak/debugstep callbacks. The callbacks mirror set_interrupt and can return VmState::Yield to suspend the running coroutine. New surface (all gated on the luau feature): - Lua::set_debug_break / set_debug_step / set_single_step and their removers - Function::set_breakpoint - Debug::get_local / set_local / locals The debug callbacks need their own trampoline instead of callback_error_ext: the latter reserves its WrappedFailure at the running frame's base, which shifts the paused function's registers and makes lua_getlocal read the wrong slots. debug_callback reserves at the top of the stack instead.
| pub fn set_local(&self, index: usize, value: Value) -> Result<bool> { | ||
| unsafe { | ||
| // `lua_setlocal` may leave the value on the stack when `index` is out of range; the | ||
| // `StackGuard` restores the top in every case. | ||
| let _sg = StackGuard::new(self.state); | ||
| assert_stack(self.state, 1); | ||
|
|
||
| self.lua.push_value(&value)?; | ||
| let name = ffi::lua_setlocal(self.state, self.level, index as c_int); | ||
| Ok(!name.is_null()) | ||
| } | ||
| } |
There was a problem hiding this comment.
I don't think this is safe operation. Some code can expect that "local" variable has a specific type (and omit checks). Breaking this invariant can be dangerous.
Change set_local return type from Result<bool> to Result<Option<String>>, returning the local's name on success and None when the index is out of range or the frame is native-compiled (Luau guards writes to native frames via LUA_CALLINFO_NATIVE; for interpreted frames a type mismatch surfaces as a Lua RuntimeError, not UB). This makes set_local symmetric with get_local and faithful to what the C API returns. The doc comment now explains both the native-frame guard and the interpreted-frame behaviour explicitly. Add three tests: out-of-range index returns None, type-mismatched write succeeds at the API level then raises a Lua RuntimeError on next VM use, and same-type write returns Ok(Some(name)).
|
This is AI-aided, not fully AI-generated. Let me know if that's a problem, I didn't see anything about it in CONTRIBUTING.md (and couldn't find one actually). On the safety question: you're right that the original API was poorly designed and the doc comment left too much unsaid. I looked at how Luau's own What I changed: the return type is now That said, I'm genuinely happy to restructure this however you think fits best in the codebase, just point me in the direction you'd prefer and I'll make it happen :) |
2 participants
On the Luau backend there is currently no way to do line-precise debugging from mlua.
Lua::set_hookis#[cfg(not(feature = "luau"))],set_interruptonly fires at coarse safepoints (calls, loop back-edges, returns) so it cannot stop on an arbitrary line, and the Luau sandbox stripsdebug.getlocal/getfenvso even when you do pause you cannot read locals.Luau's C API already has everything needed for this, mlua just had not wrapped it. This PR adds safe wrappers following the existing
set_interruptpattern. Nomlua-syschanges, the ffi was already declared.New surface, all behind the
luaufeature:Lua::set_debug_break/set_debug_step(plus removers) andset_single_step. The callbacks receive a&Debugfor the paused frame and can returnVmState::Yieldto suspend the running coroutine, same semantics asset_interrupt.Function::set_breakpoint(line, enabled) -> Option<u32>, wrappinglua_breakpoint. Returns the line Luau actually placed it on (it snaps the breakpoint to the next executable line).Debug::get_local/set_local/locals, wrappinglua_getlocal/lua_setlocal. Luau keeps locals reachable here even though the sandbox removes thedebuglibrary equivalents.Two things worth knowing, both called out in the docs/comments:
VmState::Continueon resume to step past it. The tests exercise this.callback_error_ext. It reserves itsWrappedFailureuserdata at the base of the running frame (lua_insert(state, 1)), which shifts the paused function's registers up by one and makeslua_getlocalread the wrong slots. So there is a small dedicateddebug_callbacktrampoline that reserves at the top of the stack instead, keeping the same error/panic/yield handling.Tests in
tests/luau.rscover breakpoint pause with yield/resume in a coroutine, a breakpoint on a multi-line call expression, single-step over consecutive lines and disabling it, reading and writing locals at a breakpoint, and error propagation from a break callback. The chunks are compiled with optimization 0 / debug 2 so line and local mapping stays stable.