RFC: function coroutine.noyield(fn, ...)

[Aseminaunz]

Rendered

This proposal creates a new standard library function in coroutine:

function coroutine.noyield<A..., R...>(fn: (A...) -> R..., ...: A...): R...

It runs a function within the same thread as the caller, without allowing it to yield, reflecting it in coroutine.isyieldable() and causing coroutine.yield() to error.

[gaymeowing]

Shouldn't it return a success boolean like pcall? and I think nypcall would be a more consistent name as there is ypcall (because pcall used to not yield).

[Aseminaunz]

@gaymeowing

Shouldn't it return a success boolean like pcall?

This isn't a protected call - it's a call that is not allowed to yield. Yielding from the called function causes an error. It means the called function can handle this error, if it doesn't, it gets propagated. Users who want to detect errors in the called function, including an unexpected yield, should wrap the call in pcall.

I don't believe there's much of a use case for coroutine.noyield for specifically intercepting a request to yield - and the semantics of that get considerably more complicated.

coroutine.noyield is for preventing yielding, not catching errors - errors should propagate. It shouldn't return a status boolean. If it does return a state boolean indicating that a yield attempt was made, what should it do? Jump back to the noyield call, and return false? This'd be a goto that'd introduce considerable complexity to the VM. Should it just be a wrapper that creates and runs a coroutine, closing it as soon as it yields? In that case, should it change coroutine.isyieldable() at all? (plus, that's already possible with the standard coroutine library)

A status boolean indicating if the function requested to yield would also introduce an extra return - for information that is probably not useful. If we use coroutine.noyield, we are not handling yields. Our intention is to introduce an error condition in case of a yield, as the VM already does for internal C calls.

I can't really think of a use for coroutine.noyield that doesn't want an error in case of yield or that should require detecting the specific error of a yield that can't be done already with coroutines. If what you want is to implement specific logic to handle a yield, then coroutine.noyield is not for you. Instead, create a coroutine and check if it died after resuming.

coroutine.noyield is for situations where yielding specifically should be treated as an error condition, and should have treatment no different to any other exception thrown. My opinion, but errors are panics, exceptional circumstances, they should have no place in the logic of normal execution. If you really really want to detect a yield, then check if the error string is equal to the expected error when yielding with coroutine.noyield - but I would advise against detecting specific errors thrown by the VM.

If what you want is to enforce that a function does not cause a yield:

coroutine.noyield(myFunc)

If what you want is to enforce that a function does not cause a yield, and handle an error (including yields):

local ok, ret = pcall(coroutine.noyield, myFunc)

if not ok then
    -- Error recovery here
else
    -- Proceed
end

If what you want is to detect yielding and give it a special treatment:

local coro = coroutine.create(myFunc)

local ok, ret = coroutine.resume(myFunc)

if coroutine.status(coro) ~= "dead" then
    -- Yield handling here
else
    -- Proceed
end

I think nypcall would be a more consistent name as there is ypcall (because pcall used to not yield).

Sounds consistent, but this doesn't address one of the motivations behind coroutine.noyield - it's not a protected call. Errors should be thrown, with the stack preserved. If it's not a protected call, then npcall would be a more appropiate name - but I think noyield is easier and more descriptive.

[Aseminaunz]

The proposal isn't to reintroduce Lua's old pcall which did not allow yields inside a protected call, but with a different name. It's to be able to specifically and explicitly block yields, with the added benefit of keeping the stack, as the VM already does with lua_call, in cases that depend on implementation details, with uncertain future guarantees, and are also less explicit and complex than the proposed coroutine.noyield.

[vegorov-rbx]

I don't think we will be accepting this proposal, it requires too many changes to the VM for only a limited utility (given that there are multiple available workarounds).

It shouldn't return a status boolean. If it does return a state boolean indicating that a yield attempt was made, what should it do? Jump back to the noyield call, and return false? This'd be a goto that'd introduce considerable complexity to the VM.

That solution will actually not require any changes to the VM compared to your proposal so in a way will be preferrable.

---

VM changes which will block this PR from being accepted:

• Custom error message will require changes to the VM we do not plan to incorporate
  • Library functions should be implementable using Luau C API, multiple alternative approaches can implement this functionality today in different runtimes, your proposal cannot and will require additional APIs
  • New state data will be required to track current context description
  • I can imagine alternative implementations which are even worse, so I will assume the best here
• Special rule for this C function call to not affect the C call limit will not be accepted
  • Rules are for everyone

RFC might get a chance if it drops these extra requirements.
The design could have just described an actual implementation which would work in the bounds of the current VM (and can be provided by any embedder today):

int noyield(lua_State* L)
{
    luaL_checkany(L, 1);
    return lua_call(L, lua_gettop(L) - 1, LUA_MULTRET);
}

RFC can also use some trimming.
We don't need descriptions of table library hacks, we never suggested those to use for this purpose, it could just be a few lines mentioning that such hacks exist, but we don't need their analysis as an actual solution.
Similarly, I'm not sure why the benchmarks for them are in here.
Sometimes, less is better and it's ok to link supplemental material in the PR description instead of the body.

For transparency, I would also like for you to confirm that AI was not used to generate this RFC.

[Aseminaunz]

@vegorov-rbx

VM changes which will block this PR from being accepted:

I've been taking a look around in the VM since I made the PR, and have found that the proposed design is rather complex to implement. Specifically, the special C stack detail adds considerable complexity to Luau. If users need to avoid the C stack limit, they can just do

if coroutine.isyieldable() then
    return coroutine.noyield(fn, ...)
else
    return fn(...)
end

New state data will be required to track current context description

lua_State would need to track a bool indicating if the current thread is yieldable or not by user code - I believe the benefits of the feature outweigh this cost. After looking at the VM ldo's lua_break, it'd also be useful if coroutine.noyield did not stop debug breakpoints, as a simple proxy to lua_call currently would.

Custom error message will require changes to the VM we do not plan to incorporate

The idea is to have calls that are only stopped by coroutine.noyield throw a different string when calling lua_yield, not a custom error object. I haven't seen any details in the VM that'd complicate adding another possible error string, and this shouldn't be a problem for users either.

About the size of the RFC, it's pretty big, yeah. I was a bit bored and figured I would include little experiments in collapsible sections.

AI wasn't used, I'm not a native English speaker, so I might just sound a little robotic trying to speak formally. Some of the wording is a bit repetitive, I wasn't satisfied with how shorter more concise iterations of the text got the point across.

I'll clean up details and review the proposal.

[vrn-sn]

Another consideration: yielding from Luau's C API is a "fire-and-forget" operation, so any resumption logic has to be set up before the yield is signaled. For example, a minimal implementation of task.wait might look like this:

int taskwait(lua_State* L)
{
  GlobalThreadManager.scheduleThreadResumption(L, lua_tonumber(L, 1));
  return lua_yield(L);
}

Under this proposal, we reject the yield by raising an error, but there's no way to prevent the scheduled thread resumption since it was set up before the yield was signaled. Yielding in the Luau C API isn't a handshake; there's no way for the caller to "veto" it after the fact, and supporting that would require other changes to the API.

When GlobalThreadManager attempts to resume the thread later, we'd get an unexpected additional error like cannot resume non-suspended coroutine after the proposed immediate attempt to yield across noyield boundary error.

[jackdotink]

@vrn-sn This is true, but unimportant because this is already how these mechanisms work today when attempting to yield in non-yieldable contexts.

[alexmccord]

If I recall, we really want the VM to support yielding everywhere, don't we?

[jackdotink]

I don't know if that's a goal or not, but I do know that it cannot happen because of performance constraints.