runtime/docs/async.md

# Suspendable Functions (`async`, `async*` and `sync*`)

This document describes the implementation of _suspendable_ functions (functions with `async`,
`async*` or `sync*` modifier) in Dart VM. The execution of such functions can be suspended in
the middle at `await`/`yield`/`yield*` and resumed afterwards.

When suspending a function, its local execution state (local variables and temporaries) is saved
and the control is returned to the caller of the suspended function.
When resuming a function, its local execution state is restored and execution continues within
the suspendable function from the point where it was suspended.

In order to minimize code size, the implementation is built using a variety of _stubs_ - reusable
snippets of machine code generated by the VM/AOT.
The high-level Dart logic used to implement suspendable functions (such as managing
Futures/Streams/Iterators) is factored into helper Dart methods in core library.

The rest of the document is organized as follows: first, general mechanisms for implementation of
suspendable functions are described.
After that, `async`, `async*` and `sync*` implementations are outlined using the general
mechanisms introduced before.

# Building blocks common to all suspendable functions

## SuspendState objects

SuspendState objects are allocated on the heap and encapsulate the saved state of a suspended
function. When suspending a function, its local frame (including local variables, spill slots
and expression stack) is copied from the stack to a SuspendState object on the heap.
When resuming a function, the frame is recreated and copied back from the SuspendState object
into the stack.

SuspendState objects have variable size and keep frame in the "payload" following a few fixed
fields.

In addition to a stack frame, SuspendState records a PC in the code of the suspended function
where execution was suspended and can be resumed.
The PC is also used by GC to find a stack map and scan through the pointers in the copied frame.

SuspendState object also holds data and callbacks specific to a particular kind of suspendable
function.

SuspendState object is allocated during the first suspension and can be reused for the subsequent
suspensions of the same function.

For the declaration of SuspendState see [object.h](https://github.com/dart-lang/sdk/blob/main/runtime/vm/object.h#:~:text=class%20SuspendState),
UntaggedSuspendState is declared in [raw_object.h](https://github.com/dart-lang/sdk/blob/main/runtime/vm/raw_object.h#:~:text=class%20UntaggedSuspendState).

There is also a corresponding Dart class `_SuspendState`, declared in [async_patch.dart](https://github.com/dart-lang/sdk/blob/main/sdk/lib/_internal/vm/lib/async_patch.dart#:~:text=class%20_SuspendState).
It contains Dart methods which are used to customize implementation for a particular kind of
suspendable function.

## Frame of a suspendable function

Suspendable functions are never inlined into other functions, so their local state is not mixed
with the state of their callers (but other functions may be inlined into them).

In order to have a single contiguous region of memory to copy during suspend/resume, parameters of
suspendable functions are always copied into the local frame in the function prologue (see uses of
`Function::MakesCopyOfParameters()` predicate).

In order to keep and reuse SuspendState object, each suspendable function has an artificial local
variable `:suspend_state` (see uses of `ParsedFunction::suspend_state_var()`), which is always
allocated at the fixed offset in frame. It occupies the first local variable slot
(`SuspendState::kSuspendStateVarIndex`) in case of unoptimized code or the first spill slot
in case of optimized code (see `FlowGraphAllocator::AllocateSpillSlotForSuspendState`).
The fixed location helps to find this variable in various stubs and runtime.

## Prologue and InitSuspendableFunction stub

At the very beginning of a suspendable function `null` is stored into `:suspend_state` variable.
This guarantees that `:suspend_state` variable can be accessed any time by GC and exception
handling.

After checking bounds of type arguments and types of arguments, suspendable functions call
InitSuspendableFunction stub.

InitSuspendableFunction stub does the following:

- It calls a static generic Dart method specific to a particular kind of suspendable function.
  The argument of the stub is passed as type arguments to that method.
  Dart method performs initialization specific to a particular kind of suspendable function
  (for example, it creates `_Future<T>()` for async functions).
  It returns the instance which is used as a function-specific data.

- Stub puts the function-specific data to `:suspend_state` variable, where it can be found by
  Suspend or Return stubs later.

## Suspend stub

Suspend stub is called from a suspendable function when its execution should be suspended.

Suspend stub does the following:

- It inspects `:suspend_state` variable and checks if it contains an instance of SuspendState.
  If it doesn't, then stub allocates a new instance with a payload sufficient to hold a frame of
  the suspendable function. The newly allocated SuspendState is stored into `:suspend_state`
  variable, and previous value of `:suspend_state` (coming from InitSuspendableFunction stub) is
  saved to `SuspendState.function_data`.

- In JIT mode, size of the frame may vary over time - expression stack depth varies during
  execution of unoptimized code and frame size may change during deoptimization and OSR.
  In AOT mode size of the stack frame stays the same.
  So, if stub finds an existing SuspendState object in JIT mode, it also checks if its frame
  payload has a sufficient size to hold a frame of the suspendable function. If it is not
  large enough, suspend stub calls `AllocateSuspendState` runtime entry to allocate a larger
  SuspendState object. The same runtime entry is called for slow path when allocating
  SuspendState for the first time.

- The return address from Suspend stub to the suspendable function is saved to `SuspendState.pc`.
  It will be used to resume execution later.

- The contents of the stack frame of the suspendable function between FP and SP is copied into
  SuspendState.

- Write barrier: if SuspendState object resides in the old generation, then
  EnsureRememberedAndMarkingDeferred runtime entry is called.

- If implementation of particular kind of suspendable function uses a customized Dart method
  for the suspension, then that method is called.
  Suspend stub supports passing one argument to the customization method.
  The result of the method is returned back to the caller of the suspendable function - it's
  the result of the suspendable function.
  If such method is not used, then Suspend stub returns its argument (so suspendable function
  could customize its return value).

- On architectures other than x64/ia32, the frame of the suspendable function is removed and
  stub returns directly to the caller of the suspendable function.
  On x64/ia32, in order to maintain call/return balance and avoid performance penalty,
  Suspend stub returns to the suspendable function which immediately returns to its caller.

For more details see `StubCodeCompiler::GenerateSuspendStub` in [stub_code_compiler.cc](https://github.com/dart-lang/sdk/blob/main/runtime/vm/compiler/stub_code_compiler.cc#:~:text=StubCodeCompiler::GenerateSuspendStub).

## Resume stub

Resume stub is tail-called from `_SuspendState._resume` recognized method (which is called
from Dart helpers). It is used to resume execution of the previously suspended function.

Resume stub does the following:

- Allocates Dart frame on the stack, using `SuspendState.frame_size` to calculate its size.

- Copies frame contents from SuspendState to the stack.

- In JIT mode restores pool pointer (PP).

- Checks for the following cases and calls ResumeFrame runtime entry if any of this is true:
  + If resuming with an exception.
  + In JIT mode, if Code of the suspendable function is disabled (deoptimized).
  + In JIT mode, if there is a resumption breakpoint set by debugger.

- Otherwise, jumps to `SuspendState.pc` to resume execution of the suspended function.
  On x64/ia32 the continuation PC is adjusted by adding `SuspendStubABI::kResumePcDistance`
  to skip over the epilogue which immediately follows the Suspend stub call to maintain
  call/return balance.

ResumeFrame runtime entry is called as if it was called from suspended function at continuation PC.
It handles all corner cases by throwing an exception, lazy deoptimizing or calling into
the debugger.

For more details see `StubCodeCompiler::GenerateResumeStub` in [stub_code_compiler.cc](https://github.com/dart-lang/sdk/blob/main/runtime/vm/compiler/stub_code_compiler.cc#:~:text=StubCodeCompiler::GenerateResumeStub)
and `ResumeFrame` in [runtime_entry.cc](https://github.com/dart-lang/sdk/blob/main/runtime/vm/runtime_entry.cc#:~:text=ResumeFrame).

## Return stub

Suspendable functions can use Return stub if they need to do something when execution of
a function ends (for example, complete a Future or close a Stream). In such a case,
suspendable function jumps to the Return stub instead of returning.

Return stub does the following:

- Removes the frame of the suspendable function (as if function epilogue was executed).

- Calls a Dart method specific to a particular kind of suspendable function.
  The customization method takes a value of `:suspend_state` variable and a return value
  passed from the body of the suspendable function to the stub.

- The value returned from the customization method is used as the result of
  the suspendable function.

For more details see `StubCodeCompiler::GenerateReturnStub` in [stub_code_compiler.cc](https://github.com/dart-lang/sdk/blob/main/runtime/vm/compiler/stub_code_compiler.cc#:~:text=StubCodeCompiler::GenerateReturnStub).

## Exception handling and AsyncExceptionHandler stub

Certain kinds of suspendable functions (async and async*) may need to catch all thrown exceptions
which are not caught within the function body, and perform certain actions (such as completing
the Future with an error).

This is implemented by setting `has_async_handler` bit on `ExceptionHandlers` object.
When looking for an exception handler, runtime checks if this bit is set and uses
AsyncExceptionHandler stub as a handler (see `StackFrame::FindExceptionHandler`).

AsyncExceptionHandler stub does the following:

- It inspects the value of `:suspend_state` variable. If it is `null` (meaning the prologue has not
  finished yet), the exception should not be handled and it is rethrown.
  This makes it possible for argument type checks to throw an exception  synchronously
  instead of completing a Future with an error.

- Otherwise, stub removes the frame of the suspendable function (as if function epilogue was
  executed) and calls `_SuspendState._handleException` Dart method. AsyncExceptionHandler stub
  does not use separate Dart helper methods for async and async* functions as exception handling is
  not performance sensitive and currently uses only one bit in `ExceptionHandlers` to select
  a stub handler for simplicity.

- The value returned from `_SuspendState._handleException` is used as the result of the
  suspendable function.

For more details see `StubCodeCompiler::GenerateAsyncExceptionHandlerStub` in [stub_code_compiler.cc](https://github.com/dart-lang/sdk/blob/main/runtime/vm/compiler/stub_code_compiler.cc#:~:text=StubCodeCompiler::GenerateAsyncExceptionHandlerStub).

## IL instructions

When compiling suspendable functions, the following IL instructions are used:

- `Call1ArgStub` instruction is used to call one-argument stubs such as InitSuspendableFunction.

- `Suspend` instruction is used to call Suspend stub. After calling Suspend stub,
  on x64/ia32 it also generates an epilogue right after the stub, in order to
  return to the caller after suspending without disrupting call/return balance.
  Due to this extra epilogue following the Suspend stub call, the resumption PC is
  not the same as the return address of the Suspend stub. So `Suspend` instruction
  uses 2 distinct deopt ids for the Suspend stub call and resumption PC.

- `Return` instruction jumps to a Return stub instead of returning for certain kinds
  of suspendable functions (async and async*).

# Combining all pieces together

## Async functions

See [async_patch.dart](https://github.com/dart-lang/sdk/blob/main/sdk/lib/_internal/vm/lib/async_patch.dart) for the corresponding Dart source code.

Async functions use the following customized stubs:

### InitAsync stub

InitAsync = InitSuspendableFunction stub which calls `_SuspendState._initAsync`.

`_SuspendState._initAsync` creates a `_Future<T>` instance which is used as the result of
the async function. This `_Future<T>` instance is kept in `:suspend_state` variable until
`_SuspendState` instance is created during the first `await`, and then kept in
`_SuspendState._functionData`. This instance is returned from `_SuspendState._await`,
`_SuspendState._returnAsync`, `_SuspendState._returnAsyncNotFuture` and
`_SuspendState._handleException` methods to serve as the result of the async function.

### Await stub

Await = Suspend stub which calls `_SuspendState._await`. It implements the `await` expression.

`_SuspendState._await` allocates 'then' and 'error' callback closures when called for
the first time. These callback closures resume execution of the async function via Resume stub.
It is possible to create callbacks eagerly in the InitAsync stub, but there is a significant
fraction of async functions which don't have `await` at all, so creating callbacks lazily during
the first `await` makes those functions more efficient.
If an argument of `await` is a Future, then `_SuspendState._await` attaches 'then' and 'error'
callbacks to that Future. Otherwise it schedules a micro-task to continue execution of
the suspended function later.

### AwaitWithTypeCheck stub

AwaitWithTypeCheck is a variant of Await stub which additionally passes type argument `T`
and calls `_SuspendState._awaitWithTypeCheck` in order to test if the value has a
correct `Future<T>` type before awaiting.

This runtime check is needed to maintain soundness in case value is a Future of an
incompatible type, for example:

```
final FutureOr<Object> f = Future<Object?>.value(null);
Object x = await f; // x == f, not null.
```

### ReturnAsync stub

ReturnAsync stub = Return stub which calls `_SuspendState._returnAsync`.
It is used to implement `return` statement (either explicit or implicit when reaching
the end of function).

`_SuspendState._returnAsync` completes `_Future<T>` which is used as the result of
the async function.

### ReturnAsyncNotFuture stub

ReturnAsyncNotFuture stub = Return stub which calls `_SuspendState._returnAsyncNotFuture`.

ReturnAsyncNotFuture is similar to ReturnAsync, but used when compiler can prove that
return value is not a Future. It bypasses the expensive `is Future` test.

### Execution flow in async functions

The following diagram depicts how the control is passed in a typical async function:

```
Caller          Future<T> foo() async           Stubs        Dart _SuspendState methods
  |
  *-------------------> |
                    (prologue) -------------> InitAsync
                                                  |
                                                  *----------> _initAsync
                                                              (creates _Future<T>)
                                                  | <---------
                        | <-----------------------*
                        |
                        |
                      (await) ----------------> AwaitAsync
                                                  |
                                                  *----------> _await
                                                              (setups resumption)
                                                              (returns _Future<T>)
                                                  | <---------
  | <---------------------------------------------*

Awaited Future is completed
  |
  *------------------------------------------> Resume
                                                  |
                    (after await) <---------------*
                        |
                        |
                      (return) ---------------> ReturnAsync/ReturnAsyncNotFuture
                                                  |
                                                  *----------> _returnAsync/_returnAsyncNotFuture
                                                              (completes _Future<T>)
                                                              (returns _Future<T>)
                                                  | <---------
  | <---------------------------------------------*
```

## Async* functions

See [async_patch.dart](https://github.com/dart-lang/sdk/blob/main/sdk/lib/_internal/vm/lib/async_patch.dart)
for the corresponding Dart source code.

Async* functions use the following customized stubs:

### InitAsyncStar stub

InitAsyncStar = InitSuspendableFunction stub which calls `_SuspendState._initAsyncStar`.

`_SuspendState._initAsyncStar` creates `_AsyncStarStreamController<T>` instance which is used
to control the Stream returned from the async* function. `_AsyncStarStreamController<T>` is kept
in `_SuspendState._functionData` (after the first suspension at the beginning of async* function).

## YieldAsyncStar stub and `yield`/`yield*`

YieldAsyncStar = Suspend stub which calls `_SuspendState._yieldAsyncStar`.

This stub is used to suspend async* function at the beginning (until listener is attached to
the Stream returned from async* function), and at `yield` / `yield*` statements.

When `_SuspendState._yieldAsyncStar` is called at the beginning of async* function it creates
a callback closure to resume body of the async* function (via Resume stub), creates and
returns `Stream`.

`yield` / `yield*` statements are implemented in the following way:

```
_AsyncStarStreamController controller = :suspend_state._functionData;
if (controller.add/addStream(<expr>)) {
  return;
}
if (YieldAsyncStar()) {
  return;
}
```

`_AsyncStarStreamController.add`, `_AsyncStarStreamController.addStream` and YieldAsyncStar stub
can return `true` to indicate that Stream doesn't have a listener anymore and execution of
async* function should end.

Note that YieldAsyncStar stub returns a value passed to a Resume stub when resuming async*
function, so the 2nd hasListeners check happens right before the async* function is resumed.

See `StreamingFlowGraphBuilder::BuildYieldStatement` for more details about `yield` / `yield*`.

### Await stub

Async* functions use the same Await stub which is used by async functions.

### ReturnAsyncStar stub

ReturnAsyncStar stub = Return stub which calls `_SuspendState._returnAsyncStar`.

`_SuspendState._returnAsyncStar` closes the Stream.

### Execution flow in async* functions

The following diagram depicts how the control is passed in a typical async* function:

```
Caller          Stream<T> foo() async*          Stubs        Dart helper methods
  |
  *-------------------> |
                    (prologue) -------------> InitAsyncStar
                                                  |
                                                  *----------> _SuspendState._initAsyncStar
                                                              (creates _AsyncStarStreamController<T>)
                                                  | <---------
                        | <-----------------------*
                        * ------------------> YieldAsyncStar
                                                  |
                                                  *----------> _SuspendState._yieldAsyncStar
                                                              (setups resumption)
                                                              (returns _AsyncStarStreamController.stream)
                                                  | <---------
  | <---------------------------------------------*

Stream is listened
  |
  *------------------------------------------> Resume
                                                  |
                  (after prologue) <--------------*
                        |
                        |
                      (yield) --------------------------------> _AsyncStarStreamController.add
                                                              (adds value to Stream)
                                                              (checks if there are listeners)
                        | <-----------------------------------
                        * ------------------> YieldAsyncStar
                                                  |
                                                  *----------> _SuspendState._yieldAsyncStar
                                                  | <---------
  | <---------------------------------------------*

Micro-task to run async* body
  |
  *----------------------------------------------------------> _AsyncStarStreamController.runBody
                                                              (checks if there are listeners)
                                                Resume <-------
                                                  |
                    (after yield) <---------------*
                        |
                        |
                      (return) ---------------> ReturnAsyncStar
                                                  |
                                                  *----------> _SuspendState._returnAsyncStar
                                                              (closes _AsyncStarStreamController)
                                                  | <---------
  | <---------------------------------------------*
```

## Sync* functions

See [async_patch.dart](https://github.com/dart-lang/sdk/blob/main/sdk/lib/_internal/vm/lib/async_patch.dart)
for the corresponding Dart source code.

Sync* functions use the following customized stubs:

### InitSyncStar stub

InitSyncStar = InitSuspendableFunction stub which calls `_SuspendState._initSyncStar`.

`_SuspendState._initSyncStar` creates a `_SyncStarIterable<T>` instance which is returned
from sync* function.

### SuspendSyncStarAtStart stub

SuspendSyncStarAtStart = Suspend stub which calls `_SuspendState._suspendSyncStarAtStart`.

This stub is used to suspend execution of sync* at the beginning. It is called after
InitSyncStar in the sync* function prologue. The body of sync* function doesn't run
until Iterator is not obtained from Iterable (`_SyncStarIterable<T>`) which is returned from
the sync* function.

### CloneSuspendState stub

This stub creates a copy of SuspendState object. It is used to clone state of sync*
function (suspended at the beginning) for each Iterator instance obtained from
Iterable.

See `StubCodeCompiler::GenerateCloneSuspendStateStub`.

### SuspendSyncStarAtYield stub and `yield`/`yield*`

SuspendSyncStarAtYield = Suspend stub which doesn't call helper Dart methods.

SuspendSyncStarAtYield is used to implement `yield` / `yield*` statements in sync* functions.

`yield` / `yield*` statements are implemented in the following way:

```
_SyncStarIterator iterator = :suspend_state._functionData;

iterator._current = <expr>;             // yield <expr>
  OR
iterator._yieldStarIterable = <expr>;   // yield* <expr>

SuspendSyncStarAtYield(true);
```

See `StreamingFlowGraphBuilder::BuildYieldStatement` for more details about `yield` / `yield*`.

The value passed to SuspendSyncStarAtYield is returned back from the invocation of
Resume stub. `true` indicates that iteration can continue.

### Returning from sync* functions.

Sync* function do not use Return stubs. Instead, return statements are rewritten to return `false`
in order to indicate that iteration is finished.

### Execution flow in sync* functions

The following diagram depicts how the control is passed in a typical sync* function:

```
Caller          Iterable<T> foo() sync*          Stubs        Dart helpers
  |
  *-------------------> |
                    (prologue) -------------> InitSyncStar
                                                  |
                                                  *----------> _SuspendState._initSyncStar
                                                              (creates _SyncStarIterable<T>)
                                                  | <---------
                        | <-----------------------*
                        * ------------------> SuspendSyncStarAtStart
                                                  |
                                                  *----------> _SuspendState._suspendSyncStarAtStart
                                                              (remembers _SuspendState at start)
                                                              (returns _SyncStarIterable<T>)
                                                  | <---------
  | <---------------------------------------------*

Iterable.iterator is called
  |
  *----------------------------------------------------------> _SyncStarIterable<T>.iterator
                                                              (creates _SyncStarIterator<T>)
                                                                |
                                      CloneSuspendState <-------*
                                      (makes a copy of _SuspendState at start)
                                                  |
                                                  *-----------> |
  | <------------------------------------------------------- (returns _SyncStarIterator<T>)

Iterator.moveNext is called
  |
  *----------------------------------------------------------> _SyncStarIterator<T>.moveNext
                                                              (iterates over the cached yield* iterator, if any)
                                                              (resumes sync* body to get the next element)
                                                Resume <-------
                                                  |
                  (after prologue) <--------------*
                        |
                        |
                      (yield) ---------------> SuspendSyncStarAtYield(true)
                                                  |
                                                  *---------->
                                                              (the next element is cached in _SyncStarIterator<T>._current)
                                                              (returns true indicating that the next element is available)
  | <----------------------------------------------------------

Iterator.moveNext is called
  |
  *----------------------------------------------------------> _SyncStarIterator<T>.moveNext
                                                              (iterates over the cached yield* iterator, if any)
                                                              (resumes sync* body to get the next element)
                                                Resume <-------
                                                  |
                  (after yield) <-----------------*
                        |
                        |
                  (return false) ----------------------------->
                                                              (returns false indicating that iteration is finished)
  | <----------------------------------------------------------
```