| commit | 73d3020e97e0a57a222b713d1a77464babe470da | [log] [tgz] |
|---|---|---|
| author | Liam Appelbe <liama@google.com> | Thu Feb 27 13:05:53 2025 +1300 |
| committer | dart-internal-monorepo <dart-internal-monorepo@dart-ci-internal.iam.gserviceaccount.com> | Wed Feb 26 16:46:22 2025 -0800 |
| tree | 72818b8f2b4fbde805410a6115aa945cf85f02c8 | |
| parent | c3f1efcc530f5fc2b48f0cd606825c2407e25f97 [diff] |
Use the Dart isolate ownership API on the root isolate (#163703) The isolate ownership API was [introduced recently](https://dart-review.googlesource.com/c/sdk/+/407700) to solve [some deadlock bugs](https://github.com/dart-lang/native/issues/1908) in native callbacks. A native callback is a call from native code into a Dart function. Currently all such callbacks must run that Dart function in the isolate that created the callback (called the target isolate). The only native callback primitives at the moment are `NativeCallable.isolateLocal` (blocking, but must be invoked from the same thread as the target isolate, and the target isolate must be currently entered on that thread) and `NativeCallable.listener` (non-blocking, can be invoked from any thread). To build blocking callbacks that can be called from any thread, we can use a `NativeCallable.listener`, and use a synchronization object like a mutex or a condition variable to block until the callback is complete. However, if we try to do this on the thread that is currently entered in the target isolate, we will deadlock: we invoke the listener, a message is sent to the target isolate, and we block waiting for the message to be handled, so we never pass control flow back to the isolate to handle the message, and never stop waiting. To fix this deadlock, Ffigen and Jnigen both have a mechanism that checks if we're on the target isolate's thread first: - If the native caller is already on the same thread as the target isolate, and the target isolate is entered: - Call the Dart function directly using `NativeCallable.isolateLocal` or similar - Otherwise, if the native caller is coming from a different thread: - Call the Dart function asynchronously using `NativeCallable.listener` or similar - Block until the callback finishes However, this neglects the case where we're on the target isolate's thread, but not entered into the isolate. This case happens in Flutter when the callback is invoked from the UI thread (or the platform thread when thread merging is enabled), and the target isolate is the root isolate. When the native callback is invoked, the root isolate is not entered, so we hit the second case: we send a message to the root isolate, and block to wait for a response. Since the root isolate is exclusively run on the UI thread, and we're blocking the UI thread, the message will never be handled, and we deadlock. The isolate ownership API fixes this by allowing the embedder to inform the VM that it will run a particular isolate exclusively on a particular thread, using `Dart_SetCurrentThreadOwnsIsolate`. Other native code can then query that ownership using `Dart_GetCurrentThreadOwnsIsolate`. This lets us add a third case to our conditional: - If the native caller is on the thread that is currently entered in the target isolate: - Call the Dart function directly using `NativeCallable.isolateLocal` or similar - Otherwise, if the native caller is on the thread that owns the target isolate - Enter the target isolate - Call the Dart function directly using `NativeCallable.isolateLocal `or similar - Exit the target isolate - Otherwise, the native caller is coming from an unrelated thread: - Call the Dart function asynchronously using `NativeCallable.listener` or similar - Block until the callback finishes **Note:** We don't need to set the ownership of VM managed threads, because they run in a thread pool exclusively used by the VM, so there's no way for native code to be executed on the thread (except by FFI, in which case we're entered into the isolate anyway). We only need this for Flutter's root isolate because work can be sent to the UI thread/platform thread using OS specific APIs like Android's `Looper.getMainLooper()`. https://dart.googlesource.com/external/github.com/flutter/flutter/+/0f3b092a10f7bae804ec1df1cf2e520bca8275c7
Monorepo is:
With depot_tools installed and on your path, create a directory for your monorepo checkout and run these commands to create a gclient solution in that directory:
mkdir monorepo cd monorepo gclient config --unmanaged https://dart.googlesource.com/monorepo gclient sync -D
This gives you a checkout in the monorepo directory that contains:
monorepo/ DEPS - the DEPS used for this gclient checkout commits.json - the pinned commits for Dart, flutter/engine, and flutter/flutter tools/ - scripts used to create monorepo DEPS engine/src/ - the flutter/buildroot repo flutter/ - the flutter/engine repo out/ - the build directory, where Flutter engine builds are created third_party/ - Flutter dependencies checked out by DEPS dart/ - the Dart SDK checkout. third_party - Dart dependencies, also used by Flutter flutter/ - the flutter/flutter repo
Flutter's instructions for building the engine are at Compiling the engine
They can be followed closely, with a few changes:
goma_ctl ensure_start is sufficient.Example build commands that work on linux:
MONOREPO_PATH=$PWD if [[ ! $PATH =~ (^|:)$MONOREPO_PATH/flutter/bin(:|$) ]]; then PATH=$MONOREPO_PATH/flutter/bin:$PATH fi export GOMA_DIR=$(dirname $(command -v gclient))/.cipd_bin goma_ctl ensure_start pushd engine/src flutter/tools/gn --goma --no-prebuilt-dart-sdk --unoptimized --full-dart-sdk autoninja -C out/host_debug_unopt popd
The Flutter commands used to build and run apps will use the locally built Flutter engine and Dart SDK, instead of the one downloaded by the Flutter tool, if the --local-engine option is provided.
For example, to build and run the Flutter spinning square sample on the web platform,
MONOREPO_PATH=$PWD cd flutter/examples/layers flutter --local-engine=host_debug_unopt \ -d chrome run widgets/spinning_square.dart cd $MONOREPO_PATH
To build for desktop, specify the desktop platform device in flutter run as -d macos or -d linux or -d windows. You may also need to run the command
flutter create --platforms=windows,macos,linux
on existing apps, such as sample apps. New apps created with flutter create already include these support files. Details of desktop support are at Desktop Support for Flutter
Tests in the Flutter source tree can be run with the flutter test command, run in the directory of a package containing tests. For example:
MONOREPO_PATH=$PWD cd flutter/packages/flutter flutter test --local-engine=host_debug_unopt cd $MONOREPO_PATH
Please file an issue or email the dart-engprod team with any problems with or questions about using monorepo.
We will update this documentation to address them.
flutter commands may download the engine and Dart SDK files for the configured channel, even though they will be using the local engine and its SDK.gclient sync needs to be run in an administrator session, because some installed dependencies create symlinks.