[VM] Fix deadlock between mutator and BG compiler

Here's how the deadlock can happen:

  * The vm-service background compiler asks all threads to go to a
    safepoint before it finalizes the code. Then it finalizes the
    code and notifies code observers. A code observer posts a
    message to the vm-service. Posting this message requires holding the
    monitor of the receiving isolate's message handler (see
    MessageHandler::PostMessage).

  * The vm-service mutator holds it's message handlers monitor and
    enters the isolate (see MessageHandler::HandleMessages). The
    StartIsolateScope schedules the mutator thread, which will in return
    see that BG compiler wants to safepoint, so it waits for BG compiler
    to finish.

At this point the mutator holds the MessageHandler's monitor and waits
for BG compiler to finish safepoint. The BG compiler tries to post a
message to the message handler (and the monitor is locked).

-> This CL ensures the BG compiler only arranges a safepoint before it
   starts creating the instructions object. It fills it in and exits the
   safepoint again.  Only then do we notify code observers.

Furthermore this CL ensures we transition out of VM to native state
before calling the Dart_PostCMessage() function.

Change-Id: I5320440d54ca303a1ccc7556600a42d37cd6e1de
Reviewed-on: https://dart-review.googlesource.com/c/93421
Commit-Queue: Martin Kustermann <kustermann@google.com>
Reviewed-by: Siva Annamalai <asiva@google.com>
15 files changed
tree: fbb66b6104eae898959d67c21dfea6f8faf4fa0a
  1. .github/
  2. build/
  3. client/
  4. docs/
  5. pkg/
  6. runtime/
  7. samples/
  8. samples-dev/
  9. sdk/
  10. tests/
  11. third_party/
  12. tools/
  13. utils/
  14. .clang-format
  15. .gitattributes
  16. .gitconfig
  17. .gitignore
  18. .gn
  19. .mailmap
  20. .packages
  21. .vpython
  22. AUTHORS
  23. BUILD.gn
  24. CHANGELOG.md
  25. codereview.settings
  26. CONTRIBUTING.md
  27. DEPS
  28. LICENSE
  29. PATENTS
  30. PRESUBMIT.py
  31. README.dart-sdk
  32. README.md
  33. WATCHLISTS
README.md

Dart

Dart is an open-source, scalable programming language, with robust libraries and runtimes, for building web, server, and mobile apps.

Using Dart

Visit the dartlang.org to learn more about the language, tools, getting started, and more.

Browse pub.dartlang.org for more packages and libraries contributed by the community and the Dart team.

Building Dart

If you want to build Dart yourself, here is a guide to getting the source, preparing your machine to build the SDK, and building.

There are more documents on our wiki.

Contributing to Dart

The easiest way to contribute to Dart is to file issues.

You can also contribute patches, as described in Contributing.

License & patents

See LICENSE and PATENTS.