[dyn_modules] update readme

Include description of the semantics, how the technology works,
status, and answers to common questions.

Change-Id: Ib0804bb919b3336808c39b9c2e6a7140ba32b500
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/526901
Reviewed-by: Michael Goderbauer <goderbauer@google.com>
Reviewed-by: Slava Egorov <vegorov@google.com>
Commit-Queue: Sigmund Cherem <sigmund@google.com>
diff --git a/pkg/dynamic_modules/README.md b/pkg/dynamic_modules/README.md
index c18c377..8fb87a0 100644
--- a/pkg/dynamic_modules/README.md
+++ b/pkg/dynamic_modules/README.md
@@ -1,9 +1,560 @@
-This package contains experimental dynamic modules API.
+# Dart Dynamic Modules
 
-## Status: experimental
+## Quick Disclaimers
 
-**NOTE**: This package is currently experimental and not published or
-included in the SDK.
+*Dart Dynamic Modules* is a work-in-progress and experimental. Like
+any experimental Dart feature, that means it might change in
+breaking ways and we are not yet committed to adopting it. Please be
+aware of the risks if you consider using it beyond local
+experiments.
 
-Do not take dependency on this package unless you are prepared for
-breaking changes and possibly removal of this code at any point in time.
+At this time, we are prioritizing work for Dynamic Modules and Dart
+Bytecodes that helps us improve specific developer journeys. We have
+focused on collaborative sharing of prototypes, and mobile
+development without JITing. We are not currently prioritizing work
+related to other applications of Dynamic Modules in production
+applications, like server-driven UI use-cases.
+
+Regardless of the use case, this feature needs to be treated with
+responsibility and care. Further
+[below](##technical-details--considerations--faq) we share many
+considerations, from security to policy, that you should be aware of
+before proceeding.
+
+## What are Dart Dynamic Modules?
+
+Dart Dynamic Modules allows delivering code to an existing
+application and loading it dynamically much like a dynamic import in
+JavaScript or a dynamically linked library in C.
+
+We'll dive into two parts in a bit more detail:
+
+* **Semantics**: Until now Dart primarily supported statically
+  compiled applications. This feature provides semantic reasoning
+  for how to expand applications dynamically, mechanisms to declare
+  these expansions, and validation needed to ensure these
+  expansions don't break memory safety.
+
+* **Technology**: How we implemented it for AOT applications by
+  using Dart bytecodes to deliver dynamic modules.
+
+Even though we don't have implementations for Dart web backends
+today, we have explored prototypes enough to convince ourselves that
+the semantics can be achieved using different technologies that fit
+naturally with those backends as well.
+
+## Semantics
+
+Dart Dynamic Modules makes it possible to dynamically load a set of
+Dart libraries in a running Dart application. It defines how those
+libraries can interact with the rest of the application, and
+provides utilities to facilitate their construction and correct
+usage.
+
+### Basic architecture: Leaving a closed-world behind
+
+Historically, we have mostly treated Dart applications as a
+monolith. Every time we build a Dart binary, we typically build the
+entire application as a unit. Yes, there are some exceptions in
+development scenarios where we support modular compilation, like DDC
+on the web, but this whole-program approach is generally true across
+our backends (dart2wasm, dart2js, AOT). Even features like deferred
+loading that allow developers to split their binary into pieces to
+lower latency during an initial load, operate with the whole program
+first and only later they split the code into chunks.
+
+Dart Dynamic Modules moves us away from this monolith concept. Now,
+an application can be built and more modules can be defined,
+compiled, and loaded after the fact. This represents architectural
+change: the application now acts as a ***host*** that is extensible
+through dynamic modules over time. The main semantic restriction:
+all extensions are additive. Dynamic modules cannot replace an
+existing declaration in the application, not even if that
+declaration was delivered through a dynamic module earlier.
+
+To extend an application, developers are required to define an API
+surface area that establishes how dynamic modules will interact with
+the rest of the host application. We call the declaration of this
+surface area a ***Dynamic Interface***. With a surface area defined,
+a host application is built as a single binary like before, but with
+the caveat that the API surface area is now exposed. More libraries
+can be compiled later on and loaded as dynamic modules, as long as
+they are compatible with the API surface area defined by the host.
+
+Quick recap on our terminology:
+
+* **Host application**: the application that will load the dynamic
+  module.
+* **Dynamic interface**: a declaration of the API surface area
+  exposed by a host application and made accessible to a dynamic
+  module.
+* **Dynamic module**: a set of Dart libraries and an entrypoint
+  method, that are loaded dynamically.
+
+### Typical steps to use dynamic modules
+
+Let's dive into more details on how this architecture plays out. If
+you are interested in how to use them directly and experiment with
+them, skip ahead and look at our [example](#example).
+Here we describe the conceptual steps that are needed when adopting
+an application architecture that uses dynamic modules:
+
+* Step 1: First, you will need to have a special build of Dart that
+  supports dynamic modules. We don't ship this as part of the Dart
+  or Flutter SDKs, so at this time this needs to be built from
+  sources (we show how in the [example below](#example)).
+
+* Step 2: You need to determine an API surface area. To do so, you
+  usually start by imagining what logic will be loaded dynamically
+  and how it will be integrated into your application. This helps
+  you figure out the right contract between the application and the
+  dynamic modules, when and how the application will load them, and
+  identify which APIs need to be exposed. By default, nothing is
+  exposed, not even APIs in `dart:core`, unless you opt into them.
+
+* Step 3: Once you decide which APIs you want to expose from the
+  host, you write a special `dynamic_interface.yaml` file that
+  defines the dynamic interface (see
+  [format specification][dynamic_interface.md]).
+  This file is later used as an input to all compilation steps.
+
+* Step 4: With a dynamic interface in hand, you can build the host
+  application. There are three differences from how normal Dart
+  applications are built:
+
+  * First, the compiler and runtime are slightly different in order
+    to include support for executing the dynamic modules. As we'll
+    see later below, for AOT we decided to adopt Dart bytecodes and
+    embed an interpreter in the runtime as the technology to execute
+    dynamic modules in AOT.
+
+  * Second, the application code is compiled differently by
+    informing the compiler about the dynamic interface.
+    Traditionally, Dart's AOT compiler performs closed-world
+    optimizations assuming all code is known to the compiler,
+    dynamic modules break that assumption so the compiler needs
+    guidance to produce correct and sound applications. Usually this
+    means less code is tree-shaken and fewer calls can be
+    specialized because we don't know how exposed APIs will be used.
+
+  * Third, the application code needs to embed some ***provenance***
+    information that will be used later to validate the
+    compatibility of what we load dynamically. This is important to
+    guarantee correctness and memory safety and is a consequence of
+    the technological choices we made for implementing dynamic
+    modules. (Note: provenance is work in progress and not available
+    yet in the current implementation of the package).
+
+* Step 5: At this stage the application is ready to be delivered and
+  run. The logic of what comes in a dynamic module can be defined
+  later, even after the application is running.
+
+* Step 6: Create and build a dynamic module. This involves writing a
+  new set of Dart libraries with a special entrypoint. There is an
+  important constraint, dynamic modules can only define new code. If
+  they define libraries that already exist in the host application
+  or in previous dynamic modules, the module is going to be rejected
+  during the load. To deliver a dynamic module, we first compile it
+  to a format compatible with the runtime where the application is
+  running. In the case of AOT, we use a bytecode compiler to produce
+  the dynamic module. As hinted above, bytecode compilation not only
+  produces the bytecode that we will run, but also provenance
+  attestations needed in the validation process.
+
+* Step 7: Deliver the dynamic module to the app. Since dynamic
+  modules are delivered dynamically, some effort is needed to set up
+  the right way to deliver the dynamic module properly and securely.
+  This often involves some form of transport-level security (TLS).
+
+* Step 8: Validate and run the dynamic module. Before running the
+  dynamic module — which mainly consists of invoking the designated
+  entry point of the dynamic module and returning the result to the
+  host application — we need assurances that the dynamic module
+  won’t break the integrity of the application. Without validation,
+  loading a dynamic module could break the application in unexpected
+  ways (e.g. crashes, heap corruption, and more). Validation
+  typically confirms compatibility of the dynamic module format and
+  APIs in use. Depending on the technology, the verification may be
+  done differently. Our implementation based on bytecodes relies on
+  provenance checks to provide validation. Dynamic modules are
+  allowed to load only if the same application code,
+  dynamic-interface, and version of the SDK tools were used to build
+  both the host app and the dynamic module.
+
+That's the basic principle.
+
+### Additional semantic details
+
+This isn't a complete or formal spec, but here are some details to
+expect about dynamic modules:
+
+* It's a static error to tear off or invoke a member that is not
+  exposed as "callable" in the dynamic interface. This rejection
+  typically happens in Step 6 of our conceptual steps above.
+* It's a static error to extend a class that was not exposed as
+  "extendable" in the dynamic interface.
+* It's a static error to override a member that was not exposed as
+  "can-be-overridden" in the dynamic interface.
+* It's a static error to use a class as a type that was not exposed
+  as "callable" or "can-be-used-as-type" in the dynamic interface.
+* All direct and interface calls are validated statically.
+* Dynamic calls are generally discouraged because they can't be
+  verified statically. They are, however, supported with special
+  rules:
+  * By default, compilers will reject the use of a dynamic call
+    unless the selector name used in the dynamic call was previously
+    allowed.
+
+  * A selector name can be allowed if elements with the same name
+    are exposed as "dynamically-callable" in the dynamic interface.
+    This makes it possible to use dynamic calls that originate in
+    dynamic modules but target a member in the host application.
+
+  * A selector name can also be allowed through an allow-list flag
+    of the dynamic module compiler. This makes it possible to use
+    dynamic calls that originate from dynamic modules and target
+    members declared also in dynamic modules.
+
+  * Unlike all other parts of the API surface area, runtime checks
+    are performed to enforce that the dynamic interface was
+    respected. Runtime errors may be reported on dynamic calls that
+    originate in the dynamic module if the target of the call was a
+    member of the host application that was not exposed as
+    "dynamically-callable".
+* It's a runtime error to load a dynamic module with an incompatible
+  format.
+* It's a runtime error to load a dynamic module that attempts to
+  define a library with an import-URI that already exists in the
+  program, regardless of whether that import-URI was used for a
+  library defined by the host application or a prior dynamic module.
+
+## Technology: Bytecode-based Implementation in AOT
+
+For each backend, there are many possible ways to bring dynamic
+modules to life. DDC, for instance, is a modular compiler, so a
+dynamic module can directly be implemented as a new module with very
+little change to the compiler architecture. For AOT, we decided to
+adopt an architecture that uses an interpreter.
+
+The Dart bytecode [interpreter][interpreter.cc] exists in the Dart
+VM codebase. It requires a tight integration with the runtime, which
+makes it difficult to deliver separately from the Dart runtime
+itself. At the same time, it has an implicit code-size cost on the
+runtime that we don't want to include by default on all Dart
+applications today. Because of these reasons, we don't ship it by
+default as part of our SDK today. Instead, to use the dynamic
+modules experiment, we require passing a build-time flag when
+building Dart.
+
+We also have a [dart2bytecode compiler][dart2bytecode], which we
+use to compile dynamic modules to bytecode. In the future, we expect
+the format to deliver dynamic modules will be expanded beyond
+bytecode to allow embedding resources (e.g. icons, images,
+provenance).
+
+Finally, we are in the process of putting together tools to produce
+provenance information. This includes information about how an app
+and dynamic module were created, including versioning details about
+the inputs and tools used in the process. This may be embedded from
+the bytecode compiler or provided separately. We'll update this
+README as we make progress on the implementation.
+
+The libraries in `package:dynamic_modules` currently provide an API
+to load bytecode directly. This will be updated in the near future
+once we update the format and embed provenance information. Stay
+tuned for changes.
+
+## Example
+
+The [`example/`](example/) directory contains a simple example that
+illustrates the steps described above. The example is a simple
+calculator command line interface that lets users extend it with new
+operations beyond those built-in.
+
+The example [README.md](example/README.md) contains details on how
+to navigate it. Here, we'd like to highlight how the example
+connects with some of the conceptual steps we described above.
+
+The calculator consists of 3 parts:
+
+* **An API surface area** exposes the interface used by dynamic
+  modules to define new operations, as well as the API to register
+  them. This corresponds to steps 2 and 3 above. See
+  [`dynamic_interface.yaml`](example/dynamic_interface.yaml)
+  and [`common/common.dart`](example/common/common.dart).
+
+* **The host app** defines the main REPL (read–eval–print loop) and
+  provides one built-in operation (math addition). This corresponds
+  to step 4 above. See [`host/main.dart`](example/host/main.dart).
+
+* **Dynamic modules** define new operations for the calculator
+  following the API surface area defined by the host application.
+  This corresponds to step 6 above. See the
+  [`modules/`](example/modules/) subfolder.
+
+Finally everything comes together in the [`run.sh`](example/run.sh)
+script. This script shows:
+
+* How to build Dart SDK artifacts with support for Dart Dynamic
+  Modules (Step 1 above) by providing the `--dart-dynamic-modules`
+  build-time flag.
+
+* How to use the dynamic interface to build artifacts of steps 4 and
+  6.
+
+This example doesn't show how dynamic modules are delivered over the
+network (step 7), instead bytecode files are copied over to a data
+directory as an over simplistic way to represent the delivery
+process.
+
+## Progress/Status
+
+Dynamic modules are designed so they can eventually be supported in
+any Dart backend, however we have primarily focused on the AOT
+implementation. All other backends are either unsupported or
+incomplete.
+
+What works?
+
+* AOT compilation factors dynamic-interface to prevent tree-shaking
+  and invalid optimizations and to perform runtime validation of
+  dynamic calls.
+* Modular validation of API usage against dynamic-interface.
+* Integration of bytecode interpreter in the AOT runtime.
+* Integration of bytecode interpreter in the JIT runtime.
+* Required runtime checks in the AOT runtime needed to validate
+  dynamic calls emanating from dynamic modules. For context, most
+  validation of dynamic modules is done statically and modularly
+  while compiling dynamic modules. However, the use of dynamic calls
+  in dynamic modules require an additional whole-program static
+  check and runtime checks.
+
+What doesn't work yet?
+
+* Bytecode provenance validation. There is work in progress, but the
+  APIs have not been migrated over. This is a known upcoming
+  breaking change to our APIs.
+* Whole-program validation of dynamic modules. As mentioned earlier,
+  dynamic calls also need a whole-program static check. This check
+  is not implemented yet.
+* Required runtime checks in the JIT runtime to validate dynamic
+  calls. This is missing mainly because the JIT runtime doesn't load
+  the dynamic-interface when running a host application from
+  sources.
+* Simple developer experience when authoring dynamic interfaces.
+  Currently the `dynamic_interface.yaml` format has quirks
+  especially around private members, which sometimes need to be
+  exposed due to low-level implementation details of the language.
+* Invoking FFI APIs from a dynamic module.
+
+What is currently not planned?
+
+* We are primarily focused on the AOT pipeline. We may improve
+  support in JIT, but do not plan to focus on the web backends (DDC,
+  dart2js, or dart2wasm) at this time. While this package includes
+  tests that use an old DDC implementation based on DDC's JS module
+  systems, that implementation was primarily created to validate the
+  dynamic-modules concepts and ensure the semantics were flexible
+  enough to support Dart backends other than AOT.
+
+## Technical details, Considerations, FAQ
+
+**Q: How are dynamic modules executed in AOT?**
+
+The current implementation in AOT focuses entirely on using bytecode
+to deliver dynamic modules and executing it using a bytecode
+interpreter. That said, in general Dart Dynamic Modules as a concept
+is agnostic of how code gets executed. We only define how code
+extends an existing application and how it interacts with it. In
+fact, prototypes we have built with DDC and dart2wasm leveraged
+their own JS or wasm module systems to load new code.
+
+**Q: Are dynamic modules fully sandboxed?**
+
+No. Dynamic modules run in an interpreter, but you are loading code
+into the same context of the application. Both the original host
+application code and the dynamic modules run together as a single
+program with a shared heap. Even though dynamic modules have
+restricted APIs to refer to, there are no restrictions on what those
+APIs can access indirectly.
+
+You can create some lightweight isolation by loading dynamic modules
+in a new Dart isolate. This separates the heap from the rest of the
+host application, but won’t provide a full sandbox as it will still
+be running in the same OS process as the app.
+
+We think it is an interesting future direction to explore creating
+trim-down versions of the Dart runtime that are fully sandboxed
+(e.g. absolute no access to IO or FFI), though.
+
+**Q: Is it OK to load untrusted/external/third-party code as a
+dynamic module?**
+
+No. As mentioned above, there is no sandbox. Anything you load with
+a dynamic module requires a lot of trust. Just like it's not OK to
+load untrusted third-party code without a sandbox in the same
+context as your web application, it's not OK to do so in a Dart
+application. Any code that is not developed by your team (external
+contributions, unreviewed LLM generated code, etc) should be treated
+as untrusted.
+
+**Q: Why will the API change to require provenance information?**
+
+We will require provenance information to validate compatibility
+before a dynamic module is loaded and run in a host application.
+Unlike other bytecode formats, like Java bytecodes, the Dart
+bytecode format has no stable ABI and no mechanism to verify that
+APIs used from the bytecode are compatible. As such, we need
+additional mechanisms to prove compatibility and verify that a
+dynamic module will behave as expected when loaded into a running
+host application. Since compilation to bytecode already performs
+most of the verification steps we need, we are designing the system
+to plumb provenance information as a way to give the host
+application the proof it needs to confirm everything is in order
+before proceeding.
+
+**Q: Why are Dart bytecodes and Dart dynamic modules being
+developed?**
+
+We are currently exploring improvements to the development story of
+Dart and Flutter applications. Historically we relied heavily on JIT
+compilation, but we were exploring using modular AOT compilation in
+combination with Dart bytecodes to provide alternative development
+workflows that can be supported in environments where JIT
+compilation is not feasible. We discovered that the same technology
+can be used for loading code dynamically, which led us to explore
+new development use cases of the technology. One such use-case is to
+share prototypes between team members at a low cost.
+
+**Q: Can I use dynamic modules today in my Dart/Flutter app?**
+
+We are in an early experimental stage of this feature, so we don't
+recommend taking a dependency on it for projects used in production
+today. You'll note that the feature is also not available by default
+in Dart and Flutter SDKs, in part because just enabling it
+introduces code-size bloat and performance effects that are
+unnecessary for existing applications today.
+
+We are prioritizing dynamic modules for development use cases, where
+enabling dynamic modules at build time is an option and the
+trade-offs are less of an issue.
+
+**Q: Why are you only prioritizing development use cases?**
+
+There are many considerations that need to be factored when
+supporting a feature for production use cases. In the case of Dart
+Dynamic Modules, the consideration list is long and requires a lot
+of diligence, care, and effort, both from the SDK and from actual
+developers adopting it. We decided to focus only on development use
+cases that allow us to reduce the scope of these considerations.
+
+Here are some of the top-of-mind factors for this technology:
+
+* This feature can increase application size unnecessarily. The
+  interpreter has a fixed cost, but there are additional effects on
+  AOT binaries from disabling tree-shaking and optimizations related
+  to the dynamic interface. If used naively, that extra size bloat
+  can be substantial.
+
+* If used improperly, this feature can introduce security risks.
+  Dynamic modules are loaded in a trusted environment with no extra
+  sandbox, so care needs to be in place to ensure bytecode has the
+  right provenance and is delivered safely. Our work on requiring
+  provenance is an effort to reduce this risk, but it doesn't
+  eliminate all attack vectors.
+
+* If used improperly, this feature can break applications or cause
+  them to behave unexpectedly. For example, accidentally mixing
+  bytecode compiled against a different version of the app can cause
+  issues. Our work on provenance also helps reduce this risk by
+  validating compatibility, but it doesn't reduce the complexity
+  required to manage versioning in the first place.
+
+* Proper version management involves non-trivial efforts, including
+  creating server components and properly managing creation and
+  storage of bytecode. We think a production-ready solution should
+  have clear guidance and designs for how to address this challenge.
+
+By focusing on the development use cases we found that some of these
+considerations have a lower impact or lower complexity. For
+instance, code-size is a lesser concern and we can focus on
+single-version scenarios that ignore the more general versioning
+questions. We still believe these considerations are important and
+relevant, so we have used the development scenarios to help us make
+progress on them (e.g. provenance support).
+
+**Q: Is Dart Dynamic Modules a "code-push" implementation?**
+
+No. Dart Dynamic Modules is a language runtime technology, similar
+to that available in many other languages. It resembles C dynamic
+linking, JavaScript dynamic imports, wasm module instantiations.
+
+Code-push approaches are full-stack solutions that use multiple
+technologies and services underneath to enable developers to send
+over-the-air updates that patch existing applications. A general
+code-push solution needs hosting services to track releases and
+patches over time, mechanisms to make the application updatable
+in-place, to determine what has changed between patches, to encode
+updates in a deliverable format, to deliver and apply such updates,
+and more.
+
+We believe Dart Dynamic Modules is not an ideal technology for a
+general-purpose codepush solution for a few reasons. First, the
+additive semantics: dynamic modules can only introduce new
+libraries, but can't update any existing declaration in an
+application. Second, the API surface area: dynamic modules can only
+invoke code that was previously exposed. In contrast, code-push
+solutions want to update existing declarations and want patches to
+be able to invoke anything already in the application. For a dynamic
+module to be able to invoke anything in an application, we would
+need to expose the entire app in the dynamic interface. That would
+practically disable tree-shaking and all whole-program
+optimizations, dramatically affecting the original application
+performance and code-size to the point that we doubt it would be
+suitable for use in production environments.
+
+**Q: Are there security concerns behind dynamic modules?**
+
+Yes. As hinted above in the earlier questions about how it works,
+some top security considerations are that:
+
+* A dynamic module loads in the same context as the host app,
+  without an additional sandbox. Compared to the web, this is
+  equivalent to appending `<script>` tags without using an
+  iframe. As such, similar precautions are needed to use dynamic
+  modules safely. This includes ensuring the code is trusted,
+  preventing injection scenarios that would tamper the integrity of
+  the bytecode. For example, you should never use dynamic modules to
+  load code provided by others or provided by LLMs. Also, even if
+  the source of the code is trusted, the delivery needs to be
+  secured with proper TLS.
+
+* Our bytecode doesn't have a stable ABI, so compatibility is only
+  guaranteed if the bytecode loaded matches the format supported by
+  the host app. A skew in the ABI can cause memory corruption or
+  other semantic errors and lead the application to misbehave.
+
+* Similarly, incompatible APIs can also lead to the same result.
+
+Our work to provide provenance validation is designed to help
+validate ABI and API compatibility before a load is allowed. Unlike
+Java bytecodes, Dart bytecodes don't offer verification that the
+APIs in the bytecode match the APIs in the host app, so we resort to
+provenance as an alternative technique. Just like the web,
+provenance alone does not guarantee the safety of the code being
+loaded. Special diligence is required by developers to ensure that
+only code that is trusted and delivered securely is ever loaded in
+the context of the application.
+
+**Q: Are there store policy considerations to consider when using
+dynamic modules?**
+
+Yes, there are policy considerations, regardless of how code is run.
+For example, you might want to validate that any store you are
+publishing to allows running interpreted code in a custom
+interpreter. And stores might have policies around how much an app
+is allowed to change after it has been installed by the user.
+
+[interpreter.cc]: https://github.com/dart-lang/sdk/blob/898a1e4bbfbc472dc0a9505dc7d2e4c21d6f856e/runtime/vm/interpreter.cc
+[dart2bytecode]: https://github.com/dart-lang/sdk/blob/898a1e4bbfbc472dc0a9505dc7d2e4c21d6f856e/pkg/dart2bytecode/
+[dynamic_interface.md]: https://github.com/dart-lang/sdk/blob/898a1e4bbfbc472dc0a9505dc7d2e4c21d6f856e/pkg/vm/dynamic_interface.md