[dyn_modules] update readme Include description of the semantics, how the technology works, status, and answers to common questions. Change-Id: Ib0804bb919b3336808c39b9c2e6a7140ba32b500 Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/526901 Reviewed-by: Michael Goderbauer <goderbauer@google.com> Reviewed-by: Slava Egorov <vegorov@google.com> Commit-Queue: Sigmund Cherem <sigmund@google.com>
diff --git a/pkg/dynamic_modules/README.md b/pkg/dynamic_modules/README.md index c18c377..8fb87a0 100644 --- a/pkg/dynamic_modules/README.md +++ b/pkg/dynamic_modules/README.md
@@ -1,9 +1,560 @@ -This package contains experimental dynamic modules API. +# Dart Dynamic Modules -## Status: experimental +## Quick Disclaimers -**NOTE**: This package is currently experimental and not published or -included in the SDK. +*Dart Dynamic Modules* is a work-in-progress and experimental. Like +any experimental Dart feature, that means it might change in +breaking ways and we are not yet committed to adopting it. Please be +aware of the risks if you consider using it beyond local +experiments. -Do not take dependency on this package unless you are prepared for -breaking changes and possibly removal of this code at any point in time. +At this time, we are prioritizing work for Dynamic Modules and Dart +Bytecodes that helps us improve specific developer journeys. We have +focused on collaborative sharing of prototypes, and mobile +development without JITing. We are not currently prioritizing work +related to other applications of Dynamic Modules in production +applications, like server-driven UI use-cases. + +Regardless of the use case, this feature needs to be treated with +responsibility and care. Further +[below](##technical-details--considerations--faq) we share many +considerations, from security to policy, that you should be aware of +before proceeding. + +## What are Dart Dynamic Modules? + +Dart Dynamic Modules allows delivering code to an existing +application and loading it dynamically much like a dynamic import in +JavaScript or a dynamically linked library in C. + +We'll dive into two parts in a bit more detail: + +* **Semantics**: Until now Dart primarily supported statically + compiled applications. This feature provides semantic reasoning + for how to expand applications dynamically, mechanisms to declare + these expansions, and validation needed to ensure these + expansions don't break memory safety. + +* **Technology**: How we implemented it for AOT applications by + using Dart bytecodes to deliver dynamic modules. + +Even though we don't have implementations for Dart web backends +today, we have explored prototypes enough to convince ourselves that +the semantics can be achieved using different technologies that fit +naturally with those backends as well. + +## Semantics + +Dart Dynamic Modules makes it possible to dynamically load a set of +Dart libraries in a running Dart application. It defines how those +libraries can interact with the rest of the application, and +provides utilities to facilitate their construction and correct +usage. + +### Basic architecture: Leaving a closed-world behind + +Historically, we have mostly treated Dart applications as a +monolith. Every time we build a Dart binary, we typically build the +entire application as a unit. Yes, there are some exceptions in +development scenarios where we support modular compilation, like DDC +on the web, but this whole-program approach is generally true across +our backends (dart2wasm, dart2js, AOT). Even features like deferred +loading that allow developers to split their binary into pieces to +lower latency during an initial load, operate with the whole program +first and only later they split the code into chunks. + +Dart Dynamic Modules moves us away from this monolith concept. Now, +an application can be built and more modules can be defined, +compiled, and loaded after the fact. This represents architectural +change: the application now acts as a ***host*** that is extensible +through dynamic modules over time. The main semantic restriction: +all extensions are additive. Dynamic modules cannot replace an +existing declaration in the application, not even if that +declaration was delivered through a dynamic module earlier. + +To extend an application, developers are required to define an API +surface area that establishes how dynamic modules will interact with +the rest of the host application. We call the declaration of this +surface area a ***Dynamic Interface***. With a surface area defined, +a host application is built as a single binary like before, but with +the caveat that the API surface area is now exposed. More libraries +can be compiled later on and loaded as dynamic modules, as long as +they are compatible with the API surface area defined by the host. + +Quick recap on our terminology: + +* **Host application**: the application that will load the dynamic + module. +* **Dynamic interface**: a declaration of the API surface area + exposed by a host application and made accessible to a dynamic + module. +* **Dynamic module**: a set of Dart libraries and an entrypoint + method, that are loaded dynamically. + +### Typical steps to use dynamic modules + +Let's dive into more details on how this architecture plays out. If +you are interested in how to use them directly and experiment with +them, skip ahead and look at our [example](#example). +Here we describe the conceptual steps that are needed when adopting +an application architecture that uses dynamic modules: + +* Step 1: First, you will need to have a special build of Dart that + supports dynamic modules. We don't ship this as part of the Dart + or Flutter SDKs, so at this time this needs to be built from + sources (we show how in the [example below](#example)). + +* Step 2: You need to determine an API surface area. To do so, you + usually start by imagining what logic will be loaded dynamically + and how it will be integrated into your application. This helps + you figure out the right contract between the application and the + dynamic modules, when and how the application will load them, and + identify which APIs need to be exposed. By default, nothing is + exposed, not even APIs in `dart:core`, unless you opt into them. + +* Step 3: Once you decide which APIs you want to expose from the + host, you write a special `dynamic_interface.yaml` file that + defines the dynamic interface (see + [format specification][dynamic_interface.md]). + This file is later used as an input to all compilation steps. + +* Step 4: With a dynamic interface in hand, you can build the host + application. There are three differences from how normal Dart + applications are built: + + * First, the compiler and runtime are slightly different in order + to include support for executing the dynamic modules. As we'll + see later below, for AOT we decided to adopt Dart bytecodes and + embed an interpreter in the runtime as the technology to execute + dynamic modules in AOT. + + * Second, the application code is compiled differently by + informing the compiler about the dynamic interface. + Traditionally, Dart's AOT compiler performs closed-world + optimizations assuming all code is known to the compiler, + dynamic modules break that assumption so the compiler needs + guidance to produce correct and sound applications. Usually this + means less code is tree-shaken and fewer calls can be + specialized because we don't know how exposed APIs will be used. + + * Third, the application code needs to embed some ***provenance*** + information that will be used later to validate the + compatibility of what we load dynamically. This is important to + guarantee correctness and memory safety and is a consequence of + the technological choices we made for implementing dynamic + modules. (Note: provenance is work in progress and not available + yet in the current implementation of the package). + +* Step 5: At this stage the application is ready to be delivered and + run. The logic of what comes in a dynamic module can be defined + later, even after the application is running. + +* Step 6: Create and build a dynamic module. This involves writing a + new set of Dart libraries with a special entrypoint. There is an + important constraint, dynamic modules can only define new code. If + they define libraries that already exist in the host application + or in previous dynamic modules, the module is going to be rejected + during the load. To deliver a dynamic module, we first compile it + to a format compatible with the runtime where the application is + running. In the case of AOT, we use a bytecode compiler to produce + the dynamic module. As hinted above, bytecode compilation not only + produces the bytecode that we will run, but also provenance + attestations needed in the validation process. + +* Step 7: Deliver the dynamic module to the app. Since dynamic + modules are delivered dynamically, some effort is needed to set up + the right way to deliver the dynamic module properly and securely. + This often involves some form of transport-level security (TLS). + +* Step 8: Validate and run the dynamic module. Before running the + dynamic module — which mainly consists of invoking the designated + entry point of the dynamic module and returning the result to the + host application — we need assurances that the dynamic module + won’t break the integrity of the application. Without validation, + loading a dynamic module could break the application in unexpected + ways (e.g. crashes, heap corruption, and more). Validation + typically confirms compatibility of the dynamic module format and + APIs in use. Depending on the technology, the verification may be + done differently. Our implementation based on bytecodes relies on + provenance checks to provide validation. Dynamic modules are + allowed to load only if the same application code, + dynamic-interface, and version of the SDK tools were used to build + both the host app and the dynamic module. + +That's the basic principle. + +### Additional semantic details + +This isn't a complete or formal spec, but here are some details to +expect about dynamic modules: + +* It's a static error to tear off or invoke a member that is not + exposed as "callable" in the dynamic interface. This rejection + typically happens in Step 6 of our conceptual steps above. +* It's a static error to extend a class that was not exposed as + "extendable" in the dynamic interface. +* It's a static error to override a member that was not exposed as + "can-be-overridden" in the dynamic interface. +* It's a static error to use a class as a type that was not exposed + as "callable" or "can-be-used-as-type" in the dynamic interface. +* All direct and interface calls are validated statically. +* Dynamic calls are generally discouraged because they can't be + verified statically. They are, however, supported with special + rules: + * By default, compilers will reject the use of a dynamic call + unless the selector name used in the dynamic call was previously + allowed. + + * A selector name can be allowed if elements with the same name + are exposed as "dynamically-callable" in the dynamic interface. + This makes it possible to use dynamic calls that originate in + dynamic modules but target a member in the host application. + + * A selector name can also be allowed through an allow-list flag + of the dynamic module compiler. This makes it possible to use + dynamic calls that originate from dynamic modules and target + members declared also in dynamic modules. + + * Unlike all other parts of the API surface area, runtime checks + are performed to enforce that the dynamic interface was + respected. Runtime errors may be reported on dynamic calls that + originate in the dynamic module if the target of the call was a + member of the host application that was not exposed as + "dynamically-callable". +* It's a runtime error to load a dynamic module with an incompatible + format. +* It's a runtime error to load a dynamic module that attempts to + define a library with an import-URI that already exists in the + program, regardless of whether that import-URI was used for a + library defined by the host application or a prior dynamic module. + +## Technology: Bytecode-based Implementation in AOT + +For each backend, there are many possible ways to bring dynamic +modules to life. DDC, for instance, is a modular compiler, so a +dynamic module can directly be implemented as a new module with very +little change to the compiler architecture. For AOT, we decided to +adopt an architecture that uses an interpreter. + +The Dart bytecode [interpreter][interpreter.cc] exists in the Dart +VM codebase. It requires a tight integration with the runtime, which +makes it difficult to deliver separately from the Dart runtime +itself. At the same time, it has an implicit code-size cost on the +runtime that we don't want to include by default on all Dart +applications today. Because of these reasons, we don't ship it by +default as part of our SDK today. Instead, to use the dynamic +modules experiment, we require passing a build-time flag when +building Dart. + +We also have a [dart2bytecode compiler][dart2bytecode], which we +use to compile dynamic modules to bytecode. In the future, we expect +the format to deliver dynamic modules will be expanded beyond +bytecode to allow embedding resources (e.g. icons, images, +provenance). + +Finally, we are in the process of putting together tools to produce +provenance information. This includes information about how an app +and dynamic module were created, including versioning details about +the inputs and tools used in the process. This may be embedded from +the bytecode compiler or provided separately. We'll update this +README as we make progress on the implementation. + +The libraries in `package:dynamic_modules` currently provide an API +to load bytecode directly. This will be updated in the near future +once we update the format and embed provenance information. Stay +tuned for changes. + +## Example + +The [`example/`](example/) directory contains a simple example that +illustrates the steps described above. The example is a simple +calculator command line interface that lets users extend it with new +operations beyond those built-in. + +The example [README.md](example/README.md) contains details on how +to navigate it. Here, we'd like to highlight how the example +connects with some of the conceptual steps we described above. + +The calculator consists of 3 parts: + +* **An API surface area** exposes the interface used by dynamic + modules to define new operations, as well as the API to register + them. This corresponds to steps 2 and 3 above. See + [`dynamic_interface.yaml`](example/dynamic_interface.yaml) + and [`common/common.dart`](example/common/common.dart). + +* **The host app** defines the main REPL (read–eval–print loop) and + provides one built-in operation (math addition). This corresponds + to step 4 above. See [`host/main.dart`](example/host/main.dart). + +* **Dynamic modules** define new operations for the calculator + following the API surface area defined by the host application. + This corresponds to step 6 above. See the + [`modules/`](example/modules/) subfolder. + +Finally everything comes together in the [`run.sh`](example/run.sh) +script. This script shows: + +* How to build Dart SDK artifacts with support for Dart Dynamic + Modules (Step 1 above) by providing the `--dart-dynamic-modules` + build-time flag. + +* How to use the dynamic interface to build artifacts of steps 4 and + 6. + +This example doesn't show how dynamic modules are delivered over the +network (step 7), instead bytecode files are copied over to a data +directory as an over simplistic way to represent the delivery +process. + +## Progress/Status + +Dynamic modules are designed so they can eventually be supported in +any Dart backend, however we have primarily focused on the AOT +implementation. All other backends are either unsupported or +incomplete. + +What works? + +* AOT compilation factors dynamic-interface to prevent tree-shaking + and invalid optimizations and to perform runtime validation of + dynamic calls. +* Modular validation of API usage against dynamic-interface. +* Integration of bytecode interpreter in the AOT runtime. +* Integration of bytecode interpreter in the JIT runtime. +* Required runtime checks in the AOT runtime needed to validate + dynamic calls emanating from dynamic modules. For context, most + validation of dynamic modules is done statically and modularly + while compiling dynamic modules. However, the use of dynamic calls + in dynamic modules require an additional whole-program static + check and runtime checks. + +What doesn't work yet? + +* Bytecode provenance validation. There is work in progress, but the + APIs have not been migrated over. This is a known upcoming + breaking change to our APIs. +* Whole-program validation of dynamic modules. As mentioned earlier, + dynamic calls also need a whole-program static check. This check + is not implemented yet. +* Required runtime checks in the JIT runtime to validate dynamic + calls. This is missing mainly because the JIT runtime doesn't load + the dynamic-interface when running a host application from + sources. +* Simple developer experience when authoring dynamic interfaces. + Currently the `dynamic_interface.yaml` format has quirks + especially around private members, which sometimes need to be + exposed due to low-level implementation details of the language. +* Invoking FFI APIs from a dynamic module. + +What is currently not planned? + +* We are primarily focused on the AOT pipeline. We may improve + support in JIT, but do not plan to focus on the web backends (DDC, + dart2js, or dart2wasm) at this time. While this package includes + tests that use an old DDC implementation based on DDC's JS module + systems, that implementation was primarily created to validate the + dynamic-modules concepts and ensure the semantics were flexible + enough to support Dart backends other than AOT. + +## Technical details, Considerations, FAQ + +**Q: How are dynamic modules executed in AOT?** + +The current implementation in AOT focuses entirely on using bytecode +to deliver dynamic modules and executing it using a bytecode +interpreter. That said, in general Dart Dynamic Modules as a concept +is agnostic of how code gets executed. We only define how code +extends an existing application and how it interacts with it. In +fact, prototypes we have built with DDC and dart2wasm leveraged +their own JS or wasm module systems to load new code. + +**Q: Are dynamic modules fully sandboxed?** + +No. Dynamic modules run in an interpreter, but you are loading code +into the same context of the application. Both the original host +application code and the dynamic modules run together as a single +program with a shared heap. Even though dynamic modules have +restricted APIs to refer to, there are no restrictions on what those +APIs can access indirectly. + +You can create some lightweight isolation by loading dynamic modules +in a new Dart isolate. This separates the heap from the rest of the +host application, but won’t provide a full sandbox as it will still +be running in the same OS process as the app. + +We think it is an interesting future direction to explore creating +trim-down versions of the Dart runtime that are fully sandboxed +(e.g. absolute no access to IO or FFI), though. + +**Q: Is it OK to load untrusted/external/third-party code as a +dynamic module?** + +No. As mentioned above, there is no sandbox. Anything you load with +a dynamic module requires a lot of trust. Just like it's not OK to +load untrusted third-party code without a sandbox in the same +context as your web application, it's not OK to do so in a Dart +application. Any code that is not developed by your team (external +contributions, unreviewed LLM generated code, etc) should be treated +as untrusted. + +**Q: Why will the API change to require provenance information?** + +We will require provenance information to validate compatibility +before a dynamic module is loaded and run in a host application. +Unlike other bytecode formats, like Java bytecodes, the Dart +bytecode format has no stable ABI and no mechanism to verify that +APIs used from the bytecode are compatible. As such, we need +additional mechanisms to prove compatibility and verify that a +dynamic module will behave as expected when loaded into a running +host application. Since compilation to bytecode already performs +most of the verification steps we need, we are designing the system +to plumb provenance information as a way to give the host +application the proof it needs to confirm everything is in order +before proceeding. + +**Q: Why are Dart bytecodes and Dart dynamic modules being +developed?** + +We are currently exploring improvements to the development story of +Dart and Flutter applications. Historically we relied heavily on JIT +compilation, but we were exploring using modular AOT compilation in +combination with Dart bytecodes to provide alternative development +workflows that can be supported in environments where JIT +compilation is not feasible. We discovered that the same technology +can be used for loading code dynamically, which led us to explore +new development use cases of the technology. One such use-case is to +share prototypes between team members at a low cost. + +**Q: Can I use dynamic modules today in my Dart/Flutter app?** + +We are in an early experimental stage of this feature, so we don't +recommend taking a dependency on it for projects used in production +today. You'll note that the feature is also not available by default +in Dart and Flutter SDKs, in part because just enabling it +introduces code-size bloat and performance effects that are +unnecessary for existing applications today. + +We are prioritizing dynamic modules for development use cases, where +enabling dynamic modules at build time is an option and the +trade-offs are less of an issue. + +**Q: Why are you only prioritizing development use cases?** + +There are many considerations that need to be factored when +supporting a feature for production use cases. In the case of Dart +Dynamic Modules, the consideration list is long and requires a lot +of diligence, care, and effort, both from the SDK and from actual +developers adopting it. We decided to focus only on development use +cases that allow us to reduce the scope of these considerations. + +Here are some of the top-of-mind factors for this technology: + +* This feature can increase application size unnecessarily. The + interpreter has a fixed cost, but there are additional effects on + AOT binaries from disabling tree-shaking and optimizations related + to the dynamic interface. If used naively, that extra size bloat + can be substantial. + +* If used improperly, this feature can introduce security risks. + Dynamic modules are loaded in a trusted environment with no extra + sandbox, so care needs to be in place to ensure bytecode has the + right provenance and is delivered safely. Our work on requiring + provenance is an effort to reduce this risk, but it doesn't + eliminate all attack vectors. + +* If used improperly, this feature can break applications or cause + them to behave unexpectedly. For example, accidentally mixing + bytecode compiled against a different version of the app can cause + issues. Our work on provenance also helps reduce this risk by + validating compatibility, but it doesn't reduce the complexity + required to manage versioning in the first place. + +* Proper version management involves non-trivial efforts, including + creating server components and properly managing creation and + storage of bytecode. We think a production-ready solution should + have clear guidance and designs for how to address this challenge. + +By focusing on the development use cases we found that some of these +considerations have a lower impact or lower complexity. For +instance, code-size is a lesser concern and we can focus on +single-version scenarios that ignore the more general versioning +questions. We still believe these considerations are important and +relevant, so we have used the development scenarios to help us make +progress on them (e.g. provenance support). + +**Q: Is Dart Dynamic Modules a "code-push" implementation?** + +No. Dart Dynamic Modules is a language runtime technology, similar +to that available in many other languages. It resembles C dynamic +linking, JavaScript dynamic imports, wasm module instantiations. + +Code-push approaches are full-stack solutions that use multiple +technologies and services underneath to enable developers to send +over-the-air updates that patch existing applications. A general +code-push solution needs hosting services to track releases and +patches over time, mechanisms to make the application updatable +in-place, to determine what has changed between patches, to encode +updates in a deliverable format, to deliver and apply such updates, +and more. + +We believe Dart Dynamic Modules is not an ideal technology for a +general-purpose codepush solution for a few reasons. First, the +additive semantics: dynamic modules can only introduce new +libraries, but can't update any existing declaration in an +application. Second, the API surface area: dynamic modules can only +invoke code that was previously exposed. In contrast, code-push +solutions want to update existing declarations and want patches to +be able to invoke anything already in the application. For a dynamic +module to be able to invoke anything in an application, we would +need to expose the entire app in the dynamic interface. That would +practically disable tree-shaking and all whole-program +optimizations, dramatically affecting the original application +performance and code-size to the point that we doubt it would be +suitable for use in production environments. + +**Q: Are there security concerns behind dynamic modules?** + +Yes. As hinted above in the earlier questions about how it works, +some top security considerations are that: + +* A dynamic module loads in the same context as the host app, + without an additional sandbox. Compared to the web, this is + equivalent to appending `<script>` tags without using an + iframe. As such, similar precautions are needed to use dynamic + modules safely. This includes ensuring the code is trusted, + preventing injection scenarios that would tamper the integrity of + the bytecode. For example, you should never use dynamic modules to + load code provided by others or provided by LLMs. Also, even if + the source of the code is trusted, the delivery needs to be + secured with proper TLS. + +* Our bytecode doesn't have a stable ABI, so compatibility is only + guaranteed if the bytecode loaded matches the format supported by + the host app. A skew in the ABI can cause memory corruption or + other semantic errors and lead the application to misbehave. + +* Similarly, incompatible APIs can also lead to the same result. + +Our work to provide provenance validation is designed to help +validate ABI and API compatibility before a load is allowed. Unlike +Java bytecodes, Dart bytecodes don't offer verification that the +APIs in the bytecode match the APIs in the host app, so we resort to +provenance as an alternative technique. Just like the web, +provenance alone does not guarantee the safety of the code being +loaded. Special diligence is required by developers to ensure that +only code that is trusted and delivered securely is ever loaded in +the context of the application. + +**Q: Are there store policy considerations to consider when using +dynamic modules?** + +Yes, there are policy considerations, regardless of how code is run. +For example, you might want to validate that any store you are +publishing to allows running interpreted code in a custom +interpreter. And stores might have policies around how much an app +is allowed to change after it has been installed by the user. + +[interpreter.cc]: https://github.com/dart-lang/sdk/blob/898a1e4bbfbc472dc0a9505dc7d2e4c21d6f856e/runtime/vm/interpreter.cc +[dart2bytecode]: https://github.com/dart-lang/sdk/blob/898a1e4bbfbc472dc0a9505dc7d2e4c21d6f856e/pkg/dart2bytecode/ +[dynamic_interface.md]: https://github.com/dart-lang/sdk/blob/898a1e4bbfbc472dc0a9505dc7d2e4c21d6f856e/pkg/vm/dynamic_interface.md