Build Hook Specification

Version 1: As of 2024-01-24 implemented behind --enable-experiment=native-assets

Concepts

Asset

An asset is data which is accessible from a Dart or Flutter application. To retrieve an asset at runtime, we associate with it a string as a unique identifier, the assetId. There are several example types of assets:

  • Assets which designate symbols present in the target system, process, or executable. They are identified by their name.
  • Dynamic libraries bundled into the application.

An application is compiled to run on a certain target OS and architecture. If different targets require different assets, the package developer must specify which asset to bundle for which target.

An asset has different ways of being accessible in the final application. It is either brought in “manually” by having the package developer specify a path of the asset on the current system, it can be part of the Dart or Flutter SDK, or it can be already present in the target system. If the asset is bundled “manually”, the Dart or Flutter SDK will take care of copying the asset from its specified location on the current system into the application bundle.

Assets are also called “native assets” to differentiate them from the Dart code also bundled with an application.

AssetId

An asset must have a string identifier called assetId. Dart code that uses an asset, references the asset using this assetId.

A package must prefix all assetIds it defines with: package:<package>/, <package> being the current package‘s name. This ensures assets don’t conflict between packages.

Additionally, the convention is that an asset referenced from lib/src/foo.dart in package:foo has the assetId 'package:foo/src/foo.dart'.

Target

A target specifies the operating system and architecture of the targeted platform of the application.

build.dart

Any package which needs to access assets at runtime must define a build.dart script to specify these assets. If there are assets in a package which are not shipped as part of Dart or Flutter and also not present on the target system by default, those assets will be bundled into the final application by the Dart or Flutter SDK automatically. The build.dart script takes as input information such as the target OS and architecture in a build_config.yaml file, and returns as output a list of assets which can be accessed at runtime in a build_output.yaml file.

build_config.yaml

This is the structure of the input to a build.dart script.

# Build in dry-run mode.
#
# Running in dry-run mode `<out_dir>/build_output.yaml` must be written, but
# the files it references need not exist.
dry_run: true | false

# Build Mode.
#
# A hint `build.dart` can use to determined which optimizations to enable and
# whether or not to include debug symbols in the format relevant for the asset.
#
# Not provided on dry runs.
build_mode: release | debug

# Metadata as output from build.dart from direct dependencies.
#
# Not provided on dry runs.
dependency_metadata:
  # package name of direct dependency.
  some_package_name:
    # key value pairs.
    some_key: some_value

# Preferred link mode
link_mode_preference: dynamic | static | prefer-dynamic | prefer-static

# Path to output directory where assets should be placed.
#
# This is also where `build_output.yaml` should be written.
#
# Remark: Avoid using the name "build_output.yaml" for an asset file, this is
#         forbidden.
out_dir: /absolute/path/to/out_dir/

# Name of the package that contains the `build.dart`
#
# Remark: This is entirely redundant since this is a config file specified to
# `build.dart`, and the author of `build.dart` probably knows the name of the
# package they are writing.
package_name: my_package_with_native_assets

# Path to root folder for the package that contains `build.dart`.
#
# This is useful if `build.dart` wishes to find source code files embedded in
# its own package and compile them to an asset.
#
# Note that this will be most likely a path in the pub cache when the package
# with a `build.dart` is a dependency of another package.
package_root: /absolute/path/to/my_package_with_native_assets

# Target architecture
#
# Combined with `target_os` this specifies the "target" for which assets
# should be built.
#
# Not provided on dry runs.
target_architecture: x64 | ia32 | arm | arm64 | riscv32 | riscv64

# Target operating system
#
# Combined with `target_architecture` this specifies the "target" for which
# assets should be built.
target_os: android | ios | linux | macos | windows

# Schema version of this file.
version: 1.0.0

build_output.yaml

This file is the output from running a build.dart script, and contains the list of assets to be bundled into the application.

# The list of assets.
#
# In dry runs, must contain assets for each architecture for the requested os.
assets:
  - id: 'package:my_package_with_native_assets/src/foo.dart'
    link_mode: dynamic | static | prefer-dynamic | prefer-static
    path:
      path_type: absolute | system | process | executable
      # Only provided for path_type absolute and system.
      #
      # If path_type absolute: The absolute path to the file name on the
      # current machine.
      #
      # If path_type system: The path of the dynamic library as available on 
      # the target machine's PATH.
      uri: /absolute/path/to/outdir/arbitrary_filename.whatever
    target: linux_x64
  ...

# The files used by this build.
#
# If any of the files in [dependencies] are modified after [timestamp], the
# build will be re-run.
#
# Not output on dry runs.
dependencies:
  - /absolute/path/to/my_package_with_native_assets/build.dart
  ...

# The time the build this output is for started.
#
# Must be before any of the files in dependencies are read.
timestamp: 2024-01-02 17:05:35.000

# Metadata usable for build.dart of packages directly depending on this package.
#
# Not output in dry runs.
metadata:
  # Key value pairs.
  some_key: some_value

# Schema version of this file.
version: 1.0.0

Running build.dart

The input is passed as a absolute path to a YAML file build_config.yaml encoding the input information as follows:

$ dart build.dart --config <build_config.yaml>

The Dart and Flutter SDK invoke build.dart of all packages in the transitive dependencies and pass a build_config.yaml as the --config option.

If not in dry_run mode, the build.dart file MUST:

  • Read the build_config.yaml file,
  • Create assets using the configuration from build_config.yaml, if not already existing.
  • Write asset files into the directory with path out_dir that was provided by build_config.yaml.
    • MUST avoid file name build_output.yaml.
  • Write build_output.yaml into the directory at out_dir.
    • This maps the assetIds to assets previously written into out_dir.
    • There may be multiple assets for a given assetId depending on characteristics like target, link_mode, etc.
    • If the boolean parameter dry_run is set to true in the build configuration, the list of assets must be output, but the asset files must not be written. The asset file urls are expected to refer to non-existing files.

Notes:

  • The file name inside out_dir is irrelevant, because in Dart code the asset will only be referenced by its assetId.
  • If the boolean parameter dry_run is set to true in the build configuration, then the actual creation and writing of assets can be skipped, as long as the assets are still listed in the build_output.yaml.

.dart_tool/native_assets.yaml

This is not part of the build.dart protocol.

This file is internal to the Dart SDK and Flutter SDK. It produced by native_asset_builder (as embedded in the SDKs). For each target it maps from assetId to a path type with an optional path.

These paths are used to resolve @Native() external functions in Dart code.

The path types in this file are as follows:

  • absolute paths are absolute paths on the system where the Dart executable is running. This path type is used when running in JIT mode on a developers' host machine, or in an Android app where the root path is the Android bundle.
  • relative paths are relative to the kernel or aot snapshot. This path type is used for shipping native assets in a directory together with a kernelsnapshot, aotsnapshot, or standalone executable (dartaotruntime+aotsnapshot).
  • system paths are expected to resolve on the target machine PATH.
  • process “paths” have no path, symbols are resolved in the current process.
  • executable “paths” have no path, symbols are resolved in the current executable.
# Schema version of this file.
format-version: [1, 0, 0]

# A mapping from target and assetId to an asset.
native-assets:
  # A <target>.
  linux_x64:
    # An <assetId>.
    'package:my_package_with_native_assets/src/foo.dart':
      # A list of path_type and optionally a path.
      - absolute | relative | system | process | executable
      - <url> # Only provided for absolute, relative, and system path types.
  ...

This file is used by the embedded SDK to resolve assetIds when running in development mode or build a deployable application (or standalone executable).

Wishes to improve for v2

  • Multiple asset types. Rename Asset to NativeCodeAsset and introduce DataAssets.
  • The path_types really only make sense for link_mode: dynamic.
  • Fix the mismatch between target for output but os and architecture for config.
    • Should the architecture field of native code assets be optional and be ommitted in dry runs?
  • Introduce a link.dart protocol with a link input and link output.