| [](https://github.com/dart-lang/native/actions/workflows/jnigen.yaml) |
| [](https://coveralls.io/github/dart-lang/native?branch=main) |
| [](https://pub.dev/packages/jnigen) |
| [](https://pub.dev/packages/jnigen/publisher) |
| |
| ## Introduction |
| Experimental bindings generator for Java bindings through dart:ffi and JNI. |
| |
| `jnigen` scans compiled JAR files or Java source code to generate a description of the API, then uses it to generate Dart bindings. The Dart bindings call the C bindings, which in-turn call the Java functions through JNI. Shared functionality and base classes are provided through the support library, `package:jni`. |
| |
| The configuration for binding generation is usually provided through YAML. |
| |
| Three configuration details are needed to generate the bindings. Everything else is optional: |
| |
| * _Inputs_: input can be Java source files (`source_path`), or compiled classes / JARs (`class_path`). Some maven / gradle based tooling is also provided to simplify obtaining dependencies. |
| |
| * _Outputs_: Output can be generated in package-structured (one file per class) or single file bindings. Target path to write Dart bindings needs to be specified. |
| |
| * _Classes_: Specify which classes or packages you need bindings for. Specifying a package includes all classes inside it recursively. |
| |
| Check out the [examples](jnigen/example/) to see some sample configurations. |
| |
| ## Example |
| It's possible to generate bindings for JAR libraries, or Java source files. |
| |
| Here's a simple example Java file, in a Flutter Android app. |
| |
| ```java |
| package com.example.in_app_java; |
| |
| import android.app.Activity; |
| import android.widget.Toast; |
| import androidx.annotation.Keep; |
| |
| @Keep |
| public abstract class AndroidUtils { |
| // Hide constructor |
| private AndroidUtils() {} |
| |
| public static void showToast(Activity mainActivity, CharSequence text, int duration) { |
| mainActivity.runOnUiThread(() -> Toast.makeText(mainActivity, text, duration).show()); |
| } |
| } |
| ``` |
| |
| This produces the following boilerplate: |
| |
| #### Dart Bindings: |
| |
| ```dart |
| /// Some boilerplate is omitted for clarity. |
| class AndroidUtils extends jni.JObject { |
| @override |
| late final jni.JObjType<AndroidUtils> $type = type; |
| |
| AndroidUtils.fromReference( |
| jni.JReference reference, |
| ) : super.fromReference(reference); |
| |
| static final _class = |
| jni.JClass.forName(r"com/example/in_app_java/AndroidUtils"); |
| |
| /// The type which includes information such as the signature of this class. |
| static const type = $AndroidUtilsType(); |
| |
| static final _id_showToast = _class.staticMethodId( |
| r"showToast", |
| r"(Landroid/app/Activity;Ljava/lang/CharSequence;I)V", |
| ); |
| |
| static final _showToast = ProtectedJniExtensions.lookup< |
| ffi.NativeFunction< |
| jni.JThrowablePtr Function( |
| ffi.Pointer<ffi.Void>, |
| jni.JMethodIDPtr, |
| ffi.VarArgs< |
| ( |
| ffi.Pointer<ffi.Void>, |
| ffi.Pointer<ffi.Void>, |
| ffi.Int64 |
| )>)>>("globalEnv_CallStaticVoidMethod") |
| .asFunction< |
| jni.JThrowablePtr Function(ffi.Pointer<ffi.Void>, jni.JMethodIDPtr, |
| ffi.Pointer<ffi.Void>, ffi.Pointer<ffi.Void>, int)>(); |
| |
| /// from: `static public void showToast(android.app.Activity mainActivity, java.lang.CharSequence text, int duration)` |
| static void showToast( |
| jni.JObject mainActivity, |
| jni.JObject text, |
| int duration, |
| ) { |
| _showToast(_class.reference.pointer, _id_showToast as jni.JMethodIDPtr, |
| mainActivity.reference.pointer, text.reference.pointer, duration) |
| .check(); |
| } |
| } |
| ``` |
| |
| The YAML configuration used to generate the above code looks like this: |
| |
| ```yaml |
| android_sdk_config: |
| add_gradle_deps: true |
| |
| output: |
| dart: |
| path: lib/android_utils.dart |
| structure: single_file |
| |
| source_path: |
| - 'android/app/src/main/java' |
| classes: |
| - 'com.example.in_app_java.AndroidUtils' |
| ``` |
| |
| The complete example can be found in [jnigen/example/in_app_java](jnigen/example/in_app_java), which adds few more classes to demonstrate using classes from gradle JAR and source dependencies. |
| |
| ## Supported platforms |
| | Platform | Dart Standalone | Flutter | |
| | -------- | --------------- | ------------- | |
| | Android | n/a | Supported | |
| | Linux | Supported | Supported | |
| | Windows | Supported | Supported | |
| | MacOS | Supported | Not Yet | |
| |
| On Android, the flutter application runs embedded in Android JVM. On other platforms, a JVM needs to be explicitly spawned using `Jni.spawn`. `package:jni` provides the infrastructure for initializing and managing the JNI on both Android and Non-Android platforms. |
| |
| ## Java features support |
| Currently basic features of the Java language are supported in the bindings. Each Java class is mapped to a Dart class. Bindings are generated for methods, constructors and fields. Exceptions thrown in Java are rethrown in Dart with stack trace from Java. |
| |
| More advanced features such as callbacks are not supported yet. Support for these features is tracked in the [issue tracker](https://github.com/dart-lang/native/issues?q=is%3Aopen+is%3Aissue+label%3Apackage%3Ajni%2Cpackage%3Ajnigen). |
| |
| ## Note on Dart (standalone) target |
| `package:jni` is an FFI plugin containing native code, and any bindings generated from jnigen contains native code too. |
| |
| On Flutter targets, native libraries are built automatically and bundled. On standalone platforms, no such infrastructure exists yet. As a stopgap solution, running `dart run jni:setup` in a target directory builds all JNI native dependencies of the package into `build/jni_libs`. |
| |
| The build directory has to be passed to `Jni.spawn` call. It's assumed that all dependencies are built into the same target directory, so that once JNI is initialized, generated bindings can load their respective C libraries automatically. |
| |
| ## Requirements |
| ### SDK |
| Flutter SDK is required. |
| |
| Dart standalone target is supported, but due to some problems with pubspec format, the `dart` command must be from Flutter SDK and not Dart SDK. See [dart-lang/pub#3563](https://github.com/dart-lang/pub/issues/3563). |
| |
| ### Java tooling |
| |
| **Use JDK versions 11 to 17. The newer versions will not work because of their lack of [compatibility](https://docs.gradle.org/current/userguide/compatibility.html) with Gradle.** |
| |
| Along with JDK, maven (`mvn` command) is required. On windows, it can be installed using a package manager such as [chocolatey](https://community.chocolatey.org/packages/maven) or [scoop](https://scoop.sh/#/apps?q=maven). |
| |
| __On windows, append the path of `jvm.dll` in your JDK installation to PATH.__ |
| |
| For example, on Powershell: |
| |
| ```powershell |
| $env:Path += ";${env:JAVA_HOME}\bin\server". |
| ``` |
| |
| If JAVA_HOME not set, find the `java.exe` executable and set the environment variable in Control Panel. If java is installed through a package manager, there may be a more automatic way to do this. (Eg: `scoop reset`). |
| |
| ### C tooling |
| CMake and a standard C toolchain are required to build `package:jni`. |
| |
| ## FAQs |
| |
| #### I am getting ClassNotFoundError at runtime. |
| `jnigen` does not handle getting the classes into application. It has to be done by target-specific mechanism. Such as adding a gradle dependency on Android, or manually providing classpath to `Jni.spawn` on desktop / standalone targets. |
| |
| On Android, `proguard` prunes classes which it deems inaccessible. Since JNI class lookup happens in runtime, this leads to ClassNotFound errors in release mode even if the dependency is included in gradle. [in_app_java example](jnigen/example/in_app_java/) discusses two mechanisms to prevent this: using `Keep` annotation (`androidx.annotation.Keep`) for the code written in the application itself, and [proguard-rules file](jnigen/example/in_app_java/android/app/proguard-rules.pro) for external libraries. |
| |
| Lastly, some libraries such as `java.awt` do not exist in android. Attempting to use libraries which depend on them can also lead to ClassNotFound errors. |
| |
| #### `jnigen` is not finding classes. |
| Ensure you are providing correct source and class paths, and they follow standard directory structure. If your class name is `com.abc.MyClass`, `MyClass` must be in `com/abc/MyClass.java` relative to one of the source paths, or `com/abc/MyClass.class` relative to one of the class paths specified in YAML. |
| |
| If the classes are in JAR file, make sure to provide path to JAR file itself, and not the containing directory. |
| |
| #### `jnigen` is unable to parse sources. |
| If the errors are similar to `symbol not found`, ensure all dependencies of the source are available. If such dependency is compiled, it can be included in `class_path`. |
| |
| #### How are classes mapped into bindings? |
| Each Java class generates a subclass of `JObject` class, which wraps a `jobject` reference in JNI. Nested classes use `_` as separator, `Example.NestedClass` will be mapped to `Example_NestedClass`. |
| |
| #### Does `JObject` hold a local or global reference? Does it need to be manually released? |
| Each Java object returned into Dart creates a JNI global reference. Reference deletion is taken care of by `NativeFinalizer` and that's usually sufficient. |
| |
| It's a good practice to keep the interface between languages sparse. However, if there's a need to create several references (Eg: in a loop), you can use FFI Arena mechanism (`using` function) and `releasedBy` method, or manually release the object using `release` method. |
| |
| #### Should I use `jnigen` over Method channels? |
| This is currently an experimental package. Many features are missing, and it's rough around the edges. You're welcome to try it and give feedback. |
| |
| ## YAML Configuration Reference |
| Keys ending with a colon (`:`) denote subsections. |
| |
| The typical invocation with YAML configuration is |
| |
| ``` |
| dart run jnigen --config jnigen.yaml |
| ``` |
| |
| Any configuration can be overridden through command line using `-D` or `--override` switch. For example `-Dlog_level=warning` or `-Dsummarizer.backend=asm`. (Use `.` to separate subsection and property name). |
| |
| A `*` denotes required configuration. |
| |
| | Configuration property | Type / Values | Description | |
| | ---------------------- | ------------- | ----------------------------------------------------------------------- | |
| | `preamble` | Text | Text to be pasted in the start of each generated file. | |
| | `source_path` | List of directory paths | Directories to search for source files. Note: source_path for dependencies downloaded using `maven_downloads` configuration is added automatically without the need to specify here. | |
| | `class_path` | List of directory / JAR paths | Classpath for API summary generation. This should include any JAR dependencies of the source files in `source_path`. | |
| | `classes` * | List of qualified class / package names | List of qualified class / package names. `source_path` will be scanned assuming the sources follow standard java-ish hierarchy. That is a.b.c either maps to a directory `a/b/c` or a class file `a/b/c.java`. | |
| | `enable_experiment` | List of experiment names:<br><ul><li>`interface_implementation`</li></ul> | List of enabled experiments. These features are still in development and their API might break. | |
| | `output:` | (Subsection) | This subsection will contain configuration related to output files. | |
| | `output:` >> `dart:` | (Subsection) | This subsection specifies Dart output configuration. | |
| | `output:` >> `dart:` >> `structure` | `package_structure` / `single_file` | Whether to map resulting dart bindings to file-per-class source layout, or write all bindings to single file. |
| | `output:` >> `dart:` >> `path` * | Directory path or File path | Path to write Dart bindings. Should end in `.dart` for `single_file` configurations, and end in `/` for `package_structure` (default) configuration. | |
| | `maven_downloads:` | (Subsection) | This subsection will contain configuration for automatically downloading Java dependencies (source and JAR) through maven. | |
| | `maven_downloads:` >> `source_deps` | List of maven package coordinates | Source packages to download and unpack using maven. The names should be valid maven artifact coordinates. (Eg: `org.apache.pdfbox:pdfbox:2.0.26`). The downloads do not include transitive dependencies. | |
| | `maven_downloads:` >> `source_dir` | Path | Directory in which maven sources are extracted. Defaults to `mvn_java`. It's not required to list this explicitly in source_path. | |
| | `maven_downloads:` >> `jar_only_deps` | List of maven package coordinates | JAR dependencies to download which are not mandatory transitive dependencies of `source_deps`. Often, it's required to find and include optional dependencies so that entire source is valid for further processing. | |
| | `maven_downloads:` >> `jar_dir` | Path | Directory to store downloaded JARs. Defaults to `mvn_jar`. | |
| | `log_level` | Logging level | Configure logging level. Defaults to `info`. | |
| | `android_sdk_config:` | (Subsection) | Configuration for autodetection of Android dependencies and SDK. Note that this is more experimental than others, and very likely subject to change. | |
| | `android_sdk_config:` >> `add_gradle_deps` | Boolean | If true, run a gradle stub during `jnigen` invocation, and add Android compile classpath to the classpath of jnigen. This requires a release build to have happened before, so that all dependencies are cached appropriately. | |
| | `android_sdk_config:` >> `android_example` | Directory path | In case of an Android plugin project, the plugin itself cannot be built and `add_gradle_deps` is not directly feasible. This property can be set to relative path of package example app (usually `example/` so that gradle dependencies can be collected by running a stub in this directory. See [notification_plugin example](jnigen/example/notification_plugin/jnigen.yaml) for an example. | |
| | `summarizer:` | (Subsection) | Configuration specific to summarizer component, which builds API descriptions from Java sources or JAR files. | |
| | `summarizer:` >> `backend` | `auto`, `doclet` or `asm` | Specifies the backend to use in API summary generation. `doclet` uses OpenJDK Doclet API to build summary from sources. `asm` uses ASM library to build summary from classes in `class_path` JARs. `auto` attempts to find the class in sources, and falls back to using ASM. | |
| | `summarizer:` >> `extra_args` (DEV) | List of CLI arguments | Extra arguments to pass to summarizer JAR. | |
| | `exclude:` | (Subsection) | Exclude methods or fields using regex filters. It's generally useful to exclude problematic fields or methods which, with current binding generation, can lead to syntax errors | |
| | `exclude:` >> `methods`| List of methods in `classBinaryName#methodName` format where classBinaryName is same as qualified name, but `$` preceding a nested class instead of `.`. Example: `com.example.MyClass` or `com.example.MyClass$NestedClass` | Methods to exclude. |
| | `exclude:` >> `fields` | List of fields in `classBinaryName#fieldName` format | Fields to exclude. |
| |
| It's possible to use the programmatic API instead of YAML. |
| |
| * Create a tool script. (Eg: `tool/generate_jni_bindings.dart`) |
| * import `package:jnigen/jnigen.dart` |
| * construct a `Config` object and pass it to `generateJniBindings` function. The parameters are similar to the ones described above. |
| |
| ## Android core libraries |
| These days, Android projects depend heavily on AndroidX and other libraries downloaded via gradle. We have a tracking issue to improve detection of android SDK and dependencies. (#31). Currently we can fetch the JAR dependencies of an android project, by running a gradle stub, if `android_sdk_config` >> `add_gradle_deps` is specified. |
| |
| But core libraries (the `android.**` namespace) are not downloaded through gradle. The core libraries are shipped as stub JARs with the Android SDK. (`$SDK_ROOT/platforms/android-$VERSION/android-stubs-src.jar`). |
| |
| Currently we don't have an automatic mechanism for using these. You can unpack this JAR manually into some directory and provide it as a source path. |
| |
| However there are 2 caveats to this caveat. |
| |
| * SDK stubs after version 28 are incomplete. OpenJDK Doclet API we use to generate API summaries will error on incomplete sources. |
| * The API can't process the `java.**` namespaces in the Android SDK stubs, because it expects a module layout. So if you want to generate bindings for, say, `java.lang.Math`, you cannot use the Android SDK stubs. OpenJDK sources can be used instead. |
| |
| The JAR files (`$SDK_ROOT/platforms/android-$VERSION/android.jar`) can be used instead. But compiled JARs do not include JavaDoc and method parameter names. This JAR is automatically included by Gradle when `android_sdk_config` >> `add_gradle_deps` is specified. |
| |
| ## Migrating to 0.8.0 |
| |
| Check [`jni`'s changelog](https://github.com/dart-lang/native/blob/main/pkgs/jni/CHANGELOG.md#080-wip) and [`jnigen`'s changelog](https://github.com/dart-lang/native/blob/main/pkgs/jnigen/CHANGELOG.md#080-wip). |
| |
| ## Contributing |
| See the wiki for architecture-related documents. |
| |