Initial generator: functions, structs, enums (#2)

* migrated code from dart_ffi_bindings_generator repository

Commits till now:

Prerak Mann (6):
      init stagehand dart package template, minor changes
      Updated Project Structure, Readme
      Reading pubspec configuration via yaml package
      setup tool - libclang_binding_generator
      restructure code
      setup basic classes for code_generator

* update readme: building steps

* code_generation: added function generation for primitives

* code_generator: added test for function

* code_generation: added struct generation for primitives, added tests

* code_generation: added pointer support for function and structure, added test

* tool: added script for basic clang_binding generator

* code_generator: added constant, global, C string support (ptr to utf8), added tests

* code_generator: added typedefc, added test

* tool: completed basic clang binding generation

* update readme

* simplify setup

* added basic parser logic, fixed visitChildren_wrap function

* copied clang_constants from libclang

* header_parser: revamped api and code_structure

* wrapper.c: added pointer conversion utility, updated bindings

* header_parser: basic parser completed

- updated code_generator's Func, Struct, Typedef to not use const, as it makes list unmodifiable
- added example from previous prototype

* header_parser: updated exception handling

- print stack trace of dart functions called via C

* cleanup

* Handled Disposing of pointers

- Updated dartdocs on clang_binding functions
- Added dispose methods in cx.. ..cursor,..type, ..string
- Updated parser code to handle disposing

* Added input_checker, passing compilerOpts to clang

* fix nullpointer error on compilerOpts list

* config_provider: added filtering for functions

- added new libclang example
- moved old example to example/simple

* updated lib-clang example to work with clang not in include path, restructured filters

* Multiple Changes

- Added sort method in code_generator library
- Sorted generated clang_bindings
- Added typedef declaration visitor for extracting typestring of typedef parameters
- Added new methods to wrapper.c for getting type declarations

* created a temporary logging solution

* added typedef struct parser

* cleanup

* restructured parsing

* restructured code

* header_parser: added struct parser, minor changes

- added type_extracter, moved typedeclaration_visitor into it
- cleanup imports

* parser: fixed type extraction

* updated libclang-example, minor fix for record_type spelling

* code_generator: added EnumClass, added test

* header_parser: added enum parsing

* header_parser: added typedef enums

* header_parser: extracted doxygen comment

- added clang_Cursor_getBriefCommentText_wrap in libclang bindings
- added comment extraction for functions, enums, structs

* Added args parsing and logging

- option for alternate yaml config
- added verbosity levels:[ all, fine, info, warning, severe ]

* config_provider: added output file option

* tests: added native_function_test

* code_generator: breaking change

- updated internal handling of type
- added support for NativeFunction type needed for creating typedefC
- updated tests, tool
- updated header_parser/type_extractor

* Added support for function pointers in bindings

- example/libclang-example: added clang_visitChildren_wrap method
- clang_bindings: added new methods getNumArgType, getArgType

* header_parser: changes, complete libclang_example

- completed libclang_example, all structs starting with CX
all functions starting with clang_ except those passing/returning structs b value
- added automatic typenaming to unnamed typedefs, unnamed function paramters

* minor changes

- code_generator: added factory Type constructor
- updated tests and code to prefer using factory type constructors
- added location methods to bindings

* header_parser: added support for multiple headers, header-filters

* header_parser: add structs used by functions

- updated example/libclang to auto add IndexerCallback struct
as its used in a function
- structs are now added automatically if they are returned or
passed in to a function (if struct type is parsed by extractor
it automatically adds it)

* config_provider: added sizemap for char, ints

* tests: added simple tests for functions

- test with supported types
- added automatic argument naming for functions in code_gen if they are null or only whitspaces
- code_gen_tests: only create debug file on test failure

* config_provider: added sort option to config

- updated libclang-example to use this
- renamed generated file names
- renamed package name for example

* example: added new cJSON example, bug fix

- new cJSON example, parsing a json and comparing it to json parsed by dart:convert
- bug fix: shouldInclude.* functions not adding to seen Set in includer

* linter bug fix, updated code for warnings

* minor changes

- documentation updates
- code_gen: wrong binding string type in metadata fix

* logging: improved log comments

* code_generator: removed global constants file

- lib prefixes are passed via writer now

* improved tests, minor changes

- improved header_parser_test
- updated named constructors of code_generator: Type

* header_parser: struct fields parsed

- all structs fields are excluded, if any field is struct by value or array
- added 2 functions using tool to clang bindings for parsing array element name, number
- updated libclang example
- added Broadtype.Array for code_gen (only used as indicator of type rn to exclude from parsing)

* parser: extract type from supported typdef

- Added config 'use-supported-typedefs'
- support for intptr_t
- updated test: native_function_bindings, added intptr_t

* header_parser: added workaround for array in struct

- updated libclang example

* fix: exclude array of struct in struct

* config_provider: revamped internal API

Reason: Multiple seperate things were added whenever a config spec was added to the tool
regarding-
1. validate yaml, detect unknown keys
2. tracking default value
3. extracting from map
4. Warning user if config is required and isnt provided

All this is now being managed at one place, A map of Spec is provided using
the getSpecs function, Spec has everything listed above
Adding a configuration is simply adding a Spec to the map returned by getSpecs
in config.dart

* Removed generated binding's dependency on package:ffi

- updated examples, tests
- changed Char_S type to Int8 (previously Utf8)

* temp: adding cmake

* WIP: cmakefile

* fix: loading dylib for other platforms

- updated validator, extractor
- updated examples
- config: changed libclang-dylib to libclang-dylib-folder

* Added build script for windows: tool/wrapped_libclang/build_dylib_windows

* Windows Testing

- Added build scripts(.bat files) for building dll for examples/tests
- Updated exmaple's input header globs to work on Windows (Globs on windows dont work if they start with "." or".."
- Used path package instead

* added warn about removing functions, minor changes

* config_provider: minor changes - sizemap, enum, comments

- Added enum to sizemap
- warn unknown subkeys in sizemap
- config option to disable dartdoc comment extraction

* fixed tests, minor changes

* Added warning for TU diagnostic errors

* header_parser: handling unimplemented types

- Added a BroadType.unimplemented, which can be used to determine and remove types
- Removing functions and struct members if unimplemented types are used (and show warning) (earlier exception was thrown)
- Added a flag to remove function with constant array passed as parameter

* update readme, comments

* update readme(s), linked todos to github issues

* added copyright notice to code files, update readme(s)

* Self hosting the tool: added libclang_config.yaml

- Removed clang_constants.dart file, all bindings are now in clang_bindings.dart

- Added build_dylib_macos script

* Replace platform based build scripts with build.dart

* Improved comments in accordance with styleguide

* config_provider: refactoring

* Added 3 lint rules

- prefer_final_fields
- prefer_final_locals
- prefer_final_in_for_each

* config_provider: made libclang_dylib_folder optional

User now has an option to provide a seperate folder containing the dynamic library
The default is to use the dynamic libraries in the tool/wrapped_libclang folder

* header_parser: removed file extension checks

User can provide any file path or glob matcher.
If clang can't parse a header, we print a severe log
and skip that header

* minor changes

* Updated warnings/error logs

* Minor changes

- Added test/native_function_test/build_test_dylib.dart
- Updated comments
- Added top level third_party folder
- Updated examples, test with fixed comments

* minor comment fix

* Updated preamble message

Preamble on generated files is now
/// AUTO GENERATED FILE, DO NOT EDIT.
///
/// Generated by `package:ffigen`.

* Updated build.dart, build_test_dylib.dart, Minor changes

- Added map of <String, Options> for storing default values for all platforms
- Using Options instead of global variables

* Minor changes

* minor refactoring

Co-authored-by: Daco Harkes <dacoharkes@google.com>
71 files changed
tree: 417e1b825fdc839e8d0fc253ace43ceab649bf5e
  1. bin/
  2. example/
  3. lib/
  4. test/
  5. third_party/
  6. tool/
  7. .gitignore
  8. .travis.yml
  9. analysis_options.yaml
  10. AUTHORS
  11. CHANGELOG.md
  12. LICENSE
  13. pubspec.yaml
  14. README.md
README.md

Build Status

Experimental binding generator for FFI bindings.

Example

For some header file example.h:

int sum(int a, int b);

Add configurations to Pubspec File:

ffigen:
  output: 'generated_bindings.dart'
  headers:
    - 'example.h'

Output (generated_bindings.dart).

DynamicLibrary _dylib;

/// Initialises dynamic library
void init(DynamicLibrary dylib) {
  _dylib = dylib;
}

int sum(int a,int b) {
  return _sum(a,b);
}

final _dart_sum _sum = _dylib.lookupFunction<_c_sum, _dart_sum>('sum');

typedef _c_sum = Int32 Function(Int32 a,Int32 b);

typedef _dart_sum = int Function(int a,int b);

Using this package

  • clone/download this repository.
  • Build it (see building).
  • Add this package as dev_dependency in your pubspec.yaml.
  • Configurations must be provided in the pubspec.yaml file under the key ffigen (or directly under a seperate yaml file which when u specify it passing --config filename when running the tool)
  • Run the tool- pub run ffigen:generate.

Building

A dynamic library for a wrapper to libclang needs to be generated as it is used by the parser submodule.

ubuntu/linux

  1. Install libclangdev - sudo apt-get install libclang-dev.
  2. cd tool/wrapped_libclang, then run dart build.dart.

Windows

  1. Install Visual Studio with C++ development support.
  2. Install LLVM.
  3. cd tool\wrapped_libclang, then run dart build.dart.

MacOS

  1. Install LLVM.
  2. cd tool/wrapped_libclang, then run dart build.dart.

Trying out examples

  1. cd examples/<example_u_want_to_run>, Run pub get.
  2. Run pub run ffigen:generate.

Running Tests

Dynamic library for some tests need to be built before running the examples.

  1. cd test/native_functions_test.
  2. Run dart build_test_dylib.dart.

Run tests from the root of the package with pub run test.