Add expression evaluation to ddc for google3
- Move expression evaluation to ddc in preparation for google3
- Added server to ddc to handle update and compileExpression requests
- Added tests
- Added 'experimental-output-compiled-kernel' option to ddc to generate
full kernel files only for compiled libraries, and store with
'.full.dill' extension
- Added AssetFileSystem to communicate to the asset server in the
debugger
- Made expression_compiler_worker work with full kernel files,
so removed invalidation of current file to improve performance
- Made expression_compiler_worker reuse already loaded imports
to avoid reading them from source in the incremental compiler
- Updated tests to work with DDC (for simulating webdev)
- Disabled tests that work with bazel kernel worker for now
as it does not generate full dill files yet
- Addressed code review comments from the prototype version:
https://dart-review.googlesource.com/c/sdk/+/157005
Details:
Currently, in flutter tools, expression evaluation is supported via
expression compilation, which is done by the incremental compiler in
the frontend server. The same incremental compiler is used for initial
application compilation, incremental code compilation for hot reload,
and any number of expression compilation requests.
In google3, the apps are typically too large to be compiled as a whole
in memory by the frontend server. Build in google3 is currently done by
blaze, as a distributed build using a task dependency graph. Build tasks
output kernel outline files as an interface between components produced
by individual tasks.
We are proposing an implementation of the expression compilation in
google3 that is taking advantage of full kernel files produced by the
build (supporting build changes to follow). This change introduces a
small server based on dev_compiler, which can handle following requests:
- update: load full kernel for given modules (done on app start)
- compileExpression: compile expression in a given library and module
(done when paused on a breakpoint)
Expression compilation uses previously loaded kernel files for the
application component and its dependencies to compile an expression.
Change-Id: Icf73868069faf3a2eb6d43ba78e459f8457e9e35
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/160944
Reviewed-by: Nicholas Shahan <nshahan@google.com>
Reviewed-by: Gary Roumanis <grouma@google.com>
Reviewed-by: Jens Johansen <jensj@google.com>
Reviewed-by: Jake Macdonald <jakemac@google.com>
Commit-Queue: Anna Gringauze <annagrin@google.com>
diff --git a/pkg/dev_compiler/bin/dartdevc.dart b/pkg/dev_compiler/bin/dartdevc.dart
index a906e00..68246e3 100755
--- a/pkg/dev_compiler/bin/dartdevc.dart
+++ b/pkg/dev_compiler/bin/dartdevc.dart
@@ -12,6 +12,7 @@
import 'dart:isolate';
import 'package:bazel_worker/bazel_worker.dart';
import 'package:dev_compiler/src/compiler/shared_command.dart';
+import 'package:dev_compiler/src/kernel/expression_compiler_worker.dart';
/// The entry point for the Dart Dev Compiler.
///
@@ -28,6 +29,18 @@
await _CompilerWorker(parsedArgs, workerConnection).run();
} else if (parsedArgs.isBatch) {
await runBatch(parsedArgs);
+ } else if (parsedArgs.isExpressionCompiler) {
+ ExpressionCompilerWorker worker;
+ if (sendPort != null) {
+ var receivePort = ReceivePort();
+ sendPort.send(receivePort.sendPort);
+ worker = await ExpressionCompilerWorker.createFromArgs(parsedArgs.rest,
+ requestStream: receivePort.cast<Map<String, dynamic>>(),
+ sendResponse: sendPort.send);
+ } else {
+ worker = await ExpressionCompilerWorker.createFromArgs(parsedArgs.rest);
+ }
+ await worker.start();
} else {
var result = await compile(parsedArgs);
exitCode = result.exitCode;
diff --git a/pkg/dev_compiler/lib/dev_compiler.dart b/pkg/dev_compiler/lib/dev_compiler.dart
index dedfc72..ca0a8bd 100644
--- a/pkg/dev_compiler/lib/dev_compiler.dart
+++ b/pkg/dev_compiler/lib/dev_compiler.dart
@@ -8,4 +8,5 @@
export 'src/compiler/shared_command.dart' show SharedCompilerOptions;
export 'src/kernel/command.dart' show jsProgramToCode;
export 'src/kernel/compiler.dart' show ProgramCompiler;
+export 'src/kernel/expression_compiler.dart' show ExpressionCompiler;
export 'src/kernel/target.dart' show DevCompilerTarget;
diff --git a/pkg/dev_compiler/lib/src/compiler/shared_command.dart b/pkg/dev_compiler/lib/src/compiler/shared_command.dart
index 2761eb0..358f238 100644
--- a/pkg/dev_compiler/lib/src/compiler/shared_command.dart
+++ b/pkg/dev_compiler/lib/src/compiler/shared_command.dart
@@ -61,6 +61,14 @@
/// runtime can enable synchronous stack trace deobsfuscation.
final bool inlineSourceMap;
+ /// Whether to emit the full compiled kernel.
+ ///
+ /// This is used by expression compiler worker, launched from the debugger
+ /// in webdev and google3 scenarios, for expression evaluation features.
+ /// Full kernel for compiled files is needed to be able to compile
+ /// expressions on demand in the current scope of a breakpoint.
+ final bool emitFullCompiledKernel;
+
/// Whether to emit a summary file containing API signatures.
///
/// This is required for a modular build process.
@@ -113,6 +121,7 @@
this.enableAsserts = true,
this.replCompile = false,
this.emitDebugMetadata = false,
+ this.emitFullCompiledKernel = false,
this.summaryModules = const {},
this.moduleFormats = const [],
this.moduleName,
@@ -129,6 +138,8 @@
enableAsserts: args['enable-asserts'] as bool,
replCompile: args['repl-compile'] as bool,
emitDebugMetadata: args['experimental-emit-debug-metadata'] as bool,
+ emitFullCompiledKernel:
+ args['experimental-output-compiled-kernel'] as bool,
summaryModules:
_parseCustomSummaryModules(args['summary'] as List<String>),
moduleFormats: parseModuleFormatOption(args),
@@ -181,6 +192,12 @@
help: 'Experimental option for compiler development.\n'
'Output a metadata file for debug tools next to the .js output.',
defaultsTo: false,
+ hide: true)
+ ..addFlag('experimental-output-compiled-kernel',
+ help: 'Experimental option for compiler development.\n'
+ 'Output a full kernel file for currently compiled module next to '
+ 'the .js output.',
+ defaultsTo: false,
hide: true);
}
@@ -490,6 +507,17 @@
/// See also [isBatchOrWorker].
final bool isBatch;
+ /// Whether to run in `--experimental-expression-compiler` mode.
+ ///
+ /// This is a special mode that is optimized for only compiling expressions.
+ ///
+ /// All dependencies must come from precompiled dill files, and those must
+ /// be explicitly invalidated as needed between expression compile requests.
+ /// Invalidation of dill is performed using [updateDeps] from the client (i.e.
+ /// debugger) and should be done every time a dill file changes, for example,
+ /// on hot reload or rebuild.
+ final bool isExpressionCompiler;
+
/// Whether to run in `--bazel_worker` mode, e.g. for Bazel builds.
///
/// Similar to [isBatch] but with a different protocol.
@@ -507,11 +535,14 @@
/// Note that this only makes sense when also reusing results.
final bool useIncrementalCompiler;
- ParsedArguments._(this.rest,
- {this.isBatch = false,
- this.isWorker = false,
- this.reuseResult = false,
- this.useIncrementalCompiler = false});
+ ParsedArguments._(
+ this.rest, {
+ this.isBatch = false,
+ this.isWorker = false,
+ this.reuseResult = false,
+ this.useIncrementalCompiler = false,
+ this.isExpressionCompiler = false,
+ });
/// Preprocess arguments to determine whether DDK is used in batch mode or as a
/// persistent worker.
@@ -530,6 +561,7 @@
var isBatch = false;
var reuseResult = false;
var useIncrementalCompiler = false;
+ var isExpressionCompiler = false;
Iterable<String> argsToParse = args;
@@ -548,6 +580,8 @@
reuseResult = true;
} else if (arg == '--use-incremental-compiler') {
useIncrementalCompiler = true;
+ } else if (arg == '--experimental-expression-compiler') {
+ isExpressionCompiler = true;
} else {
newArgs.add(arg);
}
@@ -556,7 +590,8 @@
isWorker: isWorker,
isBatch: isBatch,
reuseResult: reuseResult,
- useIncrementalCompiler: useIncrementalCompiler);
+ useIncrementalCompiler: useIncrementalCompiler,
+ isExpressionCompiler: isExpressionCompiler);
}
/// Whether the compiler is running in [isBatch] or [isWorker] mode.
diff --git a/pkg/dev_compiler/lib/src/kernel/command.dart b/pkg/dev_compiler/lib/src/kernel/command.dart
index ba80022..6598642 100644
--- a/pkg/dev_compiler/lib/src/kernel/command.dart
+++ b/pkg/dev_compiler/lib/src/kernel/command.dart
@@ -361,7 +361,7 @@
'the --summarize option is not supported.');
return CompilerResult(64);
}
- // TODO(jmesserly): CFE mutates the Kernel tree, so we can't save the dill
+ // Note: CFE mutates the Kernel tree, so we can't save the dill
// file if we successfully reused a cached library. If compiler state is
// unchanged, it means we used the cache.
//
@@ -376,6 +376,27 @@
kernel.BinaryPrinter(sink).writeComponentFile(component);
outFiles.add(sink.flush().then((_) => sink.close()));
}
+ if (argResults['experimental-output-compiled-kernel'] as bool) {
+ if (outPaths.length > 1) {
+ print(
+ 'If multiple output files (found ${outPaths.length}) are specified, '
+ 'the --experimental-output-compiled-kernel option is not supported.');
+ return CompilerResult(64);
+ }
+ // Note: CFE mutates the Kernel tree, so we can't save the dill
+ // file if we successfully reused a cached library. If compiler state is
+ // unchanged, it means we used the cache.
+ //
+ // In that case, we need to unbind canonical names, because they could be
+ // bound already from the previous compile.
+ if (identical(compilerState, oldCompilerState)) {
+ compiledLibraries.unbindCanonicalNames();
+ }
+ var sink =
+ File(p.withoutExtension(outPaths.first) + '.full.dill').openWrite();
+ kernel.BinaryPrinter(sink).writeComponentFile(compiledLibraries);
+ outFiles.add(sink.flush().then((_) => sink.close()));
+ }
if (argResults['summarize-text'] as bool) {
if (outPaths.length > 1) {
print(
diff --git a/pkg/dev_compiler/lib/src/kernel/compiler.dart b/pkg/dev_compiler/lib/src/kernel/compiler.dart
index a040dc9..96575f1 100644
--- a/pkg/dev_compiler/lib/src/kernel/compiler.dart
+++ b/pkg/dev_compiler/lib/src/kernel/compiler.dart
@@ -2930,11 +2930,16 @@
_staticTypeContext.enterLibrary(_currentLibrary);
_currentClass = cls;
- // emit function with additional information,
- // such as types that are used in the expression
+ // Emit function with additional information, such as types that are used
+ // in the expression. Note that typeTable can be null if this function is
+ // called from the expression compilation service, since we currently do
+ // not optimize for size of generated javascript in that scenario.
+ // TODO: figure whether or when optimizing for build time vs JavaScript
+ // size on expression evaluation is better.
+ // Issue: https://github.com/dart-lang/sdk/issues/43288
var fun = _emitFunction(functionNode, name);
- var items = _typeTable.discharge();
- var body = js_ast.Block([...items, ...fun.body.statements]);
+ var items = _typeTable?.discharge();
+ var body = js_ast.Block([...?items, ...fun.body.statements]);
return js_ast.Fun(fun.params, body);
}
@@ -3450,7 +3455,7 @@
if (fileUri == null) return null;
try {
var loc = _component.getLocation(fileUri, offset);
- if (loc == null) return null;
+ if (loc == null || loc.line < 0) return null;
return SourceLocation(offset,
sourceUrl: fileUri, line: loc.line - 1, column: loc.column - 1);
} on StateError catch (_) {
diff --git a/pkg/frontend_server/lib/src/expression_compiler.dart b/pkg/dev_compiler/lib/src/kernel/expression_compiler.dart
similarity index 94%
rename from pkg/frontend_server/lib/src/expression_compiler.dart
rename to pkg/dev_compiler/lib/src/kernel/expression_compiler.dart
index a64a904..2b26a3f 100644
--- a/pkg/frontend_server/lib/src/expression_compiler.dart
+++ b/pkg/dev_compiler/lib/src/kernel/expression_compiler.dart
@@ -1,6 +1,6 @@
-// Copyright 2020 The Chromium Authors. All rights reserved.
-// Use of this source code is governed by a BSD-style license that can be
-// found in the LICENSE file.
+// Copyright (c) 2020, the Dart project authors. Please see the AUTHORS file
+// for details. All rights reserved. Use of this source code is governed by a
+// BSD-style license that can be found in the LICENSE file.
import 'dart:async';
import 'dart:io';
@@ -210,6 +210,7 @@
final IncrementalCompiler _compiler;
final ProgramCompiler _kernel2jsCompiler;
final Component _component;
+ final List<String> errors;
DiagnosticMessageHandler onDiagnostic;
void _log(String message) {
@@ -222,7 +223,7 @@
}
ExpressionCompiler(this._compiler, this._kernel2jsCompiler, this._component,
- {this.verbose, this.onDiagnostic});
+ {this.verbose, this.onDiagnostic, this.errors});
/// Compiles [expression] in [libraryUri] at [line]:[column] to JavaScript
/// in [moduleName].
@@ -425,11 +426,15 @@
scope.cls?.name,
scope.procedure.isStatic);
- // TODO(annagrin): The condition below seems to be always false.
- // Errors are still correctly reported in the frontent_server,
- // but we end up doing unnesessary work below.
- // Add communication of error state from compiler here.
- if (_compiler.context.errors.length > 0) {
+ // TODO: make this code clear and assumptions enforceable
+ // https://github.com/dart-lang/sdk/issues/43273
+ //
+ // We assume here that ExpressionCompiler is always created using
+ // onDisgnostic method that adds to the error list that is passed
+ // to the same invocation of the ExpressionCompiler constructor.
+ // We only use the error list once - below, to detect if the frontend
+ // compilation of the expression has failed.
+ if (errors.isNotEmpty) {
return null;
}
diff --git a/pkg/dev_compiler/lib/src/kernel/expression_compiler_worker.dart b/pkg/dev_compiler/lib/src/kernel/expression_compiler_worker.dart
new file mode 100644
index 0000000..9cb61dc
--- /dev/null
+++ b/pkg/dev_compiler/lib/src/kernel/expression_compiler_worker.dart
@@ -0,0 +1,473 @@
+// Copyright (c) 2020, the Dart project authors. Please see the AUTHORS file
+// for details. All rights reserved. Use of this source code is governed by a
+// BSD-style license that can be found in the LICENSE file.
+
+import 'dart:async';
+import 'dart:convert';
+import 'dart:io';
+
+import 'package:_fe_analyzer_shared/src/messages/diagnostic_message.dart';
+import 'package:_fe_analyzer_shared/src/messages/severity.dart';
+import 'package:args/args.dart';
+import 'package:build_integration/file_system/multi_root.dart';
+import 'package:front_end/src/api_prototype/compiler_options.dart';
+import 'package:front_end/src/api_prototype/experimental_flags.dart';
+import 'package:front_end/src/api_prototype/file_system.dart';
+import 'package:front_end/src/api_unstable/ddc.dart';
+import 'package:kernel/ast.dart' show Component, Library;
+import 'package:kernel/target/targets.dart' show TargetFlags;
+import 'package:meta/meta.dart';
+import 'package:vm/http_filesystem.dart';
+
+import '../compiler/js_names.dart';
+import '../compiler/shared_command.dart';
+import 'command.dart';
+import 'compiler.dart';
+import 'expression_compiler.dart';
+import 'target.dart';
+
+/// A wrapper around asset server that redirects file read requests
+/// to http get requests to the asset server.
+class AssetFileSystem extends HttpAwareFileSystem {
+ final String server;
+ final String port;
+
+ AssetFileSystem(FileSystem original, this.server, this.port)
+ : super(original);
+
+ Uri resourceUri(Uri uri) =>
+ Uri.parse('http://$server:$port/getResource?uri=${uri.toString()}');
+
+ @override
+ FileSystemEntity entityForUri(Uri uri) {
+ if (uri.scheme == 'file') {
+ return super.entityForUri(uri);
+ }
+
+ // pass the uri to the asset server in the debugger
+ return HttpFileSystemEntity(this, resourceUri(uri));
+ }
+}
+
+/// The service that handles expression compilation requests from
+/// the debugger.
+///
+/// See design documentation and discussion in
+/// http://go/dart_expression_evaluation_webdev_google3
+///
+///
+/// [ExpressionCompilerWorker] listens to input stream of compile expression
+/// requests and outputs responses.
+///
+/// Debugger can run the service by running the dartdevc main in an isolate,
+/// which sets up the request stream and response callback using send ports.
+///
+/// Debugger also can pass an asset server's host and port so the service
+/// can read dill files from the [AssetFileSystem] that talks to the asset
+/// server behind the scenes over http.
+///
+/// Protocol:
+///
+/// - debugger and dartdevc expression evaluation service perform the initial
+/// handshake to establish two-way communication:
+///
+/// - debugger creates an isolate using dartdevc's main method with
+/// '--experimental-expression-compiler' flag and passes a send port
+/// to dartdevc for sending responces from the service to the debugger.
+///
+/// - dartdevc creates a new send port to receive requests on, and sends
+/// it back to the debugger, for sending requests to the service.
+///
+/// - debugger can now send two types of requests to the dartdevc service.
+/// The service handles the requests sequentially in a first come, first
+/// serve order.
+///
+/// - [UpdateDepsRequest]:
+/// This request is sent on (re-)build, making the dartdevc load all
+/// newly built full kernel files for given modules.
+///
+/// - [CompileExpressionRequest]:
+/// This request is sent any time an evaluateInFrame request is made to
+/// the debugger's VM service at a breakpoint - for example, on typing
+/// in an expression evaluation box, on hover over, evaluation of
+/// conditional breakpoints, evaluation of expressions in a watch window.
+///
+/// - Debugger closes the requests stream on exit, which effectively stops
+/// the service
+class ExpressionCompilerWorker {
+ final Stream<Map<String, dynamic>> requestStream;
+ final void Function(Map<String, dynamic>) sendResponse;
+
+ final _libraryForUri = <Uri, Library>{};
+ final _componentForLibrary = <Library, Component>{};
+ final _componentForModuleName = <String, Component>{};
+ final _componentModuleNames = <Component, String>{};
+ final ProcessedOptions _processedOptions;
+ final Component _sdkComponent;
+
+ ExpressionCompilerWorker._(this._processedOptions, this._sdkComponent,
+ this.requestStream, this.sendResponse);
+
+ static Future<ExpressionCompilerWorker> createFromArgs(
+ List<String> args, {
+ Stream<Map<String, dynamic>> requestStream,
+ void Function(Map<String, dynamic>) sendResponse,
+ }) {
+ // We are destructive on `args`, so make a copy.
+ args = args.toList();
+ var environmentDefines = parseAndRemoveDeclaredVariables(args);
+ var parsedArgs = argParser.parse(args);
+
+ FileSystem fileSystem = StandardFileSystem.instance;
+ var multiRoots = (parsedArgs['multi-root'] as Iterable<String>)
+ .map(Uri.base.resolve)
+ .toList();
+ var multiRootScheme = parsedArgs['multi-root-scheme'] as String;
+ if (multiRoots.isNotEmpty) {
+ fileSystem = MultiRootFileSystem(multiRootScheme, multiRoots, fileSystem);
+ }
+ var assetServerAddress = parsedArgs['asset-server-address'] as String;
+ if (assetServerAddress != null) {
+ var assetServerPort = parsedArgs['asset-server-port'] as String;
+ fileSystem = AssetFileSystem(
+ fileSystem, assetServerAddress, assetServerPort ?? '8080');
+ }
+ var experimentalFlags = parseExperimentalFlags(
+ parseExperimentalArguments(
+ parsedArgs['enable-experiment'] as List<String>),
+ onError: (e) => throw e);
+ return create(
+ librariesSpecificationUri:
+ _argToUri(parsedArgs['libraries-file'] as String),
+ packagesFile: _argToUri(parsedArgs['packages-file'] as String),
+ sdkSummary: _argToUri(parsedArgs['dart-sdk-summary'] as String),
+ fileSystem: fileSystem,
+ environmentDefines: environmentDefines,
+ experimentalFlags: experimentalFlags,
+ sdkRoot: _argToUri(parsedArgs['sdk-root'] as String),
+ trackWidgetCreation: parsedArgs['track-widget-creation'] as bool,
+ soundNullSafety: parsedArgs['sound-null-safety'] as bool,
+ verbose: parsedArgs['verbose'] as bool,
+ requestStream: requestStream,
+ sendResponse: sendResponse,
+ );
+ }
+
+ static List<String> errors = <String>[];
+ static List<String> warnings = <String>[];
+
+ /// Create the worker and load the sdk outlines.
+ static Future<ExpressionCompilerWorker> create({
+ @required Uri librariesSpecificationUri,
+ @required Uri sdkSummary,
+ @required FileSystem fileSystem,
+ Uri packagesFile,
+ Map<String, String> environmentDefines = const {},
+ Map<ExperimentalFlag, bool> experimentalFlags = const {},
+ Uri sdkRoot,
+ bool trackWidgetCreation = false,
+ bool soundNullSafety = false,
+ bool verbose = false,
+ Stream<Map<String, dynamic>> requestStream, // Defaults to read from stdin
+ void Function(Map<String, dynamic>)
+ sendResponse, // Defaults to write to stdout
+ }) async {
+ var options = CompilerOptions()
+ ..compileSdk = false
+ ..sdkRoot = sdkRoot
+ ..sdkSummary = sdkSummary
+ ..packagesFileUri = packagesFile
+ ..librariesSpecificationUri = librariesSpecificationUri
+ ..target = DevCompilerTarget(
+ TargetFlags(trackWidgetCreation: trackWidgetCreation))
+ ..fileSystem = fileSystem
+ ..omitPlatform = true
+ ..environmentDefines = environmentDefines
+ ..experimentalFlags = experimentalFlags
+ ..onDiagnostic = _onDiagnosticHandler(errors, warnings)
+ ..nnbdMode = soundNullSafety ? NnbdMode.Strong : NnbdMode.Weak
+ ..verbose = verbose;
+ requestStream ??= stdin
+ .transform(utf8.decoder.fuse(json.decoder))
+ .cast<Map<String, dynamic>>();
+ sendResponse ??= (Map<String, dynamic> response) =>
+ stdout.writeln(json.encode(response));
+ var processedOpts = ProcessedOptions(options: options);
+
+ var sdkComponent = await CompilerContext(processedOpts)
+ .runInContext<Component>((CompilerContext c) async {
+ return processedOpts.loadSdkSummary(null);
+ });
+
+ return ExpressionCompilerWorker._(
+ processedOpts, sdkComponent, requestStream, sendResponse)
+ .._update(sdkComponent, dartSdkModule);
+ }
+
+ /// Starts listening and responding to commands.
+ ///
+ /// Completes when the [requestStream] closes and we finish handling the
+ /// requests.
+ Future<void> start() async {
+ await for (var request in requestStream) {
+ try {
+ var command = request['command'] as String;
+ switch (command) {
+ case 'UpdateDeps':
+ sendResponse(
+ await _updateDeps(UpdateDepsRequest.fromJson(request)));
+ break;
+ case 'CompileExpression':
+ sendResponse(await _compileExpression(
+ CompileExpressionRequest.fromJson(request)));
+ break;
+ default:
+ throw ArgumentError(
+ 'Unrecognized command `$command`, full request was `$request`');
+ }
+ } catch (e, s) {
+ sendResponse({
+ 'exception': '$e',
+ 'stackTrace': '$s',
+ 'succeeded': false,
+ });
+ }
+ }
+ }
+
+ /// Handles a `CompileExpression` request.
+ Future<Map<String, dynamic>> _compileExpression(
+ CompileExpressionRequest request) async {
+ var libraryUri = Uri.parse(request.libraryUri);
+ if (libraryUri.scheme == 'dart') {
+ // compiling expressions inside the SDK currently fails because
+ // SDK kernel outlines do not contain information that is needed
+ // to detect the scope for expression evaluation - such as local
+ // symbols and source file line starts.
+ throw Exception('Expression compilation inside SDK is not supported yet');
+ }
+
+ var originalComponent = _componentForModuleName[request.moduleName];
+ if (originalComponent == null) {
+ throw ArgumentError(
+ 'Unable to find library `$libraryUri`, it must be loaded first.');
+ }
+
+ var component = _sdkComponent;
+
+ if (libraryUri.scheme != 'dart') {
+ var libraries =
+ _collectTransitiveDependencies(originalComponent, _sdkComponent);
+ component = Component(
+ libraries: libraries,
+ nameRoot: originalComponent.root,
+ uriToSource: originalComponent.uriToSource,
+ );
+ }
+
+ errors.clear();
+ warnings.clear();
+
+ var incrementalCompiler = IncrementalCompiler.forExpressionCompilationOnly(
+ CompilerContext(_processedOptions), component);
+
+ var finalComponent =
+ await incrementalCompiler.computeDelta(entryPoints: [libraryUri]);
+ finalComponent.computeCanonicalNames();
+
+ if (errors.isNotEmpty) {
+ return {
+ 'errors': errors,
+ 'warnings': warnings,
+ 'compiledProcedure': null,
+ 'succeeded': errors.isEmpty,
+ };
+ }
+
+ var compiler = ProgramCompiler(
+ finalComponent,
+ incrementalCompiler.getClassHierarchy(),
+ SharedCompilerOptions(
+ sourceMap: true, summarizeApi: false, moduleName: request.moduleName),
+ _componentForLibrary,
+ _componentModuleNames,
+ coreTypes: incrementalCompiler.getCoreTypes(),
+ );
+
+ var expressionCompiler = ExpressionCompiler(
+ incrementalCompiler,
+ compiler,
+ finalComponent,
+ verbose: _processedOptions.verbose,
+ onDiagnostic: _onDiagnosticHandler(errors, warnings),
+ errors: errors,
+ );
+
+ var compiledProcedure = await expressionCompiler.compileExpressionToJs(
+ request.libraryUri,
+ request.line,
+ request.column,
+ request.jsModules,
+ request.jsScope,
+ request.moduleName,
+ request.expression);
+
+ return {
+ 'errors': errors,
+ 'warnings': warnings,
+ 'compiledProcedure': compiledProcedure,
+ 'succeeded': errors.isEmpty,
+ };
+ }
+
+ List<Library> _collectTransitiveDependencies(
+ Component component, Component sdk) {
+ var libraries = <Library>{};
+ libraries.addAll(sdk.libraries);
+
+ var toVisit = <Library>[];
+ toVisit.addAll(component.libraries);
+
+ while (toVisit.isNotEmpty) {
+ var lib = toVisit.removeLast();
+ if (!libraries.contains(lib)) {
+ libraries.add(lib);
+
+ for (var dep in lib.dependencies) {
+ var uri = dep.importedLibraryReference.asLibrary.importUri;
+ var library = _libraryForUri[uri];
+ assert(library == dep.importedLibraryReference.asLibrary);
+ toVisit.add(library);
+ }
+ }
+ }
+
+ return libraries.toList();
+ }
+
+ /// Loads in the specified dill files and invalidates any existing ones.
+ Future<Map<String, dynamic>> _updateDeps(UpdateDepsRequest request) async {
+ for (var input in request.inputs) {
+ var file =
+ _processedOptions.fileSystem.entityForUri(Uri.parse(input.path));
+ var bytes = await file.readAsBytes();
+ var component = await _processedOptions.loadComponent(
+ bytes, _sdkComponent.root,
+ alwaysCreateNewNamedNodes: true);
+ _update(component, input.moduleName);
+ }
+ return {'succeeded': true};
+ }
+
+ void _update(Component component, String moduleName) {
+ // do not update dart sdk
+ if (moduleName == dartSdkModule &&
+ _componentForModuleName.containsKey(moduleName)) {
+ return;
+ }
+
+ // cleanup old components and libraries
+ if (_componentForModuleName.containsKey(moduleName)) {
+ var oldComponent = _componentForModuleName[moduleName];
+ for (var lib in oldComponent.libraries) {
+ _componentForLibrary.remove(lib);
+ _libraryForUri.remove(lib.importUri);
+ }
+ _componentModuleNames.remove(oldComponent);
+ _componentForModuleName.remove(moduleName);
+ }
+
+ // add new components and libraries
+ _componentModuleNames[component] = moduleName;
+ _componentForModuleName[moduleName] = component;
+ for (var lib in component.libraries) {
+ _componentForLibrary[lib] = component;
+ _libraryForUri[lib.importUri] = lib;
+ }
+ }
+}
+
+class CompileExpressionRequest {
+ final int column;
+ final String expression;
+ final Map<String, String> jsModules;
+ final Map<String, String> jsScope;
+ final String libraryUri;
+ final int line;
+ final String moduleName;
+
+ CompileExpressionRequest({
+ @required this.expression,
+ @required this.column,
+ @required this.jsModules,
+ @required this.jsScope,
+ @required this.libraryUri,
+ @required this.line,
+ @required this.moduleName,
+ });
+
+ factory CompileExpressionRequest.fromJson(Map<String, dynamic> json) =>
+ CompileExpressionRequest(
+ expression: json['expression'] as String,
+ line: json['line'] as int,
+ column: json['column'] as int,
+ jsModules: Map<String, String>.from(json['jsModules'] as Map),
+ jsScope: Map<String, String>.from(json['jsScope'] as Map),
+ libraryUri: json['libraryUri'] as String,
+ moduleName: json['moduleName'] as String,
+ );
+}
+
+class UpdateDepsRequest {
+ final List<InputDill> inputs;
+
+ UpdateDepsRequest(this.inputs);
+
+ factory UpdateDepsRequest.fromJson(Map<String, dynamic> json) =>
+ UpdateDepsRequest([
+ for (var input in json['inputs'] as List)
+ InputDill(input['path'] as String, input['moduleName'] as String),
+ ]);
+}
+
+class InputDill {
+ final String moduleName;
+ final String path;
+
+ InputDill(this.path, this.moduleName);
+}
+
+void Function(DiagnosticMessage) _onDiagnosticHandler(
+ List<String> errors, List<String> warnings) =>
+ (DiagnosticMessage message) {
+ switch (message.severity) {
+ case Severity.error:
+ case Severity.internalProblem:
+ errors.add(message.plainTextFormatted.join('\n'));
+ break;
+ case Severity.warning:
+ warnings.add(message.plainTextFormatted.join('\n'));
+ break;
+ case Severity.context:
+ case Severity.ignored:
+ throw 'Unexpected severity: ${message.severity}';
+ }
+ };
+
+final argParser = ArgParser()
+ ..addOption('dart-sdk-summary')
+ ..addMultiOption('enable-experiment',
+ help: 'Enable a language experiment when invoking the CFE.')
+ ..addOption('libraries-file')
+ ..addMultiOption('multi-root')
+ ..addOption('multi-root-scheme', defaultsTo: 'org-dartlang-app')
+ ..addOption('packages-file')
+ ..addOption('sdk-root')
+ ..addOption('asset-server-address')
+ ..addOption('asset-server-port')
+ ..addFlag('track-widget-creation', defaultsTo: false)
+ ..addFlag('sound-null-safety', defaultsTo: false)
+ ..addFlag('verbose', defaultsTo: false);
+
+Uri _argToUri(String uriArg) =>
+ uriArg == null ? null : Uri.base.resolve(uriArg.replaceAll('\\', '/'));
diff --git a/pkg/dev_compiler/pubspec.yaml b/pkg/dev_compiler/pubspec.yaml
index 48ed14f..73cdcc9 100644
--- a/pkg/dev_compiler/pubspec.yaml
+++ b/pkg/dev_compiler/pubspec.yaml
@@ -23,6 +23,8 @@
path: any
source_maps: any
source_span: any
+ vm:
+ path: ../vm
dev_dependencies:
analyzer: any
diff --git a/pkg/frontend_server/test/src/expression_compiler_test.dart b/pkg/dev_compiler/test/expression_compiler/expression_compiler_test.dart
similarity index 95%
rename from pkg/frontend_server/test/src/expression_compiler_test.dart
rename to pkg/dev_compiler/test/expression_compiler/expression_compiler_test.dart
index 2dabd91..7374f49 100644
--- a/pkg/frontend_server/test/src/expression_compiler_test.dart
+++ b/pkg/dev_compiler/test/expression_compiler/expression_compiler_test.dart
@@ -1,6 +1,6 @@
-// Copyright 2020 The Chromium Authors. All rights reserved.
-// Use of this source code is governed by a BSD-style license that can be
-// found in the LICENSE file.import 'dart:io' show Platform, File;
+// Copyright (c) 2020, the Dart project authors. Please see the AUTHORS file
+// for details. All rights reserved. Use of this source code is governed by a
+// BSD-style license that can be found in the LICENSE file.
import 'dart:io' show Directory, File;
@@ -11,7 +11,6 @@
show CompilerOptions;
import 'package:front_end/src/compute_platform_binaries_location.dart';
import 'package:front_end/src/fasta/incremental_serializer.dart';
-import 'package:frontend_server/src/expression_compiler.dart';
import 'package:kernel/ast.dart' show Component;
import 'package:kernel/target/targets.dart';
import 'package:path/path.dart' as p;
@@ -29,8 +28,8 @@
bool outlineOnly,
IncrementalSerializer incrementalSerializer])
: super(
- new CompilerContext(
- new ProcessedOptions(options: options, inputs: [entryPoint])),
+ CompilerContext(
+ ProcessedOptions(options: options, inputs: [entryPoint])),
initializeFrom,
outlineOnly,
incrementalSerializer);
@@ -39,8 +38,8 @@
this.entryPoint, Component componentToInitializeFrom,
[bool outlineOnly, IncrementalSerializer incrementalSerializer])
: super.fromComponent(
- new CompilerContext(
- new ProcessedOptions(options: options, inputs: [entryPoint])),
+ CompilerContext(
+ ProcessedOptions(options: options, inputs: [entryPoint])),
componentToInitializeFrom,
outlineOnly,
incrementalSerializer);
@@ -104,6 +103,7 @@
String get package => importUri.toString();
String get file => fileUri.path;
+ @override
String toString() =>
'Name: $name, File: $file, Package: $package, path: $path';
}
@@ -140,13 +140,13 @@
kernel2jsCompiler.emitModule(component);
// create expression compiler
- var evaluator = new ExpressionCompiler(
- compiler, kernel2jsCompiler, component,
+ var evaluator = ExpressionCompiler(compiler, kernel2jsCompiler, component,
verbose: setup.options.verbose,
- onDiagnostic: setup.options.onDiagnostic);
+ onDiagnostic: setup.options.onDiagnostic,
+ errors: setup.errors);
// collect all module names and paths
- Map<Uri, Module> moduleInfo = _collectModules(component);
+ var moduleInfo = _collectModules(component);
var modules =
moduleInfo.map((k, v) => MapEntry<String, String>(v.name, v.path));
@@ -161,7 +161,7 @@
var jsExpression = await evaluator.compileExpressionToJs(
module.package, line, column, modules, scope, module.name, expression);
- if (setup.errors.length > 0) {
+ if (setup.errors.isNotEmpty) {
jsExpression = setup.errors.toString().replaceAll(
RegExp(
r'org-dartlang-debug:synthetic_debug_expression:[0-9]*:[0-9]*:'),
@@ -174,7 +174,7 @@
}
Map<Uri, Module> _collectModules(Component component) {
- Map<Uri, Module> modules = <Uri, Module>{};
+ var modules = <Uri, Module>{};
for (var library in component.libraries) {
modules[library.fileUri] = Module(library.importUri, library.fileUri);
}
@@ -239,10 +239,10 @@
}
int _getEvaluationLine(String source) {
- RegExp placeholderRegExp = RegExp(r'/\* evaluation placeholder \*/');
+ var placeholderRegExp = RegExp(r'/\* evaluation placeholder \*/');
var lines = source.split('\n');
- for (int line = 0; line < lines.length; line++) {
+ for (var line = 0; line < lines.length; line++) {
var content = lines[line];
if (placeholderRegExp.firstMatch(content) != null) {
return line + 1;
@@ -253,10 +253,10 @@
}
void main() {
- SetupCompilerOptions options = SetupCompilerOptions();
+ var options = SetupCompilerOptions();
group('Expression compiler tests in extension method:', () {
- const String source = '''
+ const source = '''
extension NumberParsing on String {
int parseInt() {
var ret = int.parse(this);
@@ -306,7 +306,7 @@
});
group('Expression compiler tests in method:', () {
- const String source = '''
+ const source = '''
extension NumberParsing on String {
int parseInt() {
return int.parse(this);
@@ -559,7 +559,7 @@
});
group('Expression compiler tests in method with no field access:', () {
- const String source = '''
+ const source = '''
extension NumberParsing on String {
int parseInt() {
return int.parse(this);
@@ -720,7 +720,7 @@
});
group('Expression compiler tests in async method:', () {
- const String source = '''
+ const source = '''
class C {
C(int this.field, int this._field);
@@ -780,7 +780,7 @@
});
group('Expression compiler tests in global function:', () {
- const String source = '''
+ const source = '''
extension NumberParsing on String {
int parseInt() {
return int.parse(this);
@@ -1038,7 +1038,7 @@
});
group('Expression compiler tests in closures:', () {
- const String source = r'''
+ const source = r'''
int globalFunction() {
int x = 15;
var c = C(1, 2);
@@ -1102,7 +1102,7 @@
});
group('Expression compiler tests in method with no type use', () {
- const String source = '''
+ const source = '''
abstract class Key {
const factory Key(String value) = ValueKey;
const Key.empty();
diff --git a/pkg/dev_compiler/test/expression_compiler/expression_compiler_worker_test.dart b/pkg/dev_compiler/test/expression_compiler/expression_compiler_worker_test.dart
new file mode 100644
index 0000000..18fa6ce
--- /dev/null
+++ b/pkg/dev_compiler/test/expression_compiler/expression_compiler_worker_test.dart
@@ -0,0 +1,647 @@
+// Copyright (c) 2020, the Dart project authors. Please see the AUTHORS file
+// for details. All rights reserved. Use of this source code is governed by a
+// BSD-style license that can be found in the LICENSE file.
+
+import 'dart:async';
+import 'dart:convert';
+import 'dart:io' show Directory, File, Platform, Process, stderr, stdout;
+
+import 'package:build_integration/file_system/multi_root.dart';
+import 'package:front_end/src/api_prototype/standard_file_system.dart';
+import 'package:front_end/src/compute_platform_binaries_location.dart';
+import 'package:path/path.dart' as p;
+import 'package:pedantic/pedantic.dart';
+import 'package:test/test.dart';
+
+import 'package:dev_compiler/src/kernel/expression_compiler_worker.dart';
+
+/// Verbose mode for debugging
+bool get verbose => false;
+
+class ModuleConfiguration {
+ final String outputPath;
+ final String moduleName;
+ final String libraryUri;
+ final String jsFileName;
+ final String fullDillFileName;
+
+ ModuleConfiguration(
+ {this.outputPath,
+ this.moduleName,
+ this.libraryUri,
+ this.jsFileName,
+ this.fullDillFileName});
+
+ String get jsPath => p.join(outputPath, jsFileName);
+ String get fullDillPath => p.join(outputPath, fullDillFileName);
+}
+
+class TestProjectConfiguration {
+ final Directory rootDirectory;
+ final String outputDir = 'out';
+
+ TestProjectConfiguration(this.rootDirectory);
+
+ ModuleConfiguration get mainModule => ModuleConfiguration(
+ outputPath: outputPath,
+ moduleName: 'packages/_testPackage/main',
+ libraryUri: 'org-dartlang-app:/lib/main.dart',
+ jsFileName: 'main.js',
+ fullDillFileName: 'main.full.dill');
+
+ ModuleConfiguration get testModule => ModuleConfiguration(
+ outputPath: outputPath,
+ moduleName: 'packages/_testPackage/test_library',
+ libraryUri: 'package:_testPackage/test_library.dart',
+ jsFileName: 'test_library.js',
+ fullDillFileName: 'test_library.full.dill');
+
+ ModuleConfiguration get testModule2 => ModuleConfiguration(
+ outputPath: outputPath,
+ moduleName: 'packages/_testPackage/test_library2',
+ libraryUri: 'package:_testPackage/test_library2.dart',
+ jsFileName: 'test_library2.js',
+ fullDillFileName: 'test_library2.full.dill');
+
+ String get root => rootDirectory.path;
+ String get outputPath => p.join(root, outputDir);
+ String get packagesPath => p.join(root, '.packages');
+
+ String get sdkRoot => computePlatformBinariesLocation().path;
+ String get sdkSummaryPath => p.join(sdkRoot, 'ddc_sdk.dill');
+ String get librariesPath => p.join(sdkRoot, 'lib', 'libraries.json');
+
+ void createTestProject() {
+ var pubspec = rootDirectory.uri.resolve('pubspec.yaml');
+ File.fromUri(pubspec)
+ ..createSync()
+ ..writeAsStringSync('''
+name: _testPackage
+version: 1.0.0
+
+environment:
+ sdk: '>=2.8.0 <3.0.0'
+''');
+
+ var packages = rootDirectory.uri.resolve('.packages');
+ File.fromUri(packages)
+ ..createSync()
+ ..writeAsStringSync('''
+_testPackage:lib/
+''');
+
+ var main = rootDirectory.uri.resolve('lib/main.dart');
+ File.fromUri(main)
+ ..createSync(recursive: true)
+ ..writeAsStringSync('''
+import 'package:_testPackage/test_library.dart';
+
+var global = 0;
+
+void main() {
+ var count = 0;
+ // line 7
+ print('Global is: \${++global}');
+ print('Count is: \${++count}');
+
+ B b = new B();
+}
+''');
+
+ var testLibrary = rootDirectory.uri.resolve('lib/test_library.dart');
+ File.fromUri(testLibrary)
+ ..createSync()
+ ..writeAsStringSync('''
+import 'package:_testPackage/test_library2.dart';
+
+int testLibraryFunction(int formal) {
+ return formal; // line 4
+}
+
+int callLibraryFunction2(int formal) {
+ return testLibraryFunction2(formal); // line 8
+}
+
+class B {
+ C c() => new C();
+}
+''');
+
+ var testLibrary2 = rootDirectory.uri.resolve('lib/test_library2.dart');
+ File.fromUri(testLibrary2)
+ ..createSync()
+ ..writeAsStringSync('''
+int testLibraryFunction2(int formal) {
+ return formal; // line 2
+}
+
+class C {
+ int getNumber() => 42;
+}
+''');
+ }
+}
+
+void main() async {
+ group('Expression compiler worker (webdev simulation) - ', () {
+ ExpressionCompilerWorker worker;
+ Future workerDone;
+ StreamController<Map<String, dynamic>> requestController;
+ StreamController<Map<String, dynamic>> responseController;
+ Directory tempDir;
+ TestProjectConfiguration config;
+ List inputs;
+
+ setUpAll(() async {
+ tempDir = Directory.systemTemp.createTempSync('foo bar');
+ config = TestProjectConfiguration(tempDir);
+
+ // simulate webdev
+ config.createTestProject();
+ var kernelGenerator = DDCKernelGenerator(config);
+ await kernelGenerator.generate();
+
+ inputs = [
+ {
+ 'path': config.mainModule.fullDillPath,
+ 'moduleName': config.mainModule.moduleName
+ },
+ {
+ 'path': config.testModule.fullDillPath,
+ 'moduleName': config.testModule.moduleName
+ },
+ {
+ 'path': config.testModule2.fullDillPath,
+ 'moduleName': config.testModule2.moduleName
+ },
+ ];
+ });
+
+ tearDownAll(() async {
+ tempDir.deleteSync(recursive: true);
+ });
+
+ setUp(() async {
+ var fileSystem = MultiRootFileSystem(
+ 'org-dartlang-app', [tempDir.uri], StandardFileSystem.instance);
+
+ requestController = StreamController<Map<String, dynamic>>();
+ responseController = StreamController<Map<String, dynamic>>();
+ worker = await ExpressionCompilerWorker.create(
+ librariesSpecificationUri: Uri.file(config.librariesPath),
+ // We should be able to load everything from dill and not require
+ // source parsing. Webdev and google3 integration currently rely on that.
+ // Make the test fail on source reading by not providing a packages.
+ packagesFile: null,
+ sdkSummary: Uri.file(config.sdkSummaryPath),
+ fileSystem: fileSystem,
+ requestStream: requestController.stream,
+ sendResponse: responseController.add,
+ verbose: verbose,
+ );
+ workerDone = worker.start();
+ });
+
+ tearDown(() async {
+ unawaited(requestController.close());
+ await workerDone;
+ unawaited(responseController.close());
+ });
+
+ test('can load dependencies and compile expressions in sdk', () async {
+ requestController.add({
+ 'command': 'UpdateDeps',
+ 'inputs': inputs,
+ });
+
+ requestController.add({
+ 'command': 'CompileExpression',
+ 'expression': 'other',
+ 'line': 107,
+ 'column': 1,
+ 'jsModules': {},
+ 'jsScope': {'other': 'other'},
+ 'libraryUri': 'dart:collection',
+ 'moduleName': 'dart_sdk',
+ });
+
+ expect(
+ responseController.stream,
+ emitsInOrder([
+ equals({
+ 'succeeded': true,
+ }),
+ equals({
+ 'succeeded': true,
+ 'errors': isEmpty,
+ 'warnings': isEmpty,
+ 'compiledProcedure': contains('return other;'),
+ })
+ ]));
+ }, skip: 'Evaluating expressions in SDK is not supported yet');
+
+ test('can load dependencies and compile expressions in a library',
+ () async {
+ requestController.add({
+ 'command': 'UpdateDeps',
+ 'inputs': inputs,
+ });
+
+ requestController.add({
+ 'command': 'CompileExpression',
+ 'expression': 'formal',
+ 'line': 4,
+ 'column': 1,
+ 'jsModules': {},
+ 'jsScope': {'formal': 'formal'},
+ 'libraryUri': config.testModule.libraryUri,
+ 'moduleName': config.testModule.moduleName,
+ });
+
+ expect(
+ responseController.stream,
+ emitsInOrder([
+ equals({
+ 'succeeded': true,
+ }),
+ equals({
+ 'succeeded': true,
+ 'errors': isEmpty,
+ 'warnings': isEmpty,
+ 'compiledProcedure': contains('return formal;'),
+ })
+ ]));
+ });
+
+ test('can load dependencies and compile expressions in main', () async {
+ requestController.add({
+ 'command': 'UpdateDeps',
+ 'inputs': inputs,
+ });
+
+ requestController.add({
+ 'command': 'CompileExpression',
+ 'expression': 'count',
+ 'line': 7,
+ 'column': 1,
+ 'jsModules': {},
+ 'jsScope': {'count': 'count'},
+ 'libraryUri': config.mainModule.libraryUri,
+ 'moduleName': config.mainModule.moduleName,
+ });
+
+ expect(
+ responseController.stream,
+ emitsInOrder([
+ equals({
+ 'succeeded': true,
+ }),
+ equals({
+ 'succeeded': true,
+ 'errors': isEmpty,
+ 'warnings': isEmpty,
+ 'compiledProcedure': contains('return count;'),
+ })
+ ]));
+ });
+
+ test('can load dependencies and compile transitive expressions in main',
+ () async {
+ requestController.add({
+ 'command': 'UpdateDeps',
+ 'inputs': inputs,
+ });
+
+ requestController.add({
+ 'command': 'CompileExpression',
+ 'expression': 'B().c().getNumber()',
+ 'line': 7,
+ 'column': 1,
+ 'jsModules': {},
+ 'jsScope': {},
+ 'libraryUri': config.mainModule.libraryUri,
+ 'moduleName': config.mainModule.moduleName,
+ });
+
+ expect(
+ responseController.stream,
+ emitsInOrder([
+ equals({
+ 'succeeded': true,
+ }),
+ equals({
+ 'succeeded': true,
+ 'errors': isEmpty,
+ 'warnings': isEmpty,
+ 'compiledProcedure':
+ contains('return new test_library.B.new().c().getNumber()'),
+ })
+ ]));
+ });
+ });
+
+ group('Expression compiler worker (google3 simulation) - ', () {
+ ExpressionCompilerWorker worker;
+ Future workerDone;
+ StreamController<Map<String, dynamic>> requestController;
+ StreamController<Map<String, dynamic>> responseController;
+ Directory tempDir;
+ TestProjectConfiguration config;
+ List inputs;
+
+ setUpAll(() async {
+ tempDir = Directory.systemTemp.createTempSync('foo bar');
+ config = TestProjectConfiguration(tempDir);
+
+ // simulate google3
+ config.createTestProject();
+ var kernelGenerator = BazelKernelWorkerGenerator(config);
+ await kernelGenerator.generate();
+
+ inputs = [
+ {
+ 'path': config.mainModule.fullDillPath,
+ 'moduleName': config.mainModule.moduleName
+ },
+ ];
+ });
+
+ tearDownAll(() async {
+ tempDir.deleteSync(recursive: true);
+ });
+
+ setUp(() async {
+ var fileSystem = MultiRootFileSystem(
+ 'org-dartlang-app', [tempDir.uri], StandardFileSystem.instance);
+
+ requestController = StreamController<Map<String, dynamic>>();
+ responseController = StreamController<Map<String, dynamic>>();
+ worker = await ExpressionCompilerWorker.create(
+ librariesSpecificationUri: Uri.file(config.librariesPath),
+ packagesFile: null,
+ sdkSummary: Uri.file(config.sdkSummaryPath),
+ fileSystem: fileSystem,
+ requestStream: requestController.stream,
+ sendResponse: responseController.add,
+ verbose: verbose,
+ );
+ workerDone = worker.start();
+ });
+
+ tearDown(() async {
+ unawaited(requestController.close());
+ await workerDone;
+ unawaited(responseController.close());
+ });
+
+ test('can load dependencies and compile expressions in sdk', () async {
+ requestController.add({
+ 'command': 'UpdateDeps',
+ 'inputs': inputs,
+ });
+
+ requestController.add({
+ 'command': 'CompileExpression',
+ 'expression': 'other',
+ 'line': 107,
+ 'column': 1,
+ 'jsModules': {},
+ 'jsScope': {'other': 'other'},
+ 'libraryUri': 'dart:collection',
+ 'moduleName': 'dart_sdk',
+ });
+
+ expect(
+ responseController.stream,
+ emitsInOrder([
+ equals({
+ 'succeeded': true,
+ }),
+ equals({
+ 'succeeded': true,
+ 'errors': isEmpty,
+ 'warnings': isEmpty,
+ 'compiledProcedure': contains('return other;'),
+ })
+ ]));
+ }, skip: 'Evaluating expressions in SDK is not supported yet');
+
+ test('can load dependencies and compile expressions in a library',
+ () async {
+ requestController.add({
+ 'command': 'UpdateDeps',
+ 'inputs': inputs,
+ });
+
+ requestController.add({
+ 'command': 'CompileExpression',
+ 'expression': 'formal',
+ 'line': 4,
+ 'column': 1,
+ 'jsModules': {},
+ 'jsScope': {'formal': 'formal'},
+ 'libraryUri': config.testModule.libraryUri,
+ 'moduleName': config.mainModule.moduleName,
+ });
+
+ expect(
+ responseController.stream,
+ emitsInOrder([
+ equals({
+ 'succeeded': true,
+ }),
+ equals({
+ 'succeeded': true,
+ 'errors': isEmpty,
+ 'warnings': isEmpty,
+ 'compiledProcedure': contains('return formal;'),
+ })
+ ]));
+ });
+
+ test('can load dependencies and compile expressions in main', () async {
+ requestController.add({
+ 'command': 'UpdateDeps',
+ 'inputs': inputs,
+ });
+
+ requestController.add({
+ 'command': 'CompileExpression',
+ 'expression': 'count',
+ 'line': 7,
+ 'column': 1,
+ 'jsModules': {},
+ 'jsScope': {'count': 'count'},
+ 'libraryUri': config.mainModule.libraryUri,
+ 'moduleName': config.mainModule.moduleName,
+ });
+
+ expect(
+ responseController.stream,
+ emitsInOrder([
+ equals({
+ 'succeeded': true,
+ }),
+ equals({
+ 'succeeded': true,
+ 'errors': isEmpty,
+ 'warnings': isEmpty,
+ 'compiledProcedure': contains('return count;'),
+ })
+ ]));
+ });
+ }, skip: 'bazel kernel worker does not support full kernel generation yet');
+}
+
+/// Uses DDC to generate kernel from the test code
+/// in order to simulate webdev environment
+class DDCKernelGenerator {
+ final TestProjectConfiguration config;
+
+ DDCKernelGenerator(this.config);
+
+ Future<int> generate() async {
+ var dart = Platform.resolvedExecutable;
+ var dartdevc =
+ p.join(p.dirname(dart), 'snapshots', 'dartdevc.dart.snapshot');
+
+ Directory(config.outputPath)..createSync();
+
+ // generate test_library2.full.dill
+ var args = [
+ dartdevc,
+ config.testModule2.libraryUri,
+ '--no-summarize',
+ '-o',
+ config.testModule2.jsPath,
+ '--source-map',
+ '--experimental-emit-debug-metadata',
+ '--experimental-output-compiled-kernel',
+ '--dart-sdk-summary',
+ config.sdkSummaryPath,
+ '--multi-root',
+ config.root,
+ '--multi-root-scheme',
+ 'org-dartlang-app',
+ '--packages',
+ config.packagesPath,
+ ];
+
+ var exitCode = await runProcess(dart, args, config.root);
+ if (exitCode != 0) {
+ return exitCode;
+ }
+
+ // generate test_library.full.dill
+ args = [
+ dartdevc,
+ config.testModule.libraryUri,
+ '--no-summarize',
+ '--summary',
+ '${config.testModule2.fullDillPath}=${config.testModule2.moduleName}',
+ '-o',
+ config.testModule.jsPath,
+ '--source-map',
+ '--experimental-emit-debug-metadata',
+ '--experimental-output-compiled-kernel',
+ '--dart-sdk-summary',
+ config.sdkSummaryPath,
+ '--multi-root',
+ config.root,
+ '--multi-root-scheme',
+ 'org-dartlang-app',
+ '--packages',
+ config.packagesPath,
+ ];
+
+ exitCode = await runProcess(dart, args, config.root);
+ if (exitCode != 0) {
+ return exitCode;
+ }
+
+ // generate main.full.dill
+ args = [
+ dartdevc,
+ config.mainModule.libraryUri,
+ '--no-summarize',
+ '--summary',
+ '${config.testModule2.fullDillPath}=${config.testModule2.moduleName}',
+ '--summary',
+ '${config.testModule.fullDillPath}=${config.testModule.moduleName}',
+ '-o',
+ config.mainModule.jsPath,
+ '--source-map',
+ '--experimental-emit-debug-metadata',
+ '--experimental-output-compiled-kernel',
+ '--dart-sdk-summary',
+ config.sdkSummaryPath,
+ '--multi-root',
+ config.root,
+ '--multi-root-scheme',
+ 'org-dartlang-app',
+ '--packages',
+ config.packagesPath,
+ ];
+
+ return await runProcess(dart, args, config.root);
+ }
+}
+
+/// Uses bazel kernel worker to generate kernel from test code
+/// in order to simulate google3 environment
+/// TODO: split into 3 modules
+class BazelKernelWorkerGenerator {
+ TestProjectConfiguration config;
+
+ BazelKernelWorkerGenerator(this.config);
+
+ Future<void> generate() async {
+ var dart = Platform.resolvedExecutable;
+ var kernelWorker =
+ p.join(p.dirname(dart), 'snapshots', 'kernel_worker.dart.snapshot');
+
+ var args = [
+ kernelWorker,
+ '--target',
+ 'ddc',
+ '--output',
+ config.mainModule.fullDillPath,
+ '--dart-sdk-summary',
+ config.sdkSummaryPath,
+ '--exclude-non-sources',
+ '--source',
+ config.mainModule.libraryUri,
+ '--source',
+ config.testModule.libraryUri,
+ '--source',
+ config.testModule2.libraryUri,
+ '--multi-root',
+ config.root,
+ '--multi-root-scheme',
+ 'org-dartlang-app',
+ '--packages-file',
+ '.packages',
+ '--verbose'
+ ];
+
+ return await runProcess(dart, args, config.root);
+ }
+}
+
+Future<int> runProcess(
+ String command, List<String> args, String workingDirectory) async {
+ if (verbose) {
+ print('Running command in $workingDirectory:'
+ '\n\t $command ${args.join(' ')}, ');
+ }
+ var process =
+ await Process.start(command, args, workingDirectory: workingDirectory)
+ .then((Process process) {
+ process
+ ..stdout.transform(utf8.decoder).listen(stdout.write)
+ ..stderr.transform(utf8.decoder).listen(stderr.write);
+ return process;
+ });
+
+ return await process.exitCode;
+}
diff --git a/pkg/frontend_server/lib/frontend_server.dart b/pkg/frontend_server/lib/frontend_server.dart
index 35f5631..af48251 100644
--- a/pkg/frontend_server/lib/frontend_server.dart
+++ b/pkg/frontend_server/lib/frontend_server.dart
@@ -9,7 +9,8 @@
import 'dart:io' hide FileSystemEntity;
import 'package:args/args.dart';
-import 'package:dev_compiler/dev_compiler.dart' show DevCompilerTarget;
+import 'package:dev_compiler/dev_compiler.dart'
+ show DevCompilerTarget, ExpressionCompiler;
// front_end/src imports below that require lint `ignore_for_file`
// are a temporary state of things until frontend team builds better api
@@ -39,7 +40,6 @@
import 'src/javascript_bundle.dart';
import 'src/strong_components.dart';
-import 'src/expression_compiler.dart';
ArgParser argParser = ArgParser(allowTrailingOptions: true)
..addFlag('train',
@@ -968,7 +968,8 @@
var evaluator = new ExpressionCompiler(
_generator.generator, kernel2jsCompiler, component,
verbose: _compilerOptions.verbose,
- onDiagnostic: _compilerOptions.onDiagnostic);
+ onDiagnostic: _compilerOptions.onDiagnostic,
+ errors: errors);
var procedure = await evaluator.compileExpressionToJs(libraryUri, line,
column, jsModules, jsFrameValues, moduleName, expression);
diff --git a/pkg/frontend_server/pubspec.yaml b/pkg/frontend_server/pubspec.yaml
index 6f96f7e..7932dab 100644
--- a/pkg/frontend_server/pubspec.yaml
+++ b/pkg/frontend_server/pubspec.yaml
@@ -7,8 +7,6 @@
sdk: "^2.7.0"
dependencies:
- _fe_analyzer_shared:
- path: ../_fe_analyzer_shared
args: ^1.4.4
dev_compiler:
path: ../dev_compiler
@@ -23,6 +21,5 @@
path: ../vm
dev_dependencies:
- cli_util: any
mockito: any
test: any
