Take a first pass at adding a benchmark. It doesn't cover all of the markdown syntax yet, but it has a blob of realistic markdown and a benchmark runner. R=nweiz@google.com Review URL: https://codereview.chromium.org//1314153005 .
diff --git a/pkgs/markdown/benchmark/benchmark.dart b/pkgs/markdown/benchmark/benchmark.dart new file mode 100644 index 0000000..ed6b521 --- /dev/null +++ b/pkgs/markdown/benchmark/benchmark.dart
@@ -0,0 +1,73 @@ +// Copyright (c) 2015, the Dart project authors. Please see the AUTHORS file +// for details. All rights reserved. Use of this source code is governed by a +// BSD-style license that can be found in the LICENSE file. + +library markdown.benchmark.benchmark; + +import 'dart:io'; + +import 'package:path/path.dart' as p; + +import 'package:markdown/markdown.dart'; + +const numTrials = 100; +const runsPerTrial = 50; + +final source = loadFile("input.md"); +final expected = loadFile("output.html"); + +void main(List<String> args) { + var best = double.INFINITY; + + // Run the benchmark several times. This ensures the VM is warmed up and lets + // us see how much variance there is. + for (var i = 0; i <= numTrials; i++) { + var start = new DateTime.now(); + + // For a single benchmark, convert the source multiple times. + var result; + for (var j = 0; j < runsPerTrial; j++) { + result = markdownToHtml(source); + } + + var elapsed = + new DateTime.now().difference(start).inMilliseconds / runsPerTrial; + + // Keep track of the best run so far. + if (elapsed >= best) continue; + best = elapsed; + + // Sanity check to make sure the output is what we expect and to make sure + // the VM doesn't optimize "dead" code away. + if (result != expected) { + print("Incorrect output:\n$result"); + exit(1); + } + + // Don't print the first run. It's always terrible since the VM hasn't + // warmed up yet. + if (i == 0) continue; + printResult("Run ${padLeft('#$i', 3)}", elapsed); + } + + printResult("Best ", best); +} + +String loadFile(String name) { + var path = p.join(p.dirname(p.fromUri(Platform.script)), name); + return new File(path).readAsStringSync(); +} + +void printResult(String label, double time) { + print("$label: ${padLeft(time.toStringAsFixed(2), 4)}ms " + "${'=' * ((time * 20).toInt())}"); +} + +String padLeft(input, int length) { + var result = input.toString(); + if (result.length < length) { + result = " " * (length - result.length) + result; + } + + return result; +}
diff --git a/pkgs/markdown/benchmark/input.md b/pkgs/markdown/benchmark/input.md new file mode 100644 index 0000000..31f5f57 --- /dev/null +++ b/pkgs/markdown/benchmark/input.md
@@ -0,0 +1,496 @@ +**TODO: Add more examples to cover all of the syntax.** + +This input was taken from the test package's README to get a representative +sample of real-world markdown: + +## Writing Tests + +Tests are specified using the top-level [`test()`][test] function, and test +assertions are made using [`expect()`][expect]: + +[test]: http://www.dartdocs.org/documentation/test/latest/index.html#test/test@id_test +[expect]: http://www.dartdocs.org/documentation/test/latest/index.html#test/test@id_expect + +```dart +import "package:test/test.dart"; + +void main() { + test("String.split() splits the string on the delimiter", () { + var string = "foo,bar,baz"; + expect(string.split(","), equals(["foo", "bar", "baz"])); + }); + + test("String.trim() removes surrounding whitespace", () { + var string = " foo "; + expect(string.trim(), equals("foo")); + }); +} +``` + +Tests can be grouped together using the [`group()`] function. Each group's +description is added to the beginning of its test's descriptions. + +```dart +import "package:test/test.dart"; + +void main() { + group("String", () { + test(".split() splits the string on the delimiter", () { + var string = "foo,bar,baz"; + expect(string.split(","), equals(["foo", "bar", "baz"])); + }); + + test(".trim() removes surrounding whitespace", () { + var string = " foo "; + expect(string.trim(), equals("foo")); + }); + }); + + group("int", () { + test(".remainder() returns the remainder of division", () { + expect(11.remainder(3), equals(2)); + }); + + test(".toRadixString() returns a hex string", () { + expect(11.toRadixString(16), equals("b")); + }); + }); +} +``` + +Any matchers from the [`matcher`][matcher] package can be used with `expect()` +to do complex validations: + +[matcher]: http://www.dartdocs.org/documentation/matcher/latest/index.html#matcher/matcher + +```dart +import "package:test/test.dart"; + +void main() { + test(".split() splits the string on the delimiter", () { + expect("foo,bar,baz", allOf([ + contains("foo"), + isNot(startsWith("bar")), + endsWith("baz") + ])); + }); +} +``` + +## Running Tests + +A single test file can be run just using `pub run test:test path/to/test.dart` +(on Dart 1.10, this can be shortened to `pub run test path/to/test.dart`). + + + +Many tests can be run at a time using `pub run test:test path/to/dir`. + + + +It's also possible to run a test on the Dart VM only by invoking it using `dart +path/to/test.dart`, but this doesn't load the full test runner and will be +missing some features. + +The test runner considers any file that ends with `_test.dart` to be a test +file. If you don't pass any paths, it will run all the test files in your +`test/` directory, making it easy to test your entire application at once. + +By default, tests are run in the Dart VM, but you can run them in the browser as +well by passing `pub run test:test -p chrome path/to/test.dart`. +`test` will take care of starting the browser and loading the tests, and all +the results will be reported on the command line just like for VM tests. In +fact, you can even run tests on both platforms with a single command: `pub run +test:test -p "chrome,vm" path/to/test.dart`. + +### Restricting Tests to Certain Platforms + +Some test files only make sense to run on particular platforms. They may use +`dart:html` or `dart:io`, they might test Windows' particular filesystem +behavior, or they might use a feature that's only available in Chrome. The +[`@TestOn`][TestOn] annotation makes it easy to declare exactly which platforms +a test file should run on. Just put it at the top of your file, before any +`library` or `import` declarations: + +```dart +@TestOn("vm") + +import "dart:io"; + +import "package:test/test.dart"; + +void main() { + // ... +} +``` + +[TestOn]: http://www.dartdocs.org/documentation/test/latest/index.html#test/test.TestOn + +The string you pass to `@TestOn` is what's called a "platform selector", and it +specifies exactly which platforms a test can run on. It can be as simple as the +name of a platform, or a more complex Dart-like boolean expression involving +these platform names. + +### Platform Selector Syntax + +Platform selectors can contain identifiers, parentheses, and operators. When +loading a test, each identifier is set to `true` or `false` based on the current +platform, and the test is only loaded if the platform selector returns `true`. +The operators `||`, `&&`, `!`, and `? :` all work just like they do in Dart. The +valid identifiers are: + +* `vm`: Whether the test is running on the command-line Dart VM. + +* `dartium`: Whether the test is running on Dartium. + +* `content-shell`: Whether the test is running on the headless Dartium content + shell. + +* `chrome`: Whether the test is running on Google Chrome. + +* `phantomjs`: Whether the test is running on + [PhantomJS](http://phantomjs.org/). + +* `firefox`: Whether the test is running on Mozilla Firefox. + +* `safari`: Whether the test is running on Apple Safari. + +* `ie`: Whether the test is running on Microsoft Internet Explorer. + +* `dart-vm`: Whether the test is running on the Dart VM in any context, + including Dartium. It's identical to `!js`. + +* `browser`: Whether the test is running in any browser. + +* `js`: Whether the test has been compiled to JS. This is identical to + `!dart-vm`. + +* `blink`: Whether the test is running in a browser that uses the Blink + rendering engine. + +* `windows`: Whether the test is running on Windows. If `vm` is false, this will + be `false` as well. + +* `mac-os`: Whether the test is running on Mac OS. If `vm` is false, this will + be `false` as well. + +* `linux`: Whether the test is running on Linux. If `vm` is false, this will be + `false` as well. + +* `android`: Whether the test is running on Android. If `vm` is false, this will + be `false` as well, which means that this *won't* be true if the test is + running on an Android browser. + +* `posix`: Whether the test is running on a POSIX operating system. This is + equivalent to `!windows`. + +For example, if you wanted to run a test on every browser but Chrome, you would +write `@TestOn("browser && !chrome")`. + +### Running Tests on Dartium + +Tests can be run on [Dartium][] by passing the `-p dartium` flag. If you're +using the Dart Editor, the test runner will be able to find Dartium +automatically. On Mac OS, you can also [install it using Homebrew][homebrew]. +Otherwise, make sure there's an executable called `dartium` (on Mac OS or Linux) +or `dartium.exe` (on Windows) on your system path. + +[Dartium]: https://www.dartlang.org/tools/dartium/ +[homebrew]: https://github.com/dart-lang/homebrew-dart + +Similarly, tests can be run on the headless Dartium content shell by passing `-p +content-shell`. The content shell is installed along with Dartium when using +Homebrew. Otherwise, you can downloaded it manually [from this +page][content_shell]; if you do, make sure the executable named `content_shell` +(on Mac OS or Linux) or `content_shell.exe` (on Windows) is on your system path. + +[content_shell]: http://gsdview.appspot.com/dart-archive/channels/stable/release/latest/dartium/ + +[In the future][issue 63], there will be a more explicit way to configure the +location of both the Dartium and content shell executables. + +[issue 63]: https://github.com/dart-lang/test/issues/63 + +## Asynchronous Tests + +Tests written with `async`/`await` will work automatically. The test runner +won't consider the test finished until the returned `Future` completes. + +```dart +import "dart:async"; + +import "package:test/test.dart"; + +void main() { + test("new Future.value() returns the value", () async { + var value = await new Future.value(10); + expect(value, equals(10)); + }); +} +``` + +There are also a number of useful functions and matchers for more advanced +asynchrony. The [`completion()`][completion] matcher can be used to test +`Futures`; it ensures that the test doesn't finish until the `Future` completes, +and runs a matcher against that `Future`'s value. + +[completion]: http://www.dartdocs.org/documentation/test/latest/index.html#test/test@id_completion + +```dart +import "dart:async"; + +import "package:test/test.dart"; + +void main() { + test("new Future.value() returns the value", () { + expect(new Future.value(10), completion(equals(10))); + }); +} +``` + +The [`throwsA()`][throwsA] matcher and the various `throwsExceptionType` +matchers work with both synchronous callbacks and asynchronous `Future`s. They +ensure that a particular type of exception is thrown: + +[throwsA]: http://www.dartdocs.org/documentation/test/latest/index.html#test/test@id_throwsA + +```dart +import "dart:async"; + +import "package:test/test.dart"; + +void main() { + test("new Future.error() throws the error", () { + expect(new Future.error("oh no"), throwsA(equals("oh no"))); + expect(new Future.error(new StateError("bad state")), throwsStateError); + }); +} +``` + +The [`expectAsync()`][expectAsync] function wraps another function and has two +jobs. First, it asserts that the wrapped function is called a certain number of +times, and will cause the test to fail if it's called too often; second, it +keeps the test from finishing until the function is called the requisite number +of times. + +```dart +import "dart:async"; + +import "package:test/test.dart"; + +void main() { + test("Stream.fromIterable() emits the values in the iterable", () { + var stream = new Stream.fromIterable([1, 2, 3]); + + stream.listen(expectAsync((number) { + expect(number, inInclusiveRange(1, 3)); + }, count: 3)); + }); +} +``` + +[expectAsync]: http://www.dartdocs.org/documentation/test/latest/index.html#test/test@id_expectAsync + +## Running Tests with Custom HTML + +By default, the test runner will generate its own empty HTML file for browser +tests. However, tests that need custom HTML can create their own files. These +files have three requirements: + +* They must have the same name as the test, with `.dart` replaced by `.html`. + +* They must contain a `link` tag with `rel="x-dart-test"` and an `href` + attribute pointing to the test script. + +* They must contain `<script src="packages/test/dart.js"></script>`. + +For example, if you had a test called `custom_html_test.dart`, you might write +the following HTML file: + +```html +<!doctype html> +<!-- custom_html_test.html --> +<html> + <head> + <title>Custom HTML Test</title> + <link rel="x-dart-test" href="custom_html_test.dart"> + <script src="packages/test/dart.js"></script> + </head> + <body> + // ... + </body> +</html> +``` + +## Configuring Tests + +### Skipping Tests + +If a test, group, or entire suite isn't working yet and you just want it to stop +complaining, you can mark it as "skipped". The test or tests won't be run, and, +if you supply a reason why, that reason will be printed. In general, skipping +tests indicates that they should run but is temporarily not working. If they're +is fundamentally incompatible with a platform, [`@TestOn`/`testOn`][TestOn] +should be used instead. + +[TestOn]: #restricting-tests-to-certain-platforms + +To skip a test suite, put a `@Skip` annotation at the top of the file: + +```dart +@Skip("currently failing (see issue 1234)") + +import "package:test/test.dart"; + +void main() { + // ... +} +``` + +The string you pass should describe why the test is skipped. You don't have to +include it, but it's a good idea to document why the test isn't running. + +Groups and individual tests can be skipped by passing the `skip` parameter. This +can be either `true` or a String describing why the test is skipped. For example: + +```dart +import "package:test/test.dart"; + +void main() { + group("complicated algorithm tests", () { + // ... + }, skip: "the algorithm isn't quite right"); + + test("error-checking test", () { + // ... + }, skip: "TODO: add error-checking."); +} +``` + +### Timeouts + +By default, tests will time out after 30 seconds of inactivity. However, this +can be configured on a per-test, -group, or -suite basis. To change the timeout +for a test suite, put a `@Timeout` annotation at the top of the file: + +```dart +@Timeout(const Duration(seconds: 45)) + +import "package:test/test.dart"; + +void main() { + // ... +} +``` + +In addition to setting an absolute timeout, you can set the timeout relative to +the default using `@Timeout.factor`. For example, `@Timeout.factor(1.5)` will +set the timeout to one and a half times as long as the default—45 seconds. + +Timeouts can be set for tests and groups using the `timeout` parameter. This +parameter takes a `Timeout` object just like the annotation. For example: + +```dart +import "package:test/test.dart"; + +void main() { + group("slow tests", () { + // ... + + test("even slower test", () { + // ... + }, timeout: new Timeout.factor(2)) + }, timeout: new Timeout(new Duration(minutes: 1))); +} +``` + +Nested timeouts apply in order from outermost to innermost. That means that +"even slower test" will take two minutes to time out, since it multiplies the +group's timeout by 2. + +### Platform-Specific Configuration + +Sometimes a test may need to be configured differently for different platforms. +Windows might run your code slower than other platforms, or your DOM +manipulation might not work right on Safari yet. For these cases, you can use +the `@OnPlatform` annotation and the `onPlatform` named parameter to `test()` +and `group()`. For example: + +```dart +@OnPlatform(const { + // Give Windows some extra wiggle-room before timing out. + "windows": const Timeout.factor(2) +}) + +import "package:test/test.dart"; + +void main() { + test("do a thing", () { + // ... + }, onPlatform: { + "safari": new Skip("Safari is currently broken (see #1234)") + }); +} +``` + +Both the annotation and the parameter take a map. The map's keys are [platform +selectors](#platform-selector-syntax) which describe the platforms for which the +specialized configuration applies. Its values are instances of some of the same +annotation classes that can be used for a suite: `Skip` and `Timeout`. A value +can also be a list of these values. + +If multiple platforms match, the configuration is applied in order from first to +last, just as they would in nested groups. This means that for configuration +like duration-based timeouts, the last matching value wins. + +## Testing With `barback` + +Packages using the `barback` transformer system may need to test code that's +created or modified using transformers. The test runner handles this using the +`--pub-serve` option, which tells it to load the test code from a `pub serve` +instance rather than from the filesystem. + +Before using the `--pub-serve` option, add the `test/pub_serve` transformer to +your `pubspec.yaml`. This transformer adds the necessary bootstrapping code that +allows the test runner to load your tests properly: + +```yaml +transformers: +- test/pub_serve: + $include: test/**_test{.*,}.dart +``` + +Note that if you're using the test runner along with [`polymer`][polymer], you +have to make sure that the `test/pub_serve` transformer comes *after* the +`polymer` transformer: + +[polymer]: https://www.dartlang.org/polymer/ + +```yaml +transformers: +- polymer +- test/pub_serve: + $include: test/**_test{.*,}.dart +``` + +Then, start up `pub serve`. Make sure to pay attention to which port it's using +to serve your `test/` directory: + +```shell +$ pub serve +Loading source assets... +Loading test/pub_serve transformers... +Serving my_app web on http://localhost:8080 +Serving my_app test on http://localhost:8081 +Build completed successfully +``` + +In this case, the port is `8081`. In another terminal, pass this port to +`--pub-serve` and otherwise invoke `pub run test:test` as normal: + +```shell +$ pub run test:test --pub-serve=8081 -p chrome +"pub serve" is compiling test/my_app_test.dart... +"pub serve" is compiling test/utils_test.dart... +00:00 +42: All tests passed! +```
diff --git a/pkgs/markdown/benchmark/output.html b/pkgs/markdown/benchmark/output.html new file mode 100644 index 0000000..6c49aae --- /dev/null +++ b/pkgs/markdown/benchmark/output.html
@@ -0,0 +1,364 @@ +<p><strong>TODO: Add more examples to cover all of the syntax.</strong></p> +<p>This input was taken from the test package's README to get a representative +sample of real-world markdown:</p> +<h2>Writing Tests</h2> +<p>Tests are specified using the top-level <a href="http://www.dartdocs.org/documentation/test/latest/index.html#test/test@id_test"><code>test()</code></a> function, and test +assertions are made using <a href="http://www.dartdocs.org/documentation/test/latest/index.html#test/test@id_expect"><code>expect()</code></a>:</p> +<pre class="dart"><code>import "package:test/test.dart"; + +void main() { + test("String.split() splits the string on the delimiter", () { + var string = "foo,bar,baz"; + expect(string.split(","), equals(["foo", "bar", "baz"])); + }); + + test("String.trim() removes surrounding whitespace", () { + var string = " foo "; + expect(string.trim(), equals("foo")); + }); +} +</code></pre> +<p>Tests can be grouped together using the [`group()`] function. Each group's +description is added to the beginning of its test's descriptions.</p> +<pre class="dart"><code>import "package:test/test.dart"; + +void main() { + group("String", () { + test(".split() splits the string on the delimiter", () { + var string = "foo,bar,baz"; + expect(string.split(","), equals(["foo", "bar", "baz"])); + }); + + test(".trim() removes surrounding whitespace", () { + var string = " foo "; + expect(string.trim(), equals("foo")); + }); + }); + + group("int", () { + test(".remainder() returns the remainder of division", () { + expect(11.remainder(3), equals(2)); + }); + + test(".toRadixString() returns a hex string", () { + expect(11.toRadixString(16), equals("b")); + }); + }); +} +</code></pre> +<p>Any matchers from the <a href="http://www.dartdocs.org/documentation/matcher/latest/index.html#matcher/matcher"><code>matcher</code></a> package can be used with <code>expect()</code> +to do complex validations:</p> +<pre class="dart"><code>import "package:test/test.dart"; + +void main() { + test(".split() splits the string on the delimiter", () { + expect("foo,bar,baz", allOf([ + contains("foo"), + isNot(startsWith("bar")), + endsWith("baz") + ])); + }); +} +</code></pre> +<h2>Running Tests</h2> +<p>A single test file can be run just using <code>pub run test:test path/to/test.dart</code> +(on Dart 1.10, this can be shortened to <code>pub run test path/to/test.dart</code>).</p> +<p><a href="https://raw.githubusercontent.com/dart-lang/test/master/image/test1.gif"><img alt="Single file being run via pub run"" src="https://raw.githubusercontent.com/dart-lang/test/master/image/test1.gif"></img></a></p> +<p>Many tests can be run at a time using <code>pub run test:test path/to/dir</code>.</p> +<p><a href="https://raw.githubusercontent.com/dart-lang/test/master/image/test2.gif"><img alt="Directory being run via "pub run"." src="https://raw.githubusercontent.com/dart-lang/test/master/image/test2.gif"></img></a></p> +<p>It's also possible to run a test on the Dart VM only by invoking it using <code>dart +path/to/test.dart</code>, but this doesn't load the full test runner and will be +missing some features.</p> +<p>The test runner considers any file that ends with <code>_test.dart</code> to be a test +file. If you don't pass any paths, it will run all the test files in your +<code>test/</code> directory, making it easy to test your entire application at once.</p> +<p>By default, tests are run in the Dart VM, but you can run them in the browser as +well by passing <code>pub run test:test -p chrome path/to/test.dart</code>. +<code>test</code> will take care of starting the browser and loading the tests, and all +the results will be reported on the command line just like for VM tests. In +fact, you can even run tests on both platforms with a single command: <code>pub run +test:test -p "chrome,vm" path/to/test.dart</code>.</p> +<h3>Restricting Tests to Certain Platforms</h3> +<p>Some test files only make sense to run on particular platforms. They may use +<code>dart:html</code> or <code>dart:io</code>, they might test Windows' particular filesystem +behavior, or they might use a feature that's only available in Chrome. The +<a href="#restricting-tests-to-certain-platforms"><code>@TestOn</code></a> annotation makes it easy to declare exactly which platforms +a test file should run on. Just put it at the top of your file, before any +<code>library</code> or <code>import</code> declarations:</p> +<pre class="dart"><code>@TestOn("vm") + +import "dart:io"; + +import "package:test/test.dart"; + +void main() { + // ... +} +</code></pre> +<p>The string you pass to <code>@TestOn</code> is what's called a "platform selector", and it +specifies exactly which platforms a test can run on. It can be as simple as the +name of a platform, or a more complex Dart-like boolean expression involving +these platform names.</p> +<h3>Platform Selector Syntax</h3> +<p>Platform selectors can contain identifiers, parentheses, and operators. When +loading a test, each identifier is set to <code>true</code> or <code>false</code> based on the current +platform, and the test is only loaded if the platform selector returns <code>true</code>. +The operators <code>||</code>, <code>&&</code>, <code>!</code>, and <code>? :</code> all work just like they do in Dart. The +valid identifiers are:</p><ul><li> +<p><code>vm</code>: Whether the test is running on the command-line Dart VM.</p></li><li> +<p><code>dartium</code>: Whether the test is running on Dartium.</p></li><li> +<p><code>content-shell</code>: Whether the test is running on the headless Dartium content + shell.</p></li><li> +<p><code>chrome</code>: Whether the test is running on Google Chrome.</p></li><li> +<p><code>phantomjs</code>: Whether the test is running on + <a href="http://phantomjs.org/">PhantomJS</a>.</p></li><li> +<p><code>firefox</code>: Whether the test is running on Mozilla Firefox.</p></li><li> +<p><code>safari</code>: Whether the test is running on Apple Safari.</p></li><li> +<p><code>ie</code>: Whether the test is running on Microsoft Internet Explorer.</p></li><li> +<p><code>dart-vm</code>: Whether the test is running on the Dart VM in any context, + including Dartium. It's identical to <code>!js</code>.</p></li><li> +<p><code>browser</code>: Whether the test is running in any browser.</p></li><li> +<p><code>js</code>: Whether the test has been compiled to JS. This is identical to + <code>!dart-vm</code>.</p></li><li> +<p><code>blink</code>: Whether the test is running in a browser that uses the Blink + rendering engine.</p></li><li> +<p><code>windows</code>: Whether the test is running on Windows. If <code>vm</code> is false, this will + be <code>false</code> as well.</p></li><li> +<p><code>mac-os</code>: Whether the test is running on Mac OS. If <code>vm</code> is false, this will + be <code>false</code> as well.</p></li><li> +<p><code>linux</code>: Whether the test is running on Linux. If <code>vm</code> is false, this will be + <code>false</code> as well.</p></li><li> +<p><code>android</code>: Whether the test is running on Android. If <code>vm</code> is false, this will + be <code>false</code> as well, which means that this <em>won't</em> be true if the test is + running on an Android browser.</p></li><li> +<p><code>posix</code>: Whether the test is running on a POSIX operating system. This is + equivalent to <code>!windows</code>.</p></li></ul> +<p>For example, if you wanted to run a test on every browser but Chrome, you would +write <code>@TestOn("browser && !chrome")</code>.</p> +<h3>Running Tests on Dartium</h3> +<p>Tests can be run on <a href="https://www.dartlang.org/tools/dartium/">Dartium</a> by passing the <code>-p dartium</code> flag. If you're +using the Dart Editor, the test runner will be able to find Dartium +automatically. On Mac OS, you can also <a href="https://github.com/dart-lang/homebrew-dart">install it using Homebrew</a>. +Otherwise, make sure there's an executable called <code>dartium</code> (on Mac OS or Linux) +or <code>dartium.exe</code> (on Windows) on your system path.</p> +<p>Similarly, tests can be run on the headless Dartium content shell by passing <code>-p +content-shell</code>. The content shell is installed along with Dartium when using +Homebrew. Otherwise, you can downloaded it manually <a href="http://gsdview.appspot.com/dart-archive/channels/stable/release/latest/dartium/">from this +page</a>; if you do, make sure the executable named <code>content_shell</code> +(on Mac OS or Linux) or <code>content_shell.exe</code> (on Windows) is on your system path.</p> +<p><a href="https://github.com/dart-lang/test/issues/63">In the future</a>, there will be a more explicit way to configure the +location of both the Dartium and content shell executables.</p> +<h2>Asynchronous Tests</h2> +<p>Tests written with <code>async</code>/<code>await</code> will work automatically. The test runner +won't consider the test finished until the returned <code>Future</code> completes.</p> +<pre class="dart"><code>import "dart:async"; + +import "package:test/test.dart"; + +void main() { + test("new Future.value() returns the value", () async { + var value = await new Future.value(10); + expect(value, equals(10)); + }); +} +</code></pre> +<p>There are also a number of useful functions and matchers for more advanced +asynchrony. The <a href="http://www.dartdocs.org/documentation/test/latest/index.html#test/test@id_completion"><code>completion()</code></a> matcher can be used to test +<code>Futures</code>; it ensures that the test doesn't finish until the <code>Future</code> completes, +and runs a matcher against that <code>Future</code>'s value.</p> +<pre class="dart"><code>import "dart:async"; + +import "package:test/test.dart"; + +void main() { + test("new Future.value() returns the value", () { + expect(new Future.value(10), completion(equals(10))); + }); +} +</code></pre> +<p>The <a href="http://www.dartdocs.org/documentation/test/latest/index.html#test/test@id_throwsA"><code>throwsA()</code></a> matcher and the various <code>throwsExceptionType</code> +matchers work with both synchronous callbacks and asynchronous <code>Future</code>s. They +ensure that a particular type of exception is thrown:</p> +<pre class="dart"><code>import "dart:async"; + +import "package:test/test.dart"; + +void main() { + test("new Future.error() throws the error", () { + expect(new Future.error("oh no"), throwsA(equals("oh no"))); + expect(new Future.error(new StateError("bad state")), throwsStateError); + }); +} +</code></pre> +<p>The <a href="http://www.dartdocs.org/documentation/test/latest/index.html#test/test@id_expectAsync"><code>expectAsync()</code></a> function wraps another function and has two +jobs. First, it asserts that the wrapped function is called a certain number of +times, and will cause the test to fail if it's called too often; second, it +keeps the test from finishing until the function is called the requisite number +of times.</p> +<pre class="dart"><code>import "dart:async"; + +import "package:test/test.dart"; + +void main() { + test("Stream.fromIterable() emits the values in the iterable", () { + var stream = new Stream.fromIterable([1, 2, 3]); + + stream.listen(expectAsync((number) { + expect(number, inInclusiveRange(1, 3)); + }, count: 3)); + }); +} +</code></pre> +<h2>Running Tests with Custom HTML</h2> +<p>By default, the test runner will generate its own empty HTML file for browser +tests. However, tests that need custom HTML can create their own files. These +files have three requirements:</p><ul><li> +<p>They must have the same name as the test, with <code>.dart</code> replaced by <code>.html</code>.</p></li><li> +<p>They must contain a <code>link</code> tag with <code>rel="x-dart-test"</code> and an <code>href</code> + attribute pointing to the test script.</p></li><li> +<p>They must contain <code><script src="packages/test/dart.js"></script></code>.</p></li></ul> +<p>For example, if you had a test called <code>custom_html_test.dart</code>, you might write +the following HTML file:</p> +<pre class="html"><code><!doctype html> +<!-- custom_html_test.html --> +<html> + <head> + <title>Custom HTML Test</title> + <link rel="x-dart-test" href="custom_html_test.dart"> + <script src="packages/test/dart.js"></script> + </head> + <body> + // ... + </body> +</html> +</code></pre> +<h2>Configuring Tests</h2> +<h3>Skipping Tests</h3> +<p>If a test, group, or entire suite isn't working yet and you just want it to stop +complaining, you can mark it as "skipped". The test or tests won't be run, and, +if you supply a reason why, that reason will be printed. In general, skipping +tests indicates that they should run but is temporarily not working. If they're +is fundamentally incompatible with a platform, <a href="#restricting-tests-to-certain-platforms"><code>@TestOn</code>/<code>testOn</code></a> +should be used instead.</p> +<p>To skip a test suite, put a <code>@Skip</code> annotation at the top of the file:</p> +<pre class="dart"><code>@Skip("currently failing (see issue 1234)") + +import "package:test/test.dart"; + +void main() { + // ... +} +</code></pre> +<p>The string you pass should describe why the test is skipped. You don't have to +include it, but it's a good idea to document why the test isn't running.</p> +<p>Groups and individual tests can be skipped by passing the <code>skip</code> parameter. This +can be either <code>true</code> or a String describing why the test is skipped. For example:</p> +<pre class="dart"><code>import "package:test/test.dart"; + +void main() { + group("complicated algorithm tests", () { + // ... + }, skip: "the algorithm isn't quite right"); + + test("error-checking test", () { + // ... + }, skip: "TODO: add error-checking."); +} +</code></pre> +<h3>Timeouts</h3> +<p>By default, tests will time out after 30 seconds of inactivity. However, this +can be configured on a per-test, -group, or -suite basis. To change the timeout +for a test suite, put a <code>@Timeout</code> annotation at the top of the file:</p> +<pre class="dart"><code>@Timeout(const Duration(seconds: 45)) + +import "package:test/test.dart"; + +void main() { + // ... +} +</code></pre> +<p>In addition to setting an absolute timeout, you can set the timeout relative to +the default using <code>@Timeout.factor</code>. For example, <code>@Timeout.factor(1.5)</code> will +set the timeout to one and a half times as long as the default—45 seconds.</p> +<p>Timeouts can be set for tests and groups using the <code>timeout</code> parameter. This +parameter takes a <code>Timeout</code> object just like the annotation. For example:</p> +<pre class="dart"><code>import "package:test/test.dart"; + +void main() { + group("slow tests", () { + // ... + + test("even slower test", () { + // ... + }, timeout: new Timeout.factor(2)) + }, timeout: new Timeout(new Duration(minutes: 1))); +} +</code></pre> +<p>Nested timeouts apply in order from outermost to innermost. That means that +"even slower test" will take two minutes to time out, since it multiplies the +group's timeout by 2.</p> +<h3>Platform-Specific Configuration</h3> +<p>Sometimes a test may need to be configured differently for different platforms. +Windows might run your code slower than other platforms, or your DOM +manipulation might not work right on Safari yet. For these cases, you can use +the <code>@OnPlatform</code> annotation and the <code>onPlatform</code> named parameter to <code>test()</code> +and <code>group()</code>. For example:</p> +<pre class="dart"><code>@OnPlatform(const { + // Give Windows some extra wiggle-room before timing out. + "windows": const Timeout.factor(2) +}) + +import "package:test/test.dart"; + +void main() { + test("do a thing", () { + // ... + }, onPlatform: { + "safari": new Skip("Safari is currently broken (see #1234)") + }); +} +</code></pre> +<p>Both the annotation and the parameter take a map. The map's keys are <a href="#platform-selector-syntax">platform +selectors</a> which describe the platforms for which the +specialized configuration applies. Its values are instances of some of the same +annotation classes that can be used for a suite: <code>Skip</code> and <code>Timeout</code>. A value +can also be a list of these values.</p> +<p>If multiple platforms match, the configuration is applied in order from first to +last, just as they would in nested groups. This means that for configuration +like duration-based timeouts, the last matching value wins.</p> +<h2>Testing With <code>barback</code></h2> +<p>Packages using the <code>barback</code> transformer system may need to test code that's +created or modified using transformers. The test runner handles this using the +<code>--pub-serve</code> option, which tells it to load the test code from a <code>pub serve</code> +instance rather than from the filesystem.</p> +<p>Before using the <code>--pub-serve</code> option, add the <code>test/pub_serve</code> transformer to +your <code>pubspec.yaml</code>. This transformer adds the necessary bootstrapping code that +allows the test runner to load your tests properly:</p> +<pre class="yaml"><code>transformers: +- test/pub_serve: + $include: test/**_test{.*,}.dart +</code></pre> +<p>Note that if you're using the test runner along with <a href="https://www.dartlang.org/polymer/"><code>polymer</code></a>, you +have to make sure that the <code>test/pub_serve</code> transformer comes <em>after</em> the +<code>polymer</code> transformer:</p> +<pre class="yaml"><code>transformers: +- polymer +- test/pub_serve: + $include: test/**_test{.*,}.dart +</code></pre> +<p>Then, start up <code>pub serve</code>. Make sure to pay attention to which port it's using +to serve your <code>test/</code> directory:</p> +<pre class="shell"><code>$ pub serve +Loading source assets... +Loading test/pub_serve transformers... +Serving my_app web on http://localhost:8080 +Serving my_app test on http://localhost:8081 +Build completed successfully +</code></pre> +<p>In this case, the port is <code>8081</code>. In another terminal, pass this port to +<code>--pub-serve</code> and otherwise invoke <code>pub run test:test</code> as normal:</p> +<pre class="shell"><code>$ pub run test:test --pub-serve=8081 -p chrome +"pub serve" is compiling test/my_app_test.dart... +"pub serve" is compiling test/utils_test.dart... +00:00 +42: All tests passed! +</code></pre> \ No newline at end of file