Target to restrict TestOn annotation to library level.spawnHybridUri support for package: uris.4.x.github reporter option for use with GitHub Actions.github test reporter the default when we detect we're running on GitHub Actions.dart2js-path command line argument.packages.vm_service package.analyzer constraint to >=2.0.0 <4.0.0.--ignore-timeouts command line flag, which disables all timeouts for all tests. This can be useful when debugging, so tests don't time out during debug sessions.chrome on windows to diagnose intermittent failures.TestFailure implements Exception for compatibility with only_throw_exceptions.chain-stack-traces flag, a single log will now appear at the end.name, full-name, line, and col on test paths, which will apply the filters to only those test suites.--name) are also still respected and must match.line and col will match if any frame from the test trace matches (the test trace is the current stack trace where test is invoked).markTestSkipped outside of a test.host.dart.js file.allow_duplicate_test_names parameter to the remote listener.test_core.package:matcher.dynamic to Object?.stringContainsInOrder to account for repetitions and empty strings.analyzer constraint to >=1.0.0 <3.0.0.--test-randomize-ordering-seed argument.time field to the json reporters allSuites event type so that all event types can be unified.test_core.printOnFailure is called from outside a test zone.7.0.0).--use-data-isolate-strategy flag if you run into issues, but please also file a bug.--chain-stack-traces flag.phantomjs support completely, it was previously broken.expectAsync function type checks.scaffolding.dart, and expect.dart to allow importing a subset of the normal surface area.--chain-stack-traces flag.spawnHybridCode to default to the current packages language version.test_core and test_api deps.test_core dependency to 0.3.14.web_socket_channel dependency to support latest.test_core dependency to 0.3.13.1.0.0.test_api for stable null safety.1.x.x).6.x.x).0.15.x).2.x).2.x).spawnHybridUri on windows.node platform.package:yaml version 3.x.x.package:analyzer version 0.41.x.spawnHybridUri to respect language versioning of the spawned uri.>=2.12.0-0 <3.0.0 based on beta release guidelines.markTestSkipped API.2.10 stable and 2.11.0-dev SDKs.SILENT_OBSERVATORY environment variable to run with debugging and the JSON reporter.--help output.0.40.x of analyzer.Fake class, available through package:test_api/fake.dart. This was previously part of the Mockito package, but with null safety it is useful enough that we decided to make it available through package:test. In a future release it will be made available directly through package:test_api/test_api.dart (and hence through package:test_core/test_core.dart and package:test/test.dart).spawnHybridUri on windows.package:analyzer version 0.41.x.spawnHybridUri to respect language versioning of the spawned uri.matcher version 0.12.9 which improves the mismatch description for deep collection equality matchers and TypeMatcher.test_core which resolves an issue with the latest package:meta.IPv6 and IPv4 sockets for the node platform.package:coverage.test_core to 0.3.6.RunnerSuite.--pause-after-load.test_core to 0.3.4.test_core to 0.3.3.package_resolver dependency for the package_config dependency which is lower level.spawnHybrid APIs.dart2js compiled tests.close() method on that controller would hang.tearDown callback following a failing test case.2.4.0 for safer usage of for-loop elements.PhantomJS and provide warning when used. Support for PhantomJS will be removed in version 2.0.0.README.md for usage details.test_api 0.2.13 to work around a bug in the SDK version 2.3.0.file_reporters configuration option and --file-reporter CLI option to allow specifying a separate reporter that writes to a file instead of stdout.customHtmlTemplateFile configuration option to allow sharing an html template between testspackage:test_core.package:test_api.tearDownAll.package:test_core.package:test_api.package:test_core.package:analyzer.package:matcher. Improves output for instances of private classes.package:test_api and package:test_core.solo tests that are not run will now be reported as skipped.test_core.--total-shards specifies how many shards the suite should be split into--shard-index specifies which shard should be run--debug flag for running the VM/Chrome in debug mode.test_core and test_api.test_core.--disable-dev-shm-usage to the default Chrome flags.test_core and test_api.analyzer version 0.38.x.--server-mode to dart2js instead of --categories=Server to fix a warning about the flag deprecation.pub_semver.Utf8Decoder and the node platform.test_core.package:analyzer.expectAsync*.package:test_core.dart:isolate imports on code loaded in tests.stream_channel version 2.0.0.analyzer version 0.36.x.isA() to create TypeMatcher instances in a more fluent way.isCastError.greaterThan now compares with the > operator rather not < and not =. This could cause tests which relied on this bug to start failing.analyzer version 0.35.x.>=2.1.0.test_core and test_api.test_core and test_api.package:test_core for core functionality.package:test_api for core functionality.dart:io is not imported from the main test.dart entrypoint unless it is available.const.const on collection literals.root_line, root_column, and root_url in the JSON reported may not be populated correctly on Windows.--precompiled, the test runner now allows symlinks to reach outside the precompiled directory. This allows more efficient creation of precompiled directories (using symlinks instead of copies).<3.0.0.--precompiled flag we will now first check for a <original-test-path>.vm_test.vm.app.dill file, and if present load that directly in the isolate. Otherwise the <original-test-path>.vm_test.dart file will be used.pid field to the StartEvent in the json runner containing the pid of the VM process running the tests.0.12.42. We are simply signalling to users that this is a well supported package and is the preferred way to write Dart tests.Add support for solo test and group. When the argument is true only tests and groups marked as solo will be run. It is still recommended that users instead filter their tests by using the runner argument -n.
Updated exported package:matcher to 0.12.3 which includes these updates:
Many improvements to TypeMatcher
Can now be used directly as const TypeMatcher<MyType>().
Added a type parameter to specify the target Type.
name constructor parameter optional and marked it deprecated. It's redundant to the type parameter.Migrated all isType matchers to TypeMatcher.
Added a having function that allows chained validations of specific features of the target type.
/// Validates that the object is a [RangeError] with a message containing /// the string 'details' and `start` and `end` properties that are `null`. final _rangeMatcher = isRangeError .having((e) => e.message, 'message', contains('details')) .having((e) => e.start, 'start', isNull) .having((e) => e.end, 'end', isNull);
Deprecated the isInstanceOf class. Use TypeMatcher instead.
Improved the output of Matcher instances that fail due to type errors.
root_line, root_column, and root_url. These will be present if url is not the same as the suite url, and will represent the location in the original test suite from which the call to test originated.stdout.supportsAnsiEscapes instead of based on platform (previously both were disabled on windows).neverCalled a type that works in Dart 2 semantics.package:analyzer 0.32.0.pub_serve.dart entrypoint. This is not being treated as a breaking change because the minimum sdk constraint now points to an sdk which does not support pub serve or barback any more anyways.barback.dartium and content-shell platforms since those are removed from the Dart 2 SDK.--precompiled node tests in subdirectories.--precompiled flag is now supported for the vm platform and the node platform.--precompiled flag now serves all sources directly from the precompiled directory, and will never attempt to do its own compilation.Pass --categories=Server to dart2js when compiling tests for Node.js. This tells it that dart:html is unavailable.
Don't crash when attempting to format stack traces when running via dart path/to/test.dart.
Add an include configuration field which specifies the path to another configuration file whose configuration should be used.
Add a google platform selector variable that‘s only true on Google’s internal infrastructure.
Add a headless configuration option for Chrome.
Re-enable headless mode for Chrome by default.
Don't hang when a Node.js test fails to compile.
--excluded-tags.Platform selectors for operating systems now work for Node.js tests (#742).
fail() is now typed to return Null, so it can be used in the same places as a raw throw.
Run Chrome in headless mode unless debugging is enabled.
Node.js tests can now import modules from a top-level node_modules directory, if one exists.
Raw console.log() calls no longer crash Node.js tests.
When a browser crashes, include its standard output in the error message.
Add a pumpEventQueue() function to make it easy to wait until all asynchronous tasks are complete.
Add a neverCalled getter that returns a function that causes the test to fail if it's ever called.
When addTearDown() is called within a call to setUpAll(), it runs its callback after all tests instead of running it after the setUpAll() callback.
When running in an interactive terminal, the test runner now prints status lines as wide as the terminal and no wider.
stack_trace. Now 1.6.0.spawnHybridUri() function now allows root-relative URLs, which are interpreted as relative to the root of the package.Add a override_platforms configuration field which allows test platforms' settings (such as browsers' executables) to be overridden by the user.
Add a define_platforms configuration field which makes it possible to define new platforms that use the same logic as existing ones but have different settings.
spawnHybridUri() now interprets relative URIs correctly in browser tests.async 2.0.0.loadException to the server.stdout and stdin. This resolves test flakiness, especially in Travis with the Precise image.deserializeTimeout.FORCE_TEST_EXIT is set in the environment.analyzer.node platform for compiling tests to JavaScript and running them on Node.js.fold_stack_frames field for dart_test.yaml. This will allow users to customize which packages' frames are folded.Support args 1.0.0.
Run tear-down callbacks in the same error zone as the test function. This makes it possible to safely share Futures and Streams between tests and their tear-downs.
Add a retry option to test() and group() functions, as well as @Retry() annotation for test files and a retry configuration field for dart_test.yaml. A test with reties enabled will be re-run if it fails for a reason other than a TestFailure.
Add a --no-retry runner flag that disables retries of failing tests.
Fix a “concurrent modification during iteration” error when calling addTearDown() from within a tear down.
Add a doesNotComplete matcher that asserts that a Future never completes.
throwsA() and all related matchers will now match functions that return Futures that emit exceptions.
Respect onPlatform for groups.
Only print browser load errors once per browser.
Gracefully time out when attempting to deserialize a test suite.
Now support v0.30.0 of pkg/analyzer
The test executable now does a “hard exit” when complete to ensure lingering isolates or async code don't block completion. This may affect users trying to use the Dart service protocol or observatory.
--chain-stack-traces to conditionally chain stack traces.Fixed more blockers for compiling with dev_compiler.
Dartfmt the entire repo.
Note: 0.12.20+5-0.12.20+7 were tagged but not officially published.
dev_compiler.--pause-after-load no longer deadlocks with recent versions of Chrome.
Fix Dartified stack traces for JS-compiled tests run through pub serve.
Tighten the dependency on stream_channel to reflect the APIs being used.
Use a 1024 x 768 iframe for browser tests.
Breaking change: The expect() method no longer returns a Future, since this broke backwards-compatibility in cases where a void function was returning an expect() (such as void foo() => expect(...)). Instead, a new expectLater() function has been added that return a Future that completes when the matcher has finished running.
The verbose parameter to expect() and the formatFailure() function are deprecated.
throws*() and prints(), can be used with synchronous matcher operators like isNot().Added the StreamMatcher class, as well as several built-in stream matchers: emits(), emitsError(), emitsDone, mayEmit(), mayEmitMultiple(), emitsAnyOf(), emitsInOrder(), emitsInAnyOrder(), and neverEmits().
expect() now returns a Future for the asynchronous matchers completes, completion(), throws*(), and prints().
Add a printOnFailure() method for providing debugging information that's only printed when a test fails.
Automatically configure the term_glyph package to use ASCII glyphs when the test runner is running on Windows.
Deprecate the throws matcher in favor of throwsA().
Deprecate the Throws class. These matchers should only be constructed via throwsA().
expectAsync() function. The deprecation caused it to fail to support functions that take arguments.Add an addTearDown() function, which allows tests to register additional tear-down callbacks as they're running.
Add the spawnHybridUri() and spawnHybridCode() functions, which allow browser tests to run code on the VM.
Fix the new expectAsync functions so that they don't produce analysis errors when passed callbacks with optional arguments.
expectAsync and expectAsyncUntil, since they currently can't be made to work cleanly in strong mode. They are replaced with separate methods for each number of callback arguments:expectAsync0, expectAsync1, ... expectAsync6, andexpectAsyncUntil0, expectAsyncUntil1, ... expectAsyncUntil6.analyzer.on_platform was defined in a config file.--help documentation.analyzer.nestingMiddleware to lib/src/util/path_handler.dart to enable a cleaner separation between test-runner files and test writing files.packages/ directory.Add a skip parameter to expect(). Marking a single expect as skipped will cause the test itself to be marked as skipped.
Add a --run-skipped parameter and run_skipped configuration field that cause tests to be run even if they're marked as skipped.
yaml.Declare support for version 1.18 of the Dart SDK.
Use the latest collection package.
collection package.Add support for a global configuration file. On Windows, this file defaults to %LOCALAPPDATA%\DartTest.yaml. On Unix, it defaults to ~/.dart_test.yaml. It can also be explicitly set using the DART_TEST_CONFIG environment variable. See the configuration documentation for details.
The --name and --plain-name arguments may be passed more than once, and may be passed together. A test must match all name constraints in order to be run.
Add names and plain_names fields to the package configuration file. These allow presets to control which tests are run based on their names.
Add include_tags and exclude_tags fields to the package configuration file. These allow presets to control which tests are run based on their tags.
Add a pause_after_load field to the package configuration file. This allows presets to enable debugging mode.
Add support for test presets. These are defined using the presets field in the package configuration file. They can be selected by passing --preset or -P, or by using the add_presets field in the package configuration file.
Add an on_os field to the package configuration file that allows users to select different configuration for different operating systems.
Add an on_platform field to the package configuration file that allows users to configure all tests differently depending on which platform they run on.
Add an ios platform selector variable. This variable will only be true when the test executable itself is running on iOS, not when it's running browser tests on an iOS browser.
shelf_web_socket 0.2.0.Add a tags field to the package configuration file that allows users to provide configuration for specific tags.
The --tags and --exclude-tags command-line flags now allow boolean selector syntax. For example, you can now pass --tags "(chrome || firefox) && !slow" to select quick Chrome or Firefox tests.
Re-add help output separators.
Tighten the constraint on args.
args constraint for the features it used; this version will help ensure that users who can't use args 0.13.1 will get a working version of test.dart_test.yaml.Add SuiteEvent to the JSON reporter, which reports data about the suites in which tests are run.
Add AllSuitesEvent to the JSON reporter, which reports the total number of suites that will be run.
Add Group.testCount to the JSON reporter, which reports the total number of tests in each group.
Organize the --help output into sections.
Add a --timeout flag.
Add the ability to re-run tests while debugging. When the browser is paused at a breakpoint, the test runner will open an interactive console on the command line that can be used to restart the test.
Add support for passing any object as a description to test() and group(). These objects will be converted to strings.
Add the ability to tag tests. Tests with specific tags may be run by passing the --tags command-line argument, or excluded by passing the --exclude-tags parameter.
This feature is not yet complete. For now, tags are only intended to be added temporarily to enable use-cases like focusing on a specific test or group. Further development can be followed on the issue tracker.
Wait for a test's tear-down logic to run, even if it times out.
http_parser 2.0.0.http_multi_server 2.0.0.Add a machine-readable JSON reporter. For details, see the protocol documentation.
Skipped groups now properly print skip messages.
setUpAll() and tearDownAll().Add setUpAll() and tearDownAll() methods that run callbacks before and after all tests in a group or suite. Note that these methods are for special cases and should be avoided—they make it very easy to accidentally introduce dependencies between tests. Use setUp() and tearDown() instead if possible.
Allow setUp() and tearDown() to be called multiple times within the same group.
When a tearDown() callback runs after a signal has been caught, it can now schedule out-of-band asynchronous callbacks normally rather than having them throw exceptions.
Don't show package warnings when compiling tests with dart2js. This was accidentally enabled in 0.12.2, but was never intended.
tearDown() callback throws an error, outer tearDown() callbacks are still executed.pub serve on Dartium or content shell.http_parser 1.0.0.1.13.0.--pub-serve.1.11.0 and shelf >= 0.6.0, allowing test to remove various hacks and workarounds.Add a --pause-after-load flag that pauses the test runner after each suite is loaded so that breakpoints and other debugging annotations can be added. Currently this is only supported on browsers.
Add a Timeout.none value indicating that a test should never time out.
The dart-vm platform selector variable is now true for Dartium and content shell.
The compact reporter no longer prints status lines that only update the clock if they would get in the way of messages or errors from a test.
The expanded reporter no longer double-prints the descriptions of skipped tests.
analyzer to include 0.26.0.collection package.This version was unpublished due to issue 287.
Properly report load errors caused by failing to start browsers.
Substantially increase browser timeouts. These timeouts are the cause of a lot of flakiness, and now that they don‘t block test running there’s less harm in making them longer.
This version was unpublished due to issue 287.
This version was unpublished due to issue 287.
The compact reporter will update the timer every second, rather than only updating it occasionally.
The compact reporter will now print the full, untruncated test name before any errors or prints emitted by a test.
The expanded reporter will now always print the full, untruncated test name.
This version was unpublished due to issue 287.
This version was unpublished due to issue 287.
Improve the display of syntax errors in VM tests.
Work around a Firefox bug. Computed styles now work in tests on Firefox.
If a future matched against the completes or completion() matcher throws an error, that error is printed directly rather than being wrapped in a string. This allows such errors to be captured using the Zone API and improves formatting.
Improve support for Polymer tests. This fixes a flaky time-out error and adds support for Dartifying JavaScript stack traces when running Polymer tests via pub serve.
In order to be more extensible, all exception handling within tests now uses the Zone API.
Add a heartbeat to reset a test's timeout whenever the test interacts with the test infrastructure.
expect(), expectAsync(), and expectAsyncUntil() throw more useful errors if called outside a test body.
Convert JavaScript stack traces into Dart stack traces using source maps. This can be disabled with the new --js-trace flag.
Improve the browser test suite timeout logic to avoid timeouts when running many browser suites at once.
--verbose-trace flag to include core library frames in stack traces.0.12.0 adds support for a test runner, which can be run via pub run test:test (or pub run test in Dart 1.10). By default it runs all files recursively in the test/ directory that end in _test.dart and aren't in a packages/ directory.
The test runner supports running tests on the Dart VM and many different browsers. Test files can use the @TestOn annotation to declare which platforms they support. For more information on this and many more new features, see the README.
As part of moving to a runner-based model, most test configuration is moving out of the test file and into the runner. As such, many ancillary APIs have been removed. These APIs include skip_ and solo_ functions, Configuration and all its subclasses, TestCase, TestFunction, testConfiguration, formatStacks, filterStacks, groupSep, logMessage, testCases, BREATH_INTERVAL, currentTestCase, PASS, FAIL, ERROR, filterTests, runTests, ensureInitialized, setSoloTest, enableTest, disableTest, and withTestEnvironment.
FailureHandler, DefaultFailureHandler, configureExpectFailureHandler, and getOrCreateExpectFailureHandler which used to be exported from the matcher package have also been removed. They existed to enable integration between test and matcher that has been streamlined.
A number of APIs from matcher have been into test, including: completes, completion, ErrorFormatter, expect,fail, prints, TestFailure, Throws, and all of the throws methods. Some of these have changed slightly:
expect no longer has a named failureHandler argument.
expect added an optional formatter argument.
completion argument id renamed to description.
vm_config.dart and html_config.dart libraries.Merge in the last 0.11.x release of matcher to allow projects to use both test and unittest without conflicts.
Fix running individual tests with HtmlIndividualConfiguration when the test name contains URI-escaped values and is provided with the group query parameter.
matcher.matcher.reason named argument to expectAsync and expectAsyncUntil, which has the same definition as expect's reason argument.matcher version.stack_trace.expectAsync0, expectAsync1, and expectAsync2 - use expectAsync insteadexpectAsyncUntil0, expectAsyncUntil1, and expectAsyncUntil2 - use expectAsyncUntil insteadguardAsync - no longer neededprotectAsync0, protectAsync1, and protectAsync2 - no longer neededmatcher.dart and mirror_matchers.dart have been removed. They are now in the matcher package.mock.dart has been removed. It is now in the mock package.mock.matcher.dart and mirror_matchers.dart are now in the matcher package.mock.dart is now in the mock package.equals now allows a nested matcher as an expected list element or map value when doing deep matching.expectAsync and expectAsyncUntil now support up to 6 positional arguments and correctly handle functions with optional positional arguments with default values.Zone. This ensures that any exceptions that occur is async operations are reported back to the source test case.guardAsync, protectAsync0, protectAsync1, and protectAsync2Zone addresses the need for these methods.expectAsync replaces the now deprecated expectAsync0, expectAsync1 and expectAsync2expectAsyncUntil replaces the now deprecated expectAsyncUntil0, expectAsyncUntil1 and expectAsyncUntil2TestCase:setUp, tearDown, testFunctionenabled is now get-onlypass, fail, errorinteractive_html_config.dart has been removed.runTests, tearDown, setUp, test, group, solo_test, and solo_group now throw a StateError if called while tests are running.rerunTests has been removed.