test: make compliance golden tests "thoughtful" and clean up state (#525)

The compliance tests now evaluate test verdicts hierarchically
(Pass > Warn > Fail > Error) rather than enforcing strict JSON
equality against the goldens.

Previously, if the underlying Dart SDK fixed a bug and caused a
test to improve (e.g., from Warn to Pass), the strict equality
check would fail the test suite. Now, the tests will:
- Fail the build if a test regresses (e.g., Pass to Fail).
- Pass the build but emit a GitHub Actions `::warning::` if a
  test improves or changes benignly, prompting maintainers to
  update the goldens (`tool/update_goldens.dart`) to tighten them.

Additionally, the markdown summary check no longer fails the build
if the summary changed due to test improvements or benign differences,
provided there are no regressions in the suite.

Finally, this removes the redundant `_updateGoldens` constant and
its branching logic from `compliance_test.dart` since golden updates
are now handled entirely by the standalone `tool/update_goldens.dart`
script.
2 files changed
tree: 8fbff171754fcaff15482a082b01ce380fc3310e
  1. .github/
  2. pkgs/
  3. tool/
  4. vendor/
  5. .gitignore
  6. .gitmodules
  7. CONTRIBUTING.md
  8. LICENSE
  9. mono_repo.yaml
  10. README.md
README.md

Build Status

About Shelf

Shelf makes it easy to create and compose web servers and parts of web servers. How?

  • Expose a small set of simple types.
  • Map server logic into a simple function: a single argument for the request, the response is the return value.
  • Trivially mix and match synchronous and asynchronous processing.
  • Flexibility to return a simple string or a byte stream with the same model.

It was inspired by Connect for NodeJS and Rack for Ruby.

See the package:shelf readme for more information.

Packages

PackageDescriptionIssuesVersion
shelfA model for web server middleware that encourages composition and easy reuse.issuespub package
shelf_packages_handlerA shelf handler for serving a packages/ directory.issuespub package
shelf_proxyA shelf handler for proxying HTTP requests to another server.issuespub package
shelf_routerA convenient request router for the shelf web-framework, with support for URL-parameters, nested routers and routers generated from source annotations.issuespub package
shelf_router_generatorA package:build-compatible builder for generating request routers for the shelf web-framework based on source annotations.issuespub package
shelf_staticStatic file server support for the shelf package and ecosystem.issuespub package
shelf_test_handlerA Shelf handler that makes it easy to test HTTP interactions.issuespub package
shelf_web_socketA shelf handler that wires up a listener for every connection.issuespub package
_shelf_complianceHTTP/1.1 compliance and hardening tests.issues(not published)

Publishing automation

For information about our publishing automation and release process, see https://github.com/dart-lang/ecosystem/wiki/Publishing-automation.