Content hashing of archives (#3482)

Compute a hash of each downloaded archive and store it in: $PUB_CACHE/hosted/<hosted-url>/.hashes/<package>-<version>.sha256 (details here still subject to change)

New optional field in the package listing api for the server to provide the content-hash. If that is provided - it is verified against the downloaded archive.

When writing a pubspec.lock file, the sha256 is included in the description of each hosted package.

On pub get If the description of a package from pubspec.lock doesn't match the one in the cache, the archive is redownloaded - if the hash still doesn't match, the resolution fails with an error.

Has been moved to a follow-up PR Introduce a new option dart pub get --enforce-lockfile A mode that will NOT modify pubspec.lock. That means:

won't add hashes if missing,
will refuse to resolve if pubspec.yaml isn't satisfied,
will refuse to resolve if hashes don't match cached hashes.
will refuse to resolve if pubspec.lock is missing
will verify that the extracted package content matches the contents of the original archive.
This is useful when deploying to production.
Fixes: dart pub get --pristine/--locked #2890 and locked option in pubspec.yaml #2905

An unfortunate side-effect of this change is that all already downloaded packages will be re-downloaded (because we don't store the archives, only the extracted files) to compute their hashes.
61 files changed
tree: 9d0ae231f3678aace5a6ad99f420140f821f417f
  1. .github/
  2. bin/
  3. doc/
  4. lib/
  5. test/
  6. tool/
  7. .gitallowed
  8. .gitignore
  9. .status
  10. .test_config
  11. analysis_options.yaml
  12. AUTHORS
  13. CONTRIBUTING.md
  14. dart_test.yaml
  15. LICENSE
  16. pubspec.yaml
  17. README.md
README.md

Build Status

Pub is the package manager for Dart.

Contributing to pub

Thanks for being interested in contributing to pub! Contributing to a new project can be hard: there's a lot of new code and practices to learn. This document is intended to get you up and running as quickly as possible. For more information, see the pub tool documentation.

The first step towards contributing is to contact the pub dev team and let us know what you‘re working on, so we can be sure not to start working on the same thing at the same time. Open an issue letting us know that you’re interested in contributing and what you plan on working on. This will also let us give you specific advice about where to start.

Organization

Pub isn‘t a package, but it’s organized like one. It has four top-level directories:

  • lib/ contains the implementation of pub. Currently, it's all in lib/src/, since there are no libraries intended for public consumption.

  • test/ contains the tests for pub.

  • bin/ contains pub.dart, the entrypoint script that's run whenever a user types “pub” on the command line or runs it in the Dart editor. This is usually run through shell scripts in sdk/bin at the root of the Dart repository.

It's probably easiest to start diving into the codebase by looking at a particular pub command. Each command is encapsulated in files in lib/src/command/.

Running pub

To run pub from the Git repository, run:

dart bin/pub.dart

Testing pub

Before any change is made to pub, all tests should pass. To run a pub test, run:

dart tool/test.dart test/path/to_test.dart

To run all tests at once, run:

dart tool/test.dart

Changes to pub should be accompanied by one or more tests that exercise the new functionality. When adding a test, the best strategy is to find a similar test in test/ and follow the same patterns.

Pub tests come in two basic forms. The first, which is usually used to unit test classes and libraries internal to pub, has many tests in a single file. This is used when each test will take a short time to run. For example, test/version_test.dart contains unit tests for pub's Version class.

The other form, used by most pub tests, is usually used for integration tests of user-visible pub commands. Each test has a file to itself, which is named after the test description. This is used when tests can take a long time to run to avoid having the tests time out when running on the build bots. For example, tests/get/hosted/get_transitive_test.dart tests the resolution of transitive hosted dependencies when using dart pub get/flutter pub get.

Landing your patch

All patches to official Dart packages, including to pub, need to undergo code review before they're submitted. The full process for putting up your patch for review is documented elsewhere.