Update contributing guides (#8718)
diff --git a/BETA_TESTING.md b/BETA_TESTING.md index 4c31d90..dfae715 100644 --- a/BETA_TESTING.md +++ b/BETA_TESTING.md
@@ -35,20 +35,23 @@ In your terminal, navigate to a directory where you want to clone DevTools: `cd some/directory`. This folder must not already contain a folder named 'devtools'. - - **To clone flutter/devtools**: - - Clone the DevTools repo: `git clone git@github.com:flutter/devtools.git` - - If you haven't already, you may need to + + If you do not already have an SSH key set up for your machine, you may need to [generate a new SSH key](https://docs.github.com/en/github/authenticating-to-github/connecting-to-github-with-ssh) - to connect to Github with SSH. + to connect to Github. + + **To clone `flutter/devtools`**: + ```shell + git clone git@github.com:flutter/devtools.git + ``` **To clone your fork of flutter/devtools**: - [Fork](https://docs.github.com/en/get-started/quickstart/fork-a-repo) the DevTools repo to your own Github account. - - Clone your fork of the DevTools repo: `git clone git@github.com:your_github_account/devtools.git` - - If you haven't already, you may need to - [generate a new SSH key](https://docs.github.com/en/github/authenticating-to-github/connecting-to-github-with-ssh) - to connect to Github with SSH. + - Clone your fork of the DevTools repo: + ```shell + git clone git@github.com:<your_github_account>/devtools.git + ``` - Make sure to [configure Git to keep your fork in sync](https://docs.github.com/en/get-started/quickstart/fork-a-repo#configuring-git-to-sync-your-fork-with-the-upstream-repository) with the upstream DevTools repo. @@ -57,7 +60,7 @@ - Adding the `devtools/tool/bin` folder to your `PATH` environment variable: - **MacOS Users** - add the following to your `~/.zshrc` file (or `~/.bashrc`, `~/.bash_profile` if you use Bash), - replacing `<DEVTOOLS_DIR>` with the local path to your DevTools repo: + replacing `<DEVTOOLS_DIR>` with the absolute path to your DevTools repo: ``` export PATH=$PATH:<DEVTOOLS_DIR>/tool/bin @@ -66,7 +69,7 @@ - Open "Edit environment variables for your account" from Control Panel - Locate the `Path` variable and click **Edit** - Click the **New** button and paste in `<DEVTOOLS_DIR>/tool/bin`, replacing `<DEVTOOLS_DIR>` - with the local path to your DevTools repo. + with the absolute path to your DevTools repo. Explore the commands and helpers that the `dt` provides by running `dt -h`. @@ -76,7 +79,7 @@ `devtools` directory (this will delete any local changes you have made to your DevTools clone): ```bash git checkout master -git reset --hard origin/master +git reset --hard origin/master # (use upstream/master instead if you cloned a fork of DevTools) dt update-flutter-sdk dt pub-get --only-main --upgrade
diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 7cfda95..32639d7 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md
@@ -55,20 +55,22 @@ instructions for how to [update your PATH](https://flutter.dev/to/update-macos-path)). Typing `which flutter` and `which dart` (or `where.exe flutter` and `where.exe dart` for Windows) -into your terminal should print the path to the binaries from Flutter SDK you cloned from GitHub. +into your terminal should print the path to the binaries from the Flutter SDK you cloned from GitHub. 1. [Fork](https://docs.github.com/en/get-started/quickstart/fork-a-repo) the DevTools repo to your -own Github account, and then clone it using SSH. If you haven't already, you may need to +own Github account, and then clone it using SSH. + - If you haven't already, you may need to [generate a new SSH key](https://docs.github.com/en/github/authenticating-to-github/connecting-to-github-with-ssh) to connect to Github with SSH. -2. Make sure to [configure Git to keep your fork in sync](https://docs.github.com/en/get-started/quickstart/fork-a-repo#configuring-git-to-sync-your-fork-with-the-upstream-repository) + - Make sure to [configure Git to keep your fork in sync](https://docs.github.com/en/get-started/quickstart/fork-a-repo#configuring-git-to-sync-your-fork-with-the-upstream-repository) with the upstream DevTools repo. -3. Ensure that you have access to the DevTools repo management tool exectuable, `dt`: - - Running `flutter pub get` on the `devtools/tool` directory - - Adding the `devtools/tool/bin` folder to your `PATH` environment variable: + +2. Ensure that you have access to the DevTools repo management tool exectuable, `dt`: + - Run `flutter pub get` on the `devtools/tool` directory + - Add the `devtools/tool/bin` folder to your `PATH` environment variable: - **MacOS Users** - add the following to your `~/.zshrc` file (or `~/.bashrc`, `~/.bash_profile` if you use Bash), - replacing `<DEVTOOLS_DIR>` with the local path to your DevTools repo: + replacing `<DEVTOOLS_DIR>` with the absolute path to your DevTools repo: ``` export PATH=$PATH:<DEVTOOLS_DIR>/tool/bin @@ -77,10 +79,11 @@ - Open "Edit environment variables for your account" from Control Panel - Locate the `Path` variable and click **Edit** - Click the **New** button and paste in `<DEVTOOLS_DIR>/tool/bin`, replacing `<DEVTOOLS_DIR>` - with the local path to your DevTools repo. + with the absolute path to your DevTools repo. Explore the commands and helpers that `dt` provides by running `dt -h`. -4. **Optional:** enable and activate DCM (Dart Code Metrics) - see the [DCM section below](#enable-and-activate-dcm-dart-code-metrics) + +3. **Optional:** enable and activate DCM (Dart Code Metrics) - see the [DCM section below](#enable-and-activate-dcm-dart-code-metrics) #### Set up your IDE @@ -93,25 +96,36 @@ ### Workflow for making changes -1. Change your local Flutter SDK to the latest flutter candidate branch: `dt update-flutter-sdk --from-path` +1. Change your local Flutter SDK to the latest flutter candidate branch: + ```sh + dt update-flutter-sdk --update-on-path + ``` + > Warning: this will delete any local changes in your Flutter SDK you checked out from git. -> Note: Until https://github.com/flutter/devtools/issues/7939 is fixed, run -`dt update-flutter-sdk --use-cache` instead. +2. Create a branch from your cloned DevTools repo: + ```sh + git checkout -b myBranch + ``` -2. Create a branch from your cloned DevTools repo: `git checkout -b myBranch` -3. Ensure your branch, dependencies, and generated code are up-to-date: `dt sync` -4. Implement your changes, and commit to your branch: `git commit -m “description”` - - If your improvement is user-facing, [document it](packages/devtools_app/release_notes/README.md) in the same PR. -5. Push to your branch to GitHub: `git push origin myBranch` +3. Ensure your branch, dependencies, and generated code are up-to-date: + ```sh + dt sync + ``` + +4. Implement your changes, and commit to your branch: + ```sh + git commit -m “description” + ``` + If your improvement is user-facing, [document it](packages/devtools_app/release_notes/README.md) in the same PR. + +5. Push to your branch to GitHub: + ```sh + git push origin myBranch + ``` + 6. Navigate to the [Pull Requests](https://github.com/flutter/devtools/pulls) tab in the main [DevTools repo](https://github.com/flutter/devtools). You should see a popup to create a pull -request from the branch in your cloned repo to the DevTools master branch. Create a pull request. - - **Running the Dart Code Metrics Github workflow:** any PRs that change Dart code require the - Dart Code Metrics workflow to be run before being submitted. To trigger the workflow, add the - label `run-dcm-workflow` to your PR. If you don't have permission to add the label, your - reviewer can add it for you. - - Any DCM errors will be caught by the workflow. Fix them and push up your changes. To trigger - the DCM workflow to run again, you will need to remove and then re-add the `run-dcm-workflow` label. +request from the branch in your cloned repo to the DevTools `master` branch. Create a pull request. ### Keeping your fork in-sync @@ -128,13 +142,17 @@ - To update DCM to the same version as on GitHub bots with apt-get or brew: - 1. Locate, copy and run apt-get command searching by searching for `install dcm` in [build.yaml](https://github.com/flutter/devtools/blob/master/.github/workflows/build.yaml) + 1. Locate, copy and run the `apt-get` command searching by searching for + "install dcm" in [build.yaml](https://github.com/flutter/devtools/blob/master/.github/workflows/build.yaml). - 2. Locate version on bots by searching for `install dcm` in [build.yaml](https://github.com/flutter/devtools/blob/master/.github/workflows/build.yaml) and run `brew install cqlabs/dcm/dcm@<version on bots without -1>` + 2. Using the DCM version you just copied in the previous step (without the `-1` suffix), install + `dcm` using homebrew: `brew install cqlabs/dcm/dcm@<version on bots without -1>` - You can check you current local version with `dcm --version`. + You can check your local version to verify it matches the version in + [build.yaml](https://github.com/flutter/devtools/blob/master/.github/workflows/build.yaml): `dcm --version`. - If version of DCM on bots is outdated, consider to submit a PR to refresh the version on bots. + If the version of DCM used on the bots is outdated, consider contributing a + PR to update the version on the bots to the latest. ## Running and debugging DevTools @@ -175,17 +193,17 @@ ``` Then you can run DevTools with the server by running the following from anywhere under the `devtools/` directory: -1. To run DevTools in release mode, served with the DevTools server (this emulates the production environment): +1. To run the DevTools web app in release mode, served with the DevTools server (this emulates the production environment): ``` dt serve ``` -2. To run DevTools in debug mode with full debugging support and a connection to a live DevTools server: +2. To run the DevTools web app in debug mode, with full debugging support, and with a connection to a live DevTools server: ```sh dt run ``` Option 2 is useful for a quicker development cycle. The DevTools build time will be faster, and you will be -able to connect the DevTools web app to an IDE or DevTools for debugging purposes. +able to connect the DevTools web app to an IDE or another DevTools instance for debugging purposes. To see the full list of arguments available for either command, please pass the `-h` flag. @@ -201,11 +219,13 @@ ```js "dart.customDevTools": { - "path": "/path/to/devtools", + "path": "/absolute/path/to/devtools", "env": { - "LOCAL_DART_SDK": "/path/to/sdk" - // Path to the version that Flutter DevTools is pinned to. - "FLUTTER_ROOT": "/path/to/devtools/tool/flutter-sdk" + "LOCAL_DART_SDK": "/absolute/path/to/sdk" + // Path to the Flutter SDK that will be used to build DevTools. This may + // be the path to the included Flutter SDK under the tool/ directory or + // the path to your local Flutter SDK git checkout. + "FLUTTER_ROOT": "/absolute/path/to/devtools/tool/flutter-sdk" }, "args": [ // Arguments that will be passed along to the `dt serve` command. @@ -228,35 +248,34 @@ ### Connect DevTools to a test application -For working on most DevTools tools, a connection to a running Dart or Flutter app is required. Run any Dart of Flutter app of your choice to -connect it to DevTools. Consider running the [Flutter gallery](https://github.com/flutter/gallery#running) app, as it has plenty of interesting +For working on most DevTools tools, a connection to a running Dart or Flutter app is required. Run any Dart or Flutter app of your choice to +connect it to DevTools. Consider running [veggieseasons](https://github.com/flutter/samples/tree/main/veggieseasons) or another Flutter sample since those apps have plenty of interesting code to debug. -1. Run your Dart or Flutter app +1. Run your Dart or Flutter app. > Note: some DevTools features may be unavailable depending on the test app platform (Flutter native, Flutter web, Dart CLI, etc.) or run mode (debug, profile) you choose. -2. Copy the URI printed to the command line (you will use this uri to connect to DevTools) +2. Copy the URI printed to the command line (you will use this URI to connect to DevTools). ``` "A Dart VM Service on iPhone 14 Pro Max is available at: <copy-this-uri>" ``` -3. Paste this URI into the connect dialog in DevTools and click "Connect" +3. Paste this URI into the connect dialog in DevTools and click "Connect".  ### Running DevTools on Flutter Desktop For a faster development cycle with hot reload, you can run DevTools on Flutter desktop. Some DevTools -features only work on the web, like the embedded Perfetto trace viewer, DevTools extensions, or DevTools -analytics, but the limitations on the desktop app are few. +features only work on the web, like the embedded Perfetto trace viewer, DevTools extensions, or WASM support, +but the limitations on the desktop app are few. -To run DevTools with the desktop embedder, you can run with either of the following from `devtools/packages/devtools_app`: - -- `flutter run -d macos` -- `flutter run -d linux` +To run DevTools with the desktop embedder, you can run `flutter run -d macos` from `devtools/packages/devtools_app`, +or you can run DevTools from your IDE with the `macOS` device selected. If this fails, you may need to run `flutter create .` from `devtools/packages/devtools_app` to generate -the updated files for your platform. If you want to run DevTools on Flutter desktop for Windows, you will -need to generate the files for this platform using the same command, and then run using `flutter run -d windows`. +the updated files for your platform. If you want to run DevTools on Flutter desktop for Windows or Linux, +you will need to generate the files for this platform using the `flutter create .` command, and then run using +`flutter run -d <windows or linux>`. ### Enable and activate DCM (Dart Code Metrics) @@ -275,11 +294,14 @@ 3. Install the extension for your IDE. If you use VS Code, you can get it from [the marketplace](https://marketplace.visualstudio.com/items?itemName=dcmdev.dcm-vscode-extension). If you use IntelliJ IDEA or Android Studio, you can find the plugin [here](https://plugins.jetbrains.com/plugin/20856-dcm). 4. Reload the IDE. ->Note: DCM issues can be distinguished from the Dart analyzer issues by their name: DCM rule names contain `-`. Some of the issues can be fixed via CLI, to do so, run `dcm fix` for any directory. To apply `dcm fix` on a file save in the IDE, refer to [this guide](https://dcm.dev/docs/teams/ide-integrations/vscode/#extension-capabilities). +>Note: DCM issues can be distinguished from the Dart analyzer issues by their name: DCM rule names contain +dashes `-` instead of underscores `_`. Some of the issues can be fixed via CLI; to do so, run `dcm fix` for +any directory. To apply `dcm fix` on a file save in the IDE, refer to +[this guide](https://dcm.dev/docs/teams/ide-integrations/vscode/#extension-capabilities). ### third_party dependencies All content not authored by the Flutter team (which includes both sponsored and open-source contributors) -must go in the `third_party` directory. As an expedient to make the `third_party` code work well with our +must go in the `third_party` directory. As an expedient to make the `third_party` code works well with our build scripts, code in `third_party` should be given a stub `pubspec.yaml` file so that you can reference -the resources from the packages directory from `packages/devtools_app/web/index.html` +the resources from the packages directory from `packages/devtools_app/web/index.html`.
diff --git a/TESTING.md b/TESTING.md index a1b4b24..5af7b26 100644 --- a/TESTING.md +++ b/TESTING.md
@@ -17,30 +17,41 @@ 4. Full integration tests - Flutter web integration tests that run DevTools as a Flutter web app and connect it to a real test app on multiple platforms (Flutter mobile, Flutter web, and Dart CLI) - -**The following instructions are for unit tests, widget tests, and partial integration tests** in DevTools. For instructions -on running and writing full integration tests, please see [integration_test/README.md](integration_test/README.md). -In general, we should first try to test cover new features and bug fixes with unit tests or widget tests -before writing new integration tests, which are slower to run and are not as easy to debug or iterate upon. +5. Benchmark tests + - Performance benchmark tests that execute DevTools user journeys and verify the rendering performance + meets expected thresholds. + - Size benchmark test that verifies the size of the built DevTools web app meets the expected threshold. ## Running DevTools tests -Before running tests, make sure your Flutter SDK matches the version that will be used on -the CI. To update your local flutter version, run: +> The following instructions are for unit tests, widget tests, and partial integration tests. +> - For instructions on running and writing **full integration tests**, please see +> [integration_test/README.md](packages/devtools_app/integration_test/README.md). In general, we should first +> try to test cover new features and bug fixes with unit tests or widget tests before writing new integration tests, +> which are slower to run and are not as easy to debug or iterate upon. +> - For instructions on running and writing **benchmark tests**, please see +> [benchmark/README.md](packages/devtools_app/benchmark/README.md). -```shell -dt update-flutter-sdk --update-on-path -``` +### Prerequisites +1. Before running tests, make sure your Flutter SDK matches the version that will be used on +the CI. + > Note: this step requires that you have followed the [set up instructions](CONTRIBUTING.md#set-up-your-devtools-environment) + in the DevTools contributing guide regarding cloning the Flutter SDK from GitHub, adding the `dt` + executable to your PATH, and running `flutter pub get` in the `tool` directory. + + To update your local flutter version, run: + ```shell + dt update-flutter-sdk --update-on-path + ``` + > Warning: this will delete any local changes in your Flutter SDK you checked out from git. -> Note: Running this command requires that you have followed the [set up instructions](CONTRIBUTING.md#set-up-your-devtools-environment) in the DevTools contributing guide regarding cloning the Flutter SDK from GitHub, adding the `dt` executable to your PATH, and running `dart pub get` in the `tool` directory. +2. You may need to re-generate the testing mocks before running the tests: -You may need to re-generate the testing mocks before running the tests: + ```shell + dt generate-code --upgrade + ``` -```shell -dt generate-code --upgrade -``` - -Now you can proceed with running DevTools tests: +### Run tests ```shell cd packages/devtools_app @@ -55,13 +66,14 @@ 1. The UI has been _intentionally_ modified. 2. Something changed in the Flutter framework that would cause downstream changes for our tests. -3. The UI has been _unintentionally_ modified, in which case we should not accept the changes. +3. The UI has been _unintentionally_ modified, in which case we should **not** accept the changes. For valid golden image updates (1 and 2 above), the failing golden images will need to be updated. This can be done in one of two ways: -1. If the tests failed on the CI for a PR, we can download the generated golden images directly from GitHub. +1. **(Recommended)** If the tests failed on the CI for a PR, we can download the generated golden images directly from GitHub. > If you are developing on a non-MacOS machine, this is the only way you'll be able to update the golden images. + - Navigate to the failed Actions run for your PR on GitHub. Example:  @@ -70,14 +82,17 @@  + - Copy this command and run it locally to apply the golden updates. Please review these updates and ensure + they are acceptable golden changes. **Important:** this command will only succeed after the golden artifacts + have been uploaded to Github, which happens once all of the jobs have finished. + 2. Update the goldens locally by running the failing test(s) with the `--update-goldens` flag. + > Due to slight differences in the MacOS environment used by Github actions and your local + machine, the locally generated goldens may still cause failures on the CI. This is why option + #1 above is the recommended method for updating goldens images. - Before updating the goldens, ensure your version of Flutter matches the version of Flutter that is used - on the CI. To update your local flutter version, run: - - ```shell - dt update-flutter-sdk --update-on-path - ``` + on the CI (see [prerequsites](#prerequisites) above). - Then proceed with updating the goldens: