tree 35d57cd25f5b34b2d0a639b7f4899e48f3f53e96
parent 363da9038517e8895d69a39393c4023ba9a43ba1
author Jens Johansen <jensj@google.com> 1617798327 +0000
committer commit-bot@chromium.org <commit-bot@chromium.org> 1617798327 +0000

[CFE] Make unit_test_suites log better message when test starts to pass

In for instance https://dart-review.googlesource.com/c/sdk/+/193961/5
we had an instance of something good happening (a crashing-ish test
that started to pass) but where the author was told no (red bot) and
not given a path to make the bot happy (bad messaging).

Specifically tests failed and the best log one got was

```
To re-run this test, run:
    dart pkg/front_end/test/unit_test_suites.dart -p pkg/front_end/test/fasta/textual_outline/general/constants/js_semantics/issue45376
```

which isn't very helpful.

If running it locally a little more information is given via stdout,
but having to do that isn't great (and the little extra information
still wasn't great).

This CL updates the messaging to - hopefully - give a path to make the
bot happy. The messaging will now - directly from the log
on the bot - be

```
FAILED: pkg/front_end/test/fasta/textual_outline/general/constants/js_semantics/issue45376: Pass

To re-run this test, run:

   dart pkg/front_end/test/unit_test_suites.dart -p pkg/front_end/test/fasta/textual_outline/general/constants/js_semantics/issue45376

The test passed, but wasn't expected to. You should update the status file for this test.
There's a status entry looking something like

  general/constants/js_semantics/issue45376: FormatterCrash

which should be removed.

The status file is file:///[wherever]/sdk/pkg/front_end/testcases/textual_outline.status.
```

which
a) gives the extra information previously only available via stdout when
   running locally (the first line).
b) Gives an explanation as to why it failed
   ("The test passed, but wasn't expected to.").
c) A path to make the bot green ("You should update the status file for
   this test. [...] should be removed").

Hopefully this will make it easier to deal with in the future when
something good happens.

Change-Id: I838699abce580334c956d8b4753884fd600802ae
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/194242
Commit-Queue: Jens Johansen <jensj@google.com>
Reviewed-by: Johnni Winther <johnniwinther@google.com>