commit e2b62088712bbb64987e8922ea68cfc9b09fc6ac
author Jens Johansen <jensj@google.com> Thu Jan 28 14:16:25 2021 +0000
committer commit-bot@chromium.org <commit-bot@chromium.org> Thu Jan 28 14:16:25 2021 +0000
tree a94e5003cf0df2a834b3e925ca69be3dcb789d13
parent 019ce33409e246973a217a7f9655444bc13d2619

[CFE] Shard front-end try bots better

This CL shards the CFE try bot tests better.
* co19_2 runs was previously made faster, so using 10 shards on that is
  way too much.
* sdk_test runs was previously made faster, so using 5 shard on that is
  way too much.
* unit_tests was taking ~10 minutes and wasn't sharded. This CL shards
  it. Note that this sharding is semi-complicated because a few tests
  use git which is not usable on shard-runs. To overcome this a previous
  CL renamed the tests that use git and here the sharded runs only run
  the tests that doesn't require git. Then the "main bot" runs the tests
  that require git.
* unit tests suites was taking ~12 minutes and wasn't sharded. This CL
  shards it. Note that this sharding is semi-complicated because a few
  of the suites use git which is not usable on shard-runs. Also a single
  suite uses many more files that it is reasonable to include in the
  fileset for the shards. In both cases these are run on the "main bot"
  instead. The suit runnier was already "threaded", but simply launched
  all threads (say around 20 or something like that) at the same time.
  That's not ideal if having, say, 8 cores to work with (which is what
  the bots seem to have) - or 12 (which is what my computer has).
  Now only 'cores - 1' "threads" are run at any one time, and most
  "sub-suites" are sharded, so that if one finishes early that core
  can start another "sub-suite" - hopefully utilizing more of the
  resources avaiable on the computer running it.

In total - in my 'benchmarks' (i.e. try bot runs) - the
front-end-linux-release-x64-try bot previously rook around 35 minutes
to complete, and with these changes it takes around 15 minutes to
complete. The biggest entry in the timeline is now building dart which
takes almost half that time. It does so with fewer shards than before,
but each might be utilized for longer.

Change-Id: Ie034058d8f33aafd21bc49f2bc878484563ba01c
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/181383
Commit-Queue: Jens Johansen <jensj@google.com>
Reviewed-by: Johnni Winther <johnniwinther@google.com>